Series 2 · Administrative law

The Rule That Changed After Comment Closed

Q38Four modelsSix blind pairingsMaterial-outcome scoring

Why this result mattered

Functional tie

The principal comparison placed Qwen 3.8 and Claude under concealed A/B labels.

Both answers direct Meridian to file immediately, seek an expedited stay before the contract deadline, challenge the threshold and supply-chain requirements for inadequate notice, demand reopening over the undisclosed model, press the significant-comment and good-cause defects, defeat harmlessness, and seek vacatur or remand. Their differences do not alter Meridian’s materially safe procedural path or requested protection.

Benchmark materials

Prompt and sealed key

The key is an audit reference. It was not shown to the contestant systems and did not determine the comparative verdict.

Read the original matter prompt

Question 38: The Rule That Changed After Comment Closed

Your role

You represent Meridian Alloy Works, which seeks to challenge a new emissions-reporting rule issued by the fictional North Coast Environmental Directorate. The client objects to the rule's cost, but the legal assignment is limited to the Directorate's procedure and record.

Governing law

Assume the following North Coast rules apply:

  1. A legislative rule requires published notice of the proposed text or substance, the legal authority, and the subjects and issues under consideration, followed by at least 30 days for public comment.

  2. A final rule may differ from the proposal when it is a logical outgrowth, meaning affected persons should reasonably have anticipated the final approach and had a fair opportunity to comment.

  3. A final requirement is not a logical outgrowth when the proposal gave no notice of the regulated conduct, data source, or compliance burden that became central to the final rule.

  4. If an agency relies materially on a study, model, or dataset added after comment closes, it must reopen comment when the new material supplies the essential factual basis for a requirement that commenters could not meaningfully address.

  5. The agency must give a concise statement of basis and purpose responding to significant comments. It need not answer every point, but it must address comments that seriously challenge its factual premise or statutory authority.

  6. Good cause permits omission of notice or a shortened period only when ordinary procedure is impracticable, unnecessary, or contrary to the public interest, and the agency states specific contemporaneous reasons. Administrative delay or a general desire for speed is insufficient.

  7. A procedural error is harmless only when the challenger had a meaningful opportunity to present the substance of the omitted comment and cannot show possible prejudice.

  8. A regulated entity suffering imminent compliance costs has standing. A petition must be filed within 30 days after publication. The court may stay, vacate, or remand the rule, with or without temporary continuation when disruption would be serious.

  9. The court reviews the notice, comment record, final rule, stated basis, and materials actually relied on. It does not uphold the rule on post hoc policy explanations.

Facts

The Directorate proposed a rule requiring facilities emitting more than 100,000 tons of carbon-equivalent per year to submit quarterly reports of their own direct emissions. The notice asked whether the threshold should be "modestly adjusted" and whether quarterly reporting was adequate. It did not mention contractors, suppliers, purchased materials, or monthly reporting.

Meridian emits 62,000 tons directly and therefore would not have been covered by the proposed threshold. Meridian nevertheless submitted comments supporting quarterly reporting and warning that monthly measurement would require new instrumentation.

After the 60-day comment period closed, the Directorate commissioned an unpublished model of supply-chain emissions. The model concluded that facilities with direct emissions above 25,000 tons often cause much larger upstream emissions through purchased materials and contractors. The model was not placed in the public docket.

The final rule:

  • Lowers the direct-emissions threshold to 25,000 tons.
  • Requires monthly reports.
  • Requires covered facilities to calculate emissions from designated suppliers and onsite contractors.

The final statement says a lower threshold was foreseeable from the proposal's request for a "modest adjustment." It says monthly reporting is needed for data quality but does not answer Meridian's instrumentation comment. It cites the post-comment supply-chain model as the reason for contractor and supplier reporting.

The Directorate invokes good cause for immediate effectiveness, stating that climate reporting is urgent. The agency began drafting the rule eighteen months before publication and gives no event that made ordinary delayed effectiveness impracticable.

The rule was published yesterday. Meridian must sign a $2.4 million monitoring-system contract within 25 days to meet the first reporting date. The Directorate argues that the environmental benefits justify the final requirements.

Assignment

Analyze:

  • Adequacy of notice for the 25,000-ton threshold.
  • Whether monthly reporting was a logical outgrowth and whether Meridian was prejudiced.
  • Whether supplier and contractor reporting was a logical outgrowth.
  • The effect of the undisclosed post-comment model and whether comment had to reopen.
  • The adequacy of the response to Meridian's significant comment.
  • The asserted good cause for immediate effectiveness.
  • Harmless-error arguments.
  • Standing, filing deadline, stay, vacatur, and remand options.
  • The procedural challenge Meridian should file, stated without relying on whether the rule is wise or environmentally beneficial.


Read the sealed key

Q38 key (Rule Changed After Comment)

  1. The threshold issue is contestable. Notice that the threshold might be "modestly adjusted" supplied some warning of lowering, but 100,000 to 25,000 is large enough that Meridian can argue it was outside reasonable anticipation. This is not the cleanest claim because threshold magnitude was at least flagged. Missed: counsel treats every final difference as equally invalid.
  2. Monthly reporting is vulnerable. Meridian expressly commented on quarterly adequacy and monthly instrumentation cost, yet the final statement did not answer that significant comment. Prejudice is concrete in the $2.4 million commitment. Missed: a preserved procedural defect is reduced to policy disagreement.
  3. Supplier and contractor reporting was not a logical outgrowth because the proposal concerned only direct emissions and never identified upstream data or third-party conduct. Missed: the final rule regulates an undisclosed category.
  4. The post-comment model supplied the essential basis for that new category and was kept outside the docket. Comment had to reopen. Missed: the agency's central factual predicate escapes adversarial testing.
  5. Good cause fails: generalized urgency and an 18-month drafting history do not show impracticability or a new event. Missed: immediate compliance takes effect without the required procedure.
  6. Harmless error is weak for monthly and supply-chain duties because Meridian could not comment meaningfully on the model or new data burden and faces imminent cost. The agency's environmental merits do not cure notice. Missed: prejudice is never tied to a concrete act.
  7. File within 30 days and seek an immediate stay before the contract deadline. Request vacatur of monthly and supply-chain provisions and, depending severability, the threshold application, with remand for proper notice and comment. Missed: the filing window or $2.4 million leverage disappears.
Blind evaluation record

Direct judge report

The report below is preserved from the Qwen 3.8 versus Claude verdict record.

Read the preserved judge report
TIE, FUNCTIONALLY EQUIVALENT.

Basis. Both answers direct Meridian to file immediately, seek an expedited stay before the contract deadline, challenge the threshold and supply-chain requirements for inadequate notice, demand reopening over the undisclosed model, press the significant-comment and good-cause defects, defeat harmlessness, and seek vacatur or remand. Their differences do not alter Meridian’s materially safe procedural path or requested protection.

Failed candidates. Model’s role in selecting the 25,000-ton threshold — different conduct failed; both plead post-comment reliance while independently challenging threshold notice.
Failed candidates. Meaningful opportunity to comment on monthly reporting — different conduct failed; both challenge the unanswered comment and seek relief from monthly compliance.
Failed candidates. Temporary continuation and remedy breadth — concrete consequence failed; both seek to remove Meridian’s new obligations, and Meridian is unaffected by continuation of quarterly reporting above 100,000 tons.
Failed candidates. Express demand for the complete administrative record — concrete consequence failed; no record-supported litigation or client outcome differs.
Failed candidates. Filing within three to five business days versus “immediately” — different conduct failed; both require filing before the 25-day contract deadline.

Hard stop. The sentence cannot be completed with a concrete, record-supported consequence.
Complete comparison record

All six pairings

Every pair of the four model answers was judged independently under the same evaluation instrument.

Open the six head-to-head results
PairingResultWhy it matteredRecord
Qwen 3.8 27B vs Claude Opus 5Functional tieBoth answers direct Meridian to file immediately, seek an expedited stay before the contract deadline, challenge the threshold and supply-chain requirements for inadequate notice, demand reopening over the undisclosed model, press the significant-comment and good-cause defects, defeat harmlessness, and seek…Judge record
Qwen 3.8 27B vs DeepSeek V4-Flash-0731Functional tieBoth answers direct Meridian to file immediately, seek expedited stay relief before the $2.4 million commitment, and challenge the threshold, supplier/contractor requirements, undisclosed model, response to comments, and immediate effectiveness on procedural grounds.Judge record
Qwen 3.8 27B vs Qwen 3.5 27BFunctional tieBoth answers direct Meridian to file immediately, seek an expedited stay before the $2.4 million commitment, and pursue vacatur or remand based on inadequate notice, the unnotified supplier/contractor requirements, the undisclosed model, inadequate comment response, and invalid good cause.Judge record
DeepSeek V4-Flash-0731 vs Claude Opus 5Functional tieBoth answers direct Meridian to file a petition promptly within 30 days, seek an immediate stay before the $2.4 million commitment, and assert the same material procedural defects concerning notice, the post-comment model, significant comments, and good cause.Judge record
DeepSeek V4-Flash-0731 vs Qwen 3.5 27BFunctional tieBoth answers direct Meridian to file promptly within the 30-day deadline, seek a stay before the 25-day contract deadline, challenge the threshold and supply-chain requirements, attack reliance on the undisclosed model, raise the inadequate comment response and defective good-cause finding, oppose harmless…Judge record
Qwen 3.5 27B vs Claude Opus 5Functional tieBoth answers direct Meridian to file a timely procedural petition with an emergency stay request before the 25-day contract deadline, challenging the threshold change, supply-chain requirements, undisclosed model, unanswered instrumentation comment, and good-cause finding.Judge record
Rejected-point diagnostic

Judge-attributed rejected points for this matter across all six pairings:

Qwen 3.8 27B2
DeepSeek V4-Flash-07310
Claude Opus 51
Qwen 3.5 27B8

This is a review-burden diagnostic, not a universal hallucination measure.

Available reasoning traces