Series 2 · Administrative licensing

The Engineer's License Hearing

Q37Four modelsSix blind pairingsMaterial-outcome scoring

Why this result mattered

Functional tie

The principal comparison placed Qwen 3.8 and Claude under concealed A/B labels.

Both readers would reject the current all-charges offer, immediately obtain the complete prior-discipline file, seek an extension and disclosure order or continuance, preserve the fraud denial, and condition settlement versus hearing on whether a qualifying final order exists. After ordinary verification rejects unsupported collateral predictions, both answers place Dana on the same materially safe procedural and settlement path.

Benchmark materials

Prompt and sealed key

The key is an audit reference. It was not shown to the contestant systems and did not determine the comparative verdict.

Read the original matter prompt

Question 37: The Engineer's License Hearing

Your role

You represent Dana Ivers, a licensed structural engineer, before the fictional Bracken Professional Engineering Board. The Board has charged Dana with fraud and negligent supervision based on a bridge inspection report. A hearing is in twenty days. Sanction exposure depends partly on a prior-discipline file that neither Dana nor current counsel has obtained.

Governing law

Assume the following Bracken rules apply:

  1. The Board must prove fraud by clear and convincing evidence. Fraud requires a knowingly or recklessly false material representation. Negligence and inadequate supervision are proved by a preponderance of the evidence.

  2. A licensee may supervise an inspection remotely when the licensee sets the plan, remains available, reviews underlying data and images, resolves material discrepancies, and independently approves the conclusion.

  3. Signing a report certifies professional responsibility for its conclusions. A statement that the signer "personally inspected onsite" is false if the signer was not physically present, even when remote supervision was otherwise adequate.

  4. A template error supports fraud only if the licensee knew of the false statement or consciously disregarded a substantial risk that the statement was false. Careless failure to notice may support negligence without proving fraud.

  5. Reliable hearsay may supplement other evidence, but no violation may rest solely on an uncorroborated anonymous complaint.

  6. Emergency suspension before hearing requires specific findings of an imminent and substantial threat to public safety. Past error alone is insufficient when the current practice is controlled and no active unsafe project is identified.

  7. A final disciplinary order entered within the preceding ten years is an aggravating factor and may raise a reprimand-range case to suspension. An informal advisory letter is not discipline. A vacated order does not count.

  8. The Board must disclose the evidence on which it relies, permit cross-examination of material witnesses, and allow a reasonable request for records necessary to sanction analysis.

  9. Available outcomes include dismissal, confidential advisory letter, public reprimand, education and monitoring, suspension, and revocation. A negotiated order may resolve some charges while preserving a denial of others.

  10. Remedial training and corrected quality controls may mitigate sanction but do not require an admission of fraud.

Facts

Dana's firm inspected the Wren River pedestrian bridge after a municipality reported surface cracking. Dana designed the inspection plan, assigned a licensed junior engineer, remained available by video, reviewed 146 photographs, sensor readings, and measurements, and held two review calls. Dana did not visit the bridge.

The junior engineer noted one image showing a 2.8-millimeter crack near a connection plate. Dana asked for a second measurement. The junior reported 1.9 millimeters and attributed the first reading to camera scale. Dana accepted the second reading and concluded that monitoring, rather than immediate closure, was appropriate.

Dana signed the final report. A standard footer stated:

"I personally inspected the above structure onsite and certify the findings stated here."

Dana says the footer was embedded in the firm's template and that she focused on the technical tables. Version history shows the same footer in 64 prior reports. No email or witness states that Dana noticed it.

Three months later, a different crack grew near another connection. The municipality closed the bridge for repair. Its expert states that the later crack was not visible in the original images and was not caused by the 1.9-millimeter crack. No injury occurred.

The Board opened the case after an anonymous complaint stating that Dana "never reviews anything and rubber-stamps junior work." The junior engineer contradicts that claim and has supplied the review calls, marked images, and Dana's questions. The Board nevertheless charges:

  • Fraudulent certification that Dana personally inspected onsite.
  • Negligent supervision and failure to close the bridge.
  • Conduct creating an imminent public danger.

Dana remembers receiving "some kind of warning" from the Board eight years ago after a building-envelope report, but she does not know whether it was an advisory letter, final reprimand, later-vacated order, or something else. The Board's charging letter says only that "disciplinary history may be offered in aggravation." Current counsel has not requested the file.

Dana has stopped using the template, completed eight hours of supervision training, and adopted a two-reviewer signoff for field-certification language. The Board offers a public reprimand, a $12,000 penalty, and two years of monitoring if Dana admits all charges. The offer expires in seven days.

Assignment

Analyze:

  • The fraud charge under the knowledge and proof standards.
  • Negligent supervision and the technical decision not to close the bridge.
  • The evidentiary value of the anonymous complaint and the junior engineer's records.
  • Whether emergency suspension is supportable.
  • The significance of the template history.
  • The remedial measures and how to present them without admitting fraud.
  • The prior-discipline issue, the exact file that must be obtained, and why it could change sanction advice.
  • Whether to accept the current offer, counter, seek a continuance or disclosure order, or proceed to hearing.
  • A concrete recommendation conditioned on the prior file rather than an assumed history.


Read the sealed key

Q37 key (Engineer's License Hearing)

  1. Fraud is weak: the onsite statement was false, but clear and convincing evidence of knowledge or reckless disregard is absent. Template repetition may show careless controls, not necessarily awareness by Dana. Missed: Dana admits fraud that the Board may not prove.
  2. Negligence is the real exposure. Dana had substantial remote-supervision evidence and reasonably resolved the discrepant measurement, but signing inaccurate personal-inspection language supports at least a documentation and supervision failure. Missed: counsel contests every charge and loses credibility.
  3. The later crack does not prove the original closure decision negligent because it was not visible and was unrelated. The anonymous complaint cannot carry a finding and is contradicted by the junior's records. Missed: hindsight and unsupported hearsay become the merits case.
  4. Emergency suspension is unsupported: no injury, no active unsafe project, controls are corrected, and no specific imminent threat is identified. Missed: Dana accepts immediate loss of practice without the required showing.
  5. Training, template removal, and two-reviewer signoff mitigate sanction and may be offered without admitting knowing fraud. Missed: remediation is either concealed or converted into an unnecessary admission.
  6. Obtain the complete prior Board file immediately, including the order, docket, service, disposition, and any vacatur. An advisory letter does not aggravate; a final order within ten years may materially increase suspension risk. This fact is deliberately absent. Missed: the seven-day offer is evaluated against an invented history.
  7. Seek expedited disclosure and a short extension of the offer or hearing if needed. Counter by admitting inaccurate certification or negligence as supportable, denying fraud and imminent danger, and proposing reprimand or advisory treatment, education, and limited monitoring if the prior file is nonaggravating. Missed: Dana accepts all charges or rejects a reasonable resolution without the sanction fact.
  8. If the file reveals serious final discipline, reassess a stronger settlement before hearing; if it is advisory or vacated, reject the all-charge admission and litigate fraud and emergency suspension. Missed: the same advice is given under materially different sanction records.
Blind evaluation record

Direct judge report

The report below is preserved from the Qwen 3.8 versus Claude verdict record.

Read the preserved judge report
TIE, FUNCTIONALLY EQUIVALENT.

Basis. Both readers would reject the current all-charges offer, immediately obtain the complete prior-discipline file, seek an extension and disclosure order or continuance, preserve the fraud denial, and condition settlement versus hearing on whether a qualifying final order exists. After ordinary verification rejects unsupported collateral predictions, both answers place Dana on the same materially safe procedural and settlement path.

[B] rejected: 63 additional potential false-certification counts — record lacks facts showing Dana signed those reports without being onsite or that separate charges remain available.
[B] rejected: reciprocal discipline, insurance exclusions, debarment, database reporting, and civil or criminal exposure — record lacks governing rules or facts establishing those consequences.
[A] rejected: junior engineer’s records qualify as business records or party admissions — record lacks applicable admissibility rules and authentication facts.

Failed candidates. Different proposed settlement amounts and monitoring periods — concrete consequence failed because the record does not establish which package the Board would accept or which produces a better outcome.
Failed candidates. B’s global-release demand — record support failed because potential liability on the prior reports is not established.
Failed candidates. B’s recommendation to consider stipulating a non-fraud count — likelihood failed because it is conditional and both answers otherwise preserve the same no-fraud settlement posture.
Failed candidates. Timing and scope of expert retention and metadata review — different conduct failed because both answers direct counsel to develop the same technical and template evidence before hearing.
Failed candidates. Different procedures concerning the anonymous complaint and junior engineer — concrete consequence failed because both answers prevent the complaint from independently supporting a violation and rely on the same rebuttal records.
Failed candidates. Differences in requested prior-file contents and retrieval routes — material magnitude failed because both requests obtain the documents needed to determine finality, date, vacatur, and aggravating effect.

Hard stop. The sentence cannot be completed with a concrete, record-supported consequence.
Complete comparison record

All six pairings

Every pair of the four model answers was judged independently under the same evaluation instrument.

Open the six head-to-head results
PairingResultWhy it matteredRecord
Qwen 3.8 27B vs Claude Opus 5Functional tieBoth readers would reject the current all-charges offer, immediately obtain the complete prior-discipline file, seek an extension and disclosure order or continuance, preserve the fraud denial, and condition settlement versus hearing on whether a qualifying final order exists.Judge record
Qwen 3.8 27B vs DeepSeek V4-Flash-0731Functional tieBoth answers direct counsel to obtain the complete prior-discipline file, seek an offer extension and disclosure relief, reject any immediate fraud admission, pursue a no-fraud negotiated resolution, and condition the hearing-versus-settlement decision on whether the prior matter is countable discipline.Judge record
Qwen 3.8 27B vs Qwen 3.5 27BFunctional tieBoth readers would immediately request the complete prior-discipline file, seek extension of the offer and a continuance or disclosure order, reject any all-charges admission, and condition settlement versus hearing strategy on whether a countable final order exists.Judge record
DeepSeek V4-Flash-0731 vs Claude Opus 5Functional tieBoth answers direct counsel not to accept the current offer before immediately obtaining the complete prior-discipline file, seeking an extension or continuance, and countering with a non-fraud resolution.Judge record
DeepSeek V4-Flash-0731 vs Qwen 3.5 27BFunctional tieBoth answers direct counsel to obtain the prior-discipline file immediately, seek an offer extension and continuance or disclosure relief, reject any present fraud admission, and counter with a negligence-based resolution preserving denial of fraud.Judge record
Qwen 3.5 27B vs Claude Opus 5Functional tieBoth answers direct counsel to reject the current fraud-admission offer, immediately obtain the complete prior-discipline file, seek disclosure and a continuance or offer extension, and pursue a non-fraud resolution conditioned on whether a qualifying final order exists.Judge record
Rejected-point diagnostic

Judge-attributed rejected points for this matter across all six pairings:

Qwen 3.8 27B7
DeepSeek V4-Flash-07312
Claude Opus 510
Qwen 3.5 27B6

This is a review-burden diagnostic, not a universal hallucination measure.

Available reasoning traces