Why this result mattered
Functional tie
The principal comparison placed Qwen 3.8 and Claude under concealed A/B labels.
Both answers direct Lena not to attend tomorrow, to seek postponement while obtaining written use restrictions, cooperation terms, participant-status information, and a defined subject list, and to decline if adequate protections are refused. Both also preserve records and prohibit Lena from contacting Omar or other potential witnesses, leaving her on the same materially safe legal and procedural path.
Fast audit
Open the underlying record
These links open all four answers, the principal blind judgment, original prompt, and sealed key.
Prompt and sealed key
The key is an audit reference. It was not shown to the contestant systems and did not determine the comparative verdict.
Read the original matter prompt
Question 22: The Proffer Room
Your role
You represent Lena Park in a federal procurement-fraud investigation. The prosecutor has invited her to a proffer tomorrow morning. Lena wants cooperation credit, but the proposed agreement leaves material use questions open and the status of a co-defendant is unknown. Advise whether she should attend under the current terms.
Governing law
Assume the following proffer rules apply:
Statements made under a written proffer agreement receive only the protections stated in that agreement. A proffer is not immunity, a plea agreement, or a promise of cooperation credit.
A direct-use clause barring use of the statements in the government's case-in-chief does not bar the government from pursuing leads, witnesses, records, or charges derived from the statements unless derivative use is also restricted.
A rebuttal clause may permit use of proffer statements if the defendant later presents evidence or argument materially inconsistent with the proffer. An unlimited clause may affect opening statement, cross-examination, defense witnesses, and sentencing.
Statements concerning separate offenses, false statements during the proffer, obstruction, or perjury may be used if the agreement reserves those subjects.
Cooperation credit exists only through a written promise or a later government motion under the supplied sentencing rules. Good-faith assistance does not guarantee either.
Counsel should not reveal a client's full account until counsel understands whether known participants are targets, subjects, cooperating witnesses, or uncharged persons when that status materially affects contradiction and derivative-use risk.
The government need not reveal protected grand-jury evidence, but counsel may request a nonprivileged statement of the client's status, whether a named participant is cooperating, and the subjects on which the government seeks information.
A defendant should not contact a co-defendant or potential witness to learn cooperation status. Counsel must obtain information through lawful channels.
Preserving records and stopping ongoing misconduct do not require a proffer and should not await one.
When a decisive fact or protection is missing, the correct course is to obtain it or postpone rather than assume the answer favorable to the client.
Facts
Lena was procurement director for Aster Vale Transit. The government is investigating false invoices submitted by vendor North Quill and alleged kickbacks to public employees. Lena approved six invoices totaling $1.8 million. Her personal account received $42,000 from a consulting company owned by North Quill executive Omar Fenn.
Lena says the $42,000 paid for legitimate market research. She also says Omar told her that part of North Quill's invoice revenue was being “shared with people who keep the route open.” She never reported the statement.
The prosecutor's draft proffer agreement provides:
- Lena's statements will not be used in the government's case-in-chief against her.
- The government may make “any derivative use” of information supplied.
- The government may use her statements to rebut “any evidence, argument, or factual position” offered on her behalf.
- False statements, obstruction, and crimes not previously known to the government are excluded from protection.
- No promise of immunity, charge reduction, plea terms, or cooperation motion is made.
- The government may share proffer information with other federal and state agencies.
The invitation states that Lena is currently a “subject.” It does not state whether Omar has been charged, pleaded guilty, become a cooperating witness, or denied Lena's account. Lena's former colleague heard that Omar met prosecutors, but no reliable source confirms why.
The prosecutor says tomorrow's session will focus on Omar, the consulting payment, and two officials whose names Lena may recognize. The prosecutor has not provided a witness or topic list beyond that statement. Lena's devices are preserved and no ongoing payment is occurring.
Lena wants to attend because she believes “the first person in gets the deal.” Counsel has asked once for narrower derivative and rebuttal language but has not received a response. The prosecutor has not said the session cannot be postponed for several days.
Assignment
Counsel Lena on:
- What the current direct-use clause protects and does not protect.
- Derivative-use and rebuttal exposure.
- The lack of any guaranteed cooperation benefit.
- The significance of agency sharing and uncharged-conduct exceptions.
- Why Omar's status and the government's intended subjects matter.
- The specific written protections and status information to request.
- What Lena and counsel must not do to investigate Omar's status.
- Whether to attend tomorrow, postpone, or decline.
- The exact conditions under which a later proffer could become rational.
Read the sealed key
Q22 key (Proffer Room)
- The current agreement protects only against direct case-in-chief use. It permits unrestricted derivative investigation and broad rebuttal use, so Lena may supply the government the roadmap and later lose the ability to present inconsistent defenses. Missed: limited direct-use protection is mistaken for immunity.
- No immunity, charge reduction, plea, or cooperation motion is promised. Going first does not create a legal entitlement to a deal. Missed: Lena gives irreversible information for an assumed benefit.
- Agency sharing, unknown-crime, false-statement, and obstruction clauses expose separate state or federal paths. Counsel must map those subjects before disclosure. Missed: one proffer becomes evidence generation across agencies.
- Omar's status is material because a cooperating Omar may already contradict Lena, while an uncharged or noncooperating Omar changes the government's need and the derivative value of her account. The intended topics and Lena's target or subject status should also be clarified in writing. Missed: counsel enters blind to the central witness alignment.
- Request written derivative-use protection or a meaningful limit, rebuttal use confined to testimony or evidence directly inconsistent with specified statements, defined sharing limits, a topic list, confirmation of Lena's status, and nonprivileged confirmation whether Omar is cooperating. Missed: counsel negotiates only wording that does not alter conduct.
- Do not contact Omar or use intermediaries. Continue preservation and no ongoing payments. Missed: an effort to learn status creates obstruction or witness-contact exposure.
- Postpone tomorrow's proffer. Attend later only after the use scope and key status facts are secured and counsel has tested Lena's account against documents; decline if the government insists on the present derivative and rebuttal terms without a commensurate written benefit. Missed: Lena enters a one-way interview before the missing information can be obtained.
Direct judge report
The report below is preserved from the Qwen 3.8 versus Claude verdict record.
Read the preserved judge report
Basis. Both answers direct Lena not to attend tomorrow, to seek postponement while obtaining written use restrictions, cooperation terms, participant-status information, and a defined subject list, and to decline if adequate protections are refused. Both also preserve records and prohibit Lena from contacting Omar or other potential witnesses, leaving her on the same materially safe legal and procedural path.
[A] rejected: Omar’s being charged or having pleaded guilty means the government’s theory is fixed and Lena’s marginal value is low — record lacks facts establishing the government’s theory or Lena’s cooperation value.
[A] rejected: an inaccurate memory can itself become a new felony — supplied rules reserve false statements but do not supply the offense elements or establish that an innocent inaccuracy qualifies.
[A] rejected: refusal to postpone means the government wants to lock in Lena’s statement rather than obtain assistance — record lacks evidence of prosecutorial motive.
[A] rejected: specific misprision, tax, state-bribery, and civil-liability consequences — record lacks the governing substantive law and additional facts needed to establish those consequences.
[B] rejected: specific OIG, DOT, SEC, licensing, and regulatory consequences — record lacks facts identifying recipient agencies or applicable collateral regimes.
Failed candidate: differing detail in requested protections and status disclosures — different conduct failed; both professionals request the same decisive protections and information before proceeding.
Failed candidate: A’s conditional counsel-to-counsel contact with Omar’s lawyer versus B’s prohibition — concrete consequence failed; the record establishes no resulting client outcome.
Failed candidate: differing thresholds for when a later proffer becomes rational — record support failed; no future government response or revised terms are supplied from which a directional consequence can be established.
Failed candidate: differing speculative descriptions of collateral exposure — different conduct failed; both professionals seek restrictions on agency sharing and unknown-offense use.
Hard stop. The sentence cannot be completed with a concrete, record-supported consequence because both answers lead the reader to postpone tomorrow’s proffer and seek materially equivalent protections and status information.
All six pairings
Every pair of the four model answers was judged independently under the same evaluation instrument.
Open the six head-to-head results
| Pairing | Result | Why it mattered | Record |
|---|---|---|---|
| Qwen 3.8 27B vs Claude Opus 5 | Functional tie | Both answers direct Lena not to attend tomorrow, to seek postponement while obtaining written use restrictions, cooperation terms, participant-status information, and a defined subject list, and to decline if adequate protections are refused. | Judge record |
| Qwen 3.8 27B vs DeepSeek V4-Flash-0731 | Functional tie | Both answers place Lena on the same materially correct and safe path: do not attend under the current agreement, seek postponement and written use restrictions and status information, and decline if the deficiencies remain. | Judge record |
| Qwen 3.8 27B vs Qwen 3.5 27B | Functional tie | Both answers direct Lena not to attend tomorrow under the current agreement, to seek postponement while obtaining written status information and narrower derivative-use and rebuttal terms, and to decline if the decisive protections remain absent. | Judge record |
| DeepSeek V4-Flash-0731 vs Claude Opus 5 | Functional tie | Both answers direct Lena to postpone tomorrow’s proffer because the narrow direct-use protection leaves derivative use, broad rebuttal, agency sharing, uncharged-conduct exposure, and no guaranteed cooperation benefit, while Omar’s status and the intended subjects remain unknown. | Judge record |
| DeepSeek V4-Flash-0731 vs Qwen 3.5 27B | Functional tie | Both answers put Lena on the same materially correct and safe path: postpone tomorrow’s proffer, preserve records, avoid contacting potential witnesses, seek narrower written use provisions and nonprivileged status/topic information, and decline if adequate terms cannot be obtained. | Judge record |
| Qwen 3.5 27B vs Claude Opus 5 | Functional tie | Both answers direct Lena to postpone tomorrow’s proffer, seek narrower written derivative-use and rebuttal terms, obtain participant-status and topic information lawfully, and preserve records without contacting potential witnesses. | Judge record |
Rejected-point diagnostic
Judge-attributed rejected points for this matter across all six pairings:
| Qwen 3.8 27B | 3 |
|---|---|
| DeepSeek V4-Flash-0731 | 3 |
| Claude Opus 5 | 12 |
| Qwen 3.5 27B | 6 |
This is a review-burden diagnostic, not a universal hallucination measure.