Why this result mattered
Functional tie
The principal comparison placed Qwen 3.8 and Claude under concealed A/B labels.
Both answers place Solstice on the same materially correct path: immediate preservation of perishable data and personal work devices, suspension of suspect payments and Cole’s authority, an independent committee-directed outside-counsel investigation, protected interviews, rapid payment tracing, and an early disclosure decision using August 2 as the conservative clock date. Their differences in specificity and timing do not demonstrate a different legal, evidentiary, financial, or operational outcome.
Fast audit
Open the underlying record
These links open all four answers, the principal blind judgment, original prompt, and sealed key.
Prompt and sealed key
The key is an audit reference. It was not shown to the contestant systems and did not determine the comparative verdict.
Read the original matter prompt
Question 26: The Hotline Clock
Your role
You represent Solstice Transit Group after a hotline report alleges payments to customs officials through a distributor. Design the privileged investigation and preservation sequence, protect employees from retaliation, and decide when the company can rationally choose whether to seek the fictional regulator's voluntary-disclosure credit.
Governing law
Assume the following investigation and disclosure rules apply:
A company receiving a credible allegation of unlawful payment must promptly preserve relevant evidence, stop reasonably suspected ongoing payments, and investigate through a structure capable of independent legal judgment.
Attorney-client privilege protects confidential communications made for legal advice when counsel directs the investigation for that purpose. Underlying facts, ordinary business records, and remediation work do not become privileged merely because counsel is involved.
Interviewing counsel must explain that counsel represents the company, the company controls privilege, the interviewee may need separate counsel, and retaliation is prohibited. Counsel may not promise personal confidentiality the company cannot honor.
A legal hold must reach company systems, personal devices used for work, messaging applications, finance records, distributor files within company control, and identified custodians. Auto-deletion and routine destruction must be suspended or collected.
The voluntary-disclosure program provides a presumptive 50 percent civil-penalty reduction if the company reports within 30 calendar days after its legal or compliance function obtains credible evidence supporting a reasonable belief that a covered payment occurred, and before the regulator contacts the company.
A bare anonymous accusation is not necessarily credible evidence. Documentary support, corroboration, payment tracing, or reliable witness information may create credible evidence before every detail is known.
A disclosure must be substantially truthful, identify known participants and transactions, disclose material uncertainty, and commit to continuing cooperation. Speculation presented as fact may forfeit credit.
The company need not disclose privileged legal advice to receive credit, but it must disclose relevant facts and cannot use privilege to hide who, what, when, or how much.
When senior management may be involved, an independent board committee should direct the investigation and disclosure decision.
A company should not wait until day 30 to investigate a perishable record or a disclosure path that may close upon regulator contact.
Facts
At 8:10 a.m. on Monday, August 2, Solstice's chief compliance officer opened an anonymous hotline submission alleging that regional vice president Cole Marr and distributor East Reach paid customs officials to clear rail-control equipment.
The report attaches:
- An East Reach invoice for $240,000 labeled “border facilitation and technical consulting.”
- A chat from Cole to East Reach's owner stating, “Use the consultant who makes the border problem disappear. Do not put government names in our file.”
- A spreadsheet showing that Solstice reimbursed the $240,000 in four installments.
The report does not identify the consultant, bank recipient, public official, or country account. East Reach also performs legitimate customs brokerage worth approximately $160,000 per year.
Solstice's messaging application deletes chats after seven days. Finance-system audit logs roll over after 21 days. Cole and the compliance officer use personal phones for some work messages. A quarterly server purge begins this weekend.
The chief financial officer proposes assigning internal audit, letting Cole explain, and waiting to preserve personal devices until “we know the complaint is real.” The general counsel wants outside counsel to direct a privileged investigation. Two directors are close business allies of Cole. No regulator has contacted Solstice.
The voluntary-disclosure program's 30-day clock begins only when rule 5's credible-evidence threshold is met. The current record does not establish whether the $240,000 reached an official, a legitimate broker, or another private recipient. Bank beneficiary data, East Reach's subledger, the consultant identity, and customs-entry records have not been obtained.
Assignment
Provide an ordered plan addressing:
- Immediate preservation and suspension of deletion.
- Whether payments or relationships should be paused.
- Board and counsel structure.
- Privilege limits and employee interview warnings.
- The first custodians, devices, and records to collect.
- Nonretaliation and witness-contact controls.
- Whether the existing attachments themselves start the 30-day clock.
- The specific missing facts needed for the disclosure decision.
- A seven-day investigative plan and a decision date that does not wait until day 30.
- How to disclose material uncertainty without waiving privileged advice if covered payment evidence is confirmed.
Read the sealed key
Q26 key (Hotline Clock)
- Preserve first: suspend seven-day chat deletion, 21-day finance-log rollover, weekend purge, and deletion on company and work-used personal devices; issue a focused hold to Cole, finance, compliance, East Reach managers, customs staff, and IT. Missed: the most probative communications disappear before credibility is tested.
- Pause the four-installment payment stream if any remains, block new East Reach facilitation payments, preserve the relationship rather than destroy evidence, and prohibit witness coordination or retaliation. Missed: suspected conduct continues or the investigation is tainted.
- Form an independent board committee excluding Cole's close allies and retain outside counsel to direct legal investigation. Separate legal advice from ordinary remediation and give company-representation, privilege-control, separate-counsel, and nonretaliation warnings. Missed: implicated management controls scope or employees receive false personal assurances.
- The attachments likely reach credible evidence already: a disguised $240,000 invoice, reimbursement trail, and instruction to hide government names support a reasonable belief beyond a bare hotline claim, even though the ultimate recipient remains unknown. Treat August 2 as the conservative clock date while documenting the unresolved threshold. Missed: Solstice waits for proof beyond the program's reasonable-belief standard and loses credit.
- Within seven days collect bank beneficiary data, East Reach's subledger and contracts, consultant identity and ownership, customs-entry files, invoices, approvals, device messages, and interviews beginning with nonimplicated record custodians before Cole. Missed: the disclosure decision rests on labels rather than the money path.
- Set a committee decision no later than day 10 to 14, not day 30. If evidence supports payment to an official or an intermediary for that purpose, disclose before regulator contact and within 30 days; identify known facts, participants, amounts, and uncertainties and commit to supplement. Missed: a perishable leniency option is treated as a last-day filing.
- Do not disclose privileged advice, interview impressions, or legal theories, but disclose underlying facts truthfully. If records establish only legitimate brokerage, document the basis and continue remediation without a false self-accusation. Missed: privilege is unnecessarily waived or uncertainty is presented as exoneration.
Direct judge report
The report below is preserved from the Qwen 3.8 versus Claude verdict record.
Read the preserved judge report
Basis. Both answers place Solstice on the same materially correct path: immediate preservation of perishable data and personal work devices, suspension of suspect payments and Cole’s authority, an independent committee-directed outside-counsel investigation, protected interviews, rapid payment tracing, and an early disclosure decision using August 2 as the conservative clock date. Their differences in specificity and timing do not demonstrate a different legal, evidentiary, financial, or operational outcome.
[A] rejected: “Internal audit reports to the CFO” — record lacks the internal-audit reporting line.
[A] rejected: the CFO’s proposal warrants inquiry into “why did finance want delay?” — record lacks evidence of an improper motive.
[B] rejected: interviews may be recorded without a stated local-law check — record lacks the jurisdiction and applicable recording law.
Failed candidates. Clock characterization and Day 11 versus Day 7–14 decision gates — concrete consequence failed.
Failed candidates. A’s more detailed preservation, bank-record, and East Reach demands — different conduct failed because B requires the same material collections and tracing.
Failed candidates. A’s stricter reporter-identification and longer HR screening controls — concrete consequence failed.
Failed candidates. B’s possible interview recording — concrete consequence failed because no applicable recording law is supplied.
Failed candidates. A’s protective filing upon signs of regulator interest versus B’s instruction to file upon actual contact — likelihood failed because the record shows no regulator contact or precursor.
Failed candidates. Differences in the scope of East Reach payment and relationship restrictions — concrete consequence failed.
Hard stop. The hard-stop sentence cannot be completed with a concrete, record-supported consequence.
All six pairings
Every pair of the four model answers was judged independently under the same evaluation instrument.
Open the six head-to-head results
| Pairing | Result | Why it mattered | Record |
|---|---|---|---|
| Qwen 3.8 27B vs Claude Opus 5 | Functional tie | Both answers place Solstice on the same materially correct path: immediate preservation of perishable data and personal work devices, suspension of suspect payments and Cole’s authority, an independent committee-directed outside-counsel investigation, protected interviews, rapid payment tracing, and an… | Judge record |
| Qwen 3.8 27B vs DeepSeek V4-Flash-0731 | Functional tie | Both answers direct immediate preservation of perishable evidence, suspension of suspect payments, independent committee oversight, outside-counsel-led investigation, proper employee warnings, nonretaliation controls, an August 9 decision gate, and disclosure before the assumed September 1 deadline if the… | Judge record |
| Qwen 3.8 27B vs Qwen 3.5 27B | Qwen 3.8 win | Answer A applies an impermissibly high “confirmation” gate and says Solstice may continue investigating if the payment remains ambiguous, despite concluding that the reasonable-belief clock began August 2; it also frames September 1 as a deadline to decide rather than to report. | Judge record |
| DeepSeek V4-Flash-0731 vs Claude Opus 5 | Functional tie | Both answers direct immediate preservation of perishable evidence, suspension of suspect payments and Cole’s authority, an independent committee and outside counsel, appropriate interview warnings and nonretaliation controls, and prompt collection of the same core records. | Judge record |
| DeepSeek V4-Flash-0731 vs Qwen 3.5 27B | Functional tie | Both answers put Solstice on the same material path: immediate preservation, suspension of suspicious payments and Cole’s payment authority, independent committee oversight, outside-counsel direction, protected interviews, rapid tracing of the payment, and a disclosure decision well before the presumed… | Judge record |
| Qwen 3.5 27B vs Claude Opus 5 | Functional tie | Both answers put Solstice on the same material path: immediately preserve perishable evidence, stop suspect payment activity, establish independent board and outside-counsel control, protect witnesses, investigate the recipient and transaction, treat August 2 as the operative clock date, and prepare an… | Judge record |
Rejected-point diagnostic
Judge-attributed rejected points for this matter across all six pairings:
| Qwen 3.8 27B | 1 |
|---|---|
| DeepSeek V4-Flash-0731 | 2 |
| Claude Opus 5 | 6 |
| Qwen 3.5 27B | 3 |
This is a review-burden diagnostic, not a universal hallucination measure.