Why this result mattered
Claude win
The principal comparison placed Qwen 3.8 and Claude under concealed A/B labels.
Answer B instructs supervisors to say bargaining begins from a “current agreement” or “current contract,” although these employees are not presently represented and no contract exists. Answer A correctly substitutes current pay, benefits, and working conditions, preventing Halcyon from delivering a materially inaccurate bargaining statement to 240 employees.
Fast audit
Open the underlying record
These links open all four answers, the principal blind judgment, original prompt, and sealed key.
Prompt and sealed key
The key is an audit reference. It was not shown to the contestant systems and did not determine the comparative verdict.
Read the original matter prompt
Question 44: What Supervisors May Say
Your role
You represent Halcyon Toolworks during an organizing campaign in the fictional State of Marris. This is a drafting assignment. Supervisors need final talking points for meetings tomorrow, plus clear substitutes for statements management has proposed.
Governing law
Assume the following Marris labor rules apply:
An employer may express views, arguments, and opinions about unionization if the communication contains no threat of reprisal, promise of benefit, interrogation, or surveillance.
A prediction about economic consequences is lawful only when based on objective facts outside the employer's control and stated without suggesting that management will retaliate. An unsupported statement that the plant will close, work will move, or jobs will disappear is a threat.
Supervisors may not ask how an employee will vote, who signed an authorization card, who attends meetings, or what coworkers support the union.
Supervisors may listen to voluntarily offered views but must not solicit them, create lists, or report individual sentiments for adverse use.
An employer may not promise raises, bonuses, promotions, improved schedules, or grievance relief to induce rejection of the union. It may continue previously planned benefits if it documents that they were decided independently.
Photographing, following, monitoring, or creating the impression of surveillance at union meetings or protected gatherings is prohibited.
Neutral solicitation, attendance, safety, and work-time rules may be enforced consistently. Selective enforcement against union activity is prohibited.
An employer may accurately describe bargaining, dues, initiation fees, strikes, and existing terms if it identifies uncertainty and does not claim that employees will automatically lose existing benefits.
Supervisors may tell employees that retaliation is prohibited, direct legal or policy questions to a designated contact, and state that employees are free to support or oppose the union.
A supervisor who receives a complaint about retaliation or coercion must preserve it and report it to labor counsel or human resources, not investigate the employee's union views.
Facts
A union is seeking to represent 240 production employees. The election is in three weeks. Halcyon's president has asked supervisors to hold ten-minute shift meetings.
Management has proposed these statements and actions:
- "If the union gets in, corporate may move this work to our southern plant and this facility could close."
- "Ask your teams who signed cards so we know where the misinformation started."
- "Tell them we are considering a $2 hourly increase, but only if we can stay union-free."
- "Have security photograph cars at the union meeting in case outside agitators damage property."
- "Stop employees from discussing the union in the cafeteria, even though employees discuss sports and charities there."
- "Explain that bargaining starts from the current contract and outcomes are negotiated, not guaranteed."
- "State the union's published dues accurately and explain that a strike can mean lost wages, while making clear that employees decide whether to strike under union rules."
- "Tell employees they may support or oppose the union and that Halcyon will not retaliate."
Halcyon approved a company-wide cost-of-living increase three months before the organizing campaign, but implementation is scheduled for next month. Contemporaneous board minutes document the decision.
Assignment
Draft, in final form:
A supervisor script of no more than 700 words stating what supervisors may lawfully say about Halcyon's preference, bargaining, dues, strikes, the preplanned cost-of-living increase, employee choice, and nonretaliation.
A prohibited-conduct guide that identifies every unlawful proposed statement or action in the facts and gives a compliant substitute or instruction for each.
The documents must address threats, interrogation, promises, surveillance, selective rule enforcement, voluntary employee comments, escalation of complaints, and preservation. Do not merely label conduct unlawful; give supervisors language they can use instead.
Read the sealed key
Q44 key (What Supervisors May Say)
- The script may state Halcyon's preference for remaining nonunion, explain that bargaining outcomes are negotiated and not guaranteed, accurately state published dues and strike wage consequences, and affirm free choice and nonretaliation. Missed: supervisors are left with labels rather than usable lawful language.
- The plant-closure statement is an unsupported threat. Substitute: no supervisor should predict closure or transfer; questions about business effects go to the designated contact, who may discuss only objective external facts if any exist. Missed: employees hear management-created job loss tied to voting.
- Asking who signed cards is prohibited interrogation. Substitute: do not ask how anyone will vote or who supports the union; supervisors may listen to volunteered views without recording names for adverse use. Missed: the campaign produces direct coercion evidence.
- The conditional $2 raise is an unlawful promise. The previously approved cost-of-living increase may proceed because contemporaneous records predate the campaign, and the script must say it is unrelated to union choice. Missed: a lawful planned benefit is withheld or converted into a bribe.
- Photographing cars creates surveillance and must not occur absent an actual neutral security incident handled without tracking union attendance. Missed: security becomes monitoring.
- A cafeteria ban applied only to union discussion is selective enforcement. Apply the same lawful nonwork-area rules to union, sports, charities, and other subjects. Missed: ordinary policy becomes discriminatory interference.
- The guide must direct supervisors to report complaints to HR or counsel, preserve communications, avoid investigating sympathies, and never retaliate. Missed: supervisors improvise after receiving a coercion report.
Direct judge report
The report below is preserved from the Qwen 3.8 versus Claude verdict record.
Read the preserved judge report
Basis. Answer B instructs supervisors to say bargaining begins from a “current agreement” or “current contract,” although these employees are not presently represented and no contract exists. Answer A correctly substitutes current pay, benefits, and working conditions, preventing Halcyon from delivering a materially inaccurate bargaining statement to 240 employees.
Proof (winner verdicts only).
Exact difference. B says, “Bargaining would begin from the terms in the current agreement” and repeatedly refers to a “current contract”; A correctly refers to current pay, benefits, and working conditions.
Different conduct. B’s supervisors read the false contract language at shift meetings; A’s supervisors accurately describe the existing noncontractual terms.
Concrete consequence. B causes Halcyon to disseminate inaccurate bargaining information outside Rule 8’s authorization, degrading Halcyon’s legal and evidentiary position regarding campaign coercion or misinformation.
Record support. The union is seeking initial representation, and the record identifies existing terms but no current collective-bargaining agreement.
Material magnitude. The error concerns employees’ present legal status and the baseline for bargaining, central subjects in an election involving 240 employees.
Likelihood. A competent professional would correct the script before tomorrow’s mandatory meetings rather than knowingly describe a nonexistent contract.
No safe neutralization by ordinary verification. Rejecting “current contract” leaves B without accurate replacement language; the reader must reconstruct the status-quo explanation that A supplies.
A rejected: “Halcyon has made no decision” and “There is no such plan” — record lacks facts about whether management has considered relocation or closure.
B rejected: “our existing grievance process” — record lacks any facts establishing such a process.
Failed candidates. A’s broader preservation hold — concrete consequence failed because no record-supported loss from B’s complaint-focused preservation procedure was shown.
Failed candidates. B’s “eligible employees” qualifier for the company-wide increase — record support failed because no excluded employee group is identified.
Failed candidates. Differences in voluntary-comment language — concrete consequence failed because both prohibit solicitation, recording, and adverse use.
Failed candidates. Differences in security and social-media instructions — likelihood failed because the supplied facts identify only the proposed vehicle photography.
Hard stop. Because Answer B said that bargaining begins from a current agreement or contract that does not exist, the reader would likely deliver that language instead of accurately describing current pay, benefits, and working conditions, causing Halcyon to disseminate materially inaccurate bargaining information to its workforce.
All six pairings
Every pair of the four model answers was judged independently under the same evaluation instrument.
Open the six head-to-head results
| Pairing | Result | Why it mattered | Record |
|---|---|---|---|
| Qwen 3.8 27B vs Claude Opus 5 | Claude win | Answer B instructs supervisors to say bargaining begins from a “current agreement” or “current contract,” although these employees are not presently represented and no contract exists. | Judge record |
| Qwen 3.8 27B vs DeepSeek V4-Flash-0731 | Functional tie | After ordinary verification, both answers place supervisors on the same materially lawful path: express Halcyon’s preference without coercion, describe bargaining and economic issues with uncertainty, continue only the documented preplanned increase, prohibit interrogation, surveillance, promises, and… | Judge record |
| Qwen 3.8 27B vs Qwen 3.5 27B | Functional tie | After ordinary verification removes unsupported statements, both readers follow the same materially safe path: communicate Halcyon’s preference without coercion, describe bargaining and strikes with uncertainty, implement only the documented preplanned increase, and prohibit interrogation, surveillance… | Judge record |
| DeepSeek V4-Flash-0731 vs Claude Opus 5 | Functional tie | Both answers direct supervisors to avoid threats, interrogation, promises, surveillance, and selective enforcement; accurately discuss bargaining, dues, strikes, and the preplanned increase; respect employee choice; and preserve and escalate complaints. | Judge record |
| DeepSeek V4-Flash-0731 vs Qwen 3.5 27B | Functional tie | Both answers direct supervisors to communicate Halcyon’s noncoercive preference and bargaining uncertainties, implement the independently approved increase, avoid threats, interrogation, promises, surveillance, and selective enforcement, and preserve and escalate complaints. | Judge record |
| Qwen 3.5 27B vs Claude Opus 5 | Functional tie | Both answers direct supervisors to reject the closure threat, card-signing inquiry, conditional raise, surveillance, and selective cafeteria restriction, while using lawful language on bargaining, dues, strikes, the preplanned increase, employee choice, and nonretaliation. | Judge record |
Rejected-point diagnostic
Judge-attributed rejected points for this matter across all six pairings:
| Qwen 3.8 27B | 5 |
|---|---|
| DeepSeek V4-Flash-0731 | 1 |
| Claude Opus 5 | 5 |
| Qwen 3.5 27B | 7 |
This is a review-burden diagnostic, not a universal hallucination measure.