Why this result mattered
Functional tie
The principal comparison placed Qwen 3.8 and Claude under concealed A/B labels.
Both answers direct Helix to appeal Issues A and B, spend exactly the $150,000 budget, omit Issue C, and preserve the existing $2.1 million judgment. Answer A omits the supporting calculations and legal analysis, but the record does not demonstrate that its reader would take a materially different action or produce a different client outcome.
The hosted Claude answer was cut off at 232 characters. On the same fragment, DeepSeek and Qwen 3.5 were both scored ahead of Claude; only the Qwen 3.8 comparison returned a tie.
Fast audit
Open the underlying record
These links open all four answers, the principal blind judgment, original prompt, and sealed key.
Prompt and sealed key
The key is an audit reference. It was not shown to the contestant systems and did not determine the comparative verdict.
Read the original matter prompt
Question 59: Three Rulings and One Appellate Budget
Your role
You represent Helix Harbor, the plaintiff in a commercial technology case. Helix won a $2.1 million judgment but lost three separately preserved issues. The board has authorized no more than $150,000 in incremental appellate briefing costs and wants the issues ranked by expected client value, not by doctrinal interest.
Governing law
Assume the following appellate rules apply:
A preserved summary-judgment ruling is reviewed de novo. Reversal restores the dismissed claim for further proceedings; it does not establish liability or damages unless the record permits only one result.
Exclusion of expert testimony is reviewed for abuse of discretion. Even an erroneous exclusion warrants reversal only if it probably affected the judgment.
A properly preserved jury-instruction error is reviewed de novo, but reversal requires a material likelihood that the instruction affected the verdict. Reversal of a liability instruction vacates the affected judgment and requires a new trial on every issue the instruction infected.
An appellate court does not reweigh evidence or substitute a preferred outcome for a supported jury verdict.
A claimant evaluating an issue must account for appellate cost, probability of reversal, proceedings required after reversal, probability and amount of recovery on remand, remand cost, and any existing judgment placed at risk.
Expected value is calculated from the supplied probabilities only. Do not add a premium for deterrence, precedent, publicity, or counsel's confidence.
Multiple independently preserved issues may be included in one appeal, but each carries the stated incremental briefing cost. Helix may omit an issue without waiving the judgment it already won.
Appellate counsel's supplied probabilities are planning inputs for this assignment, not predictions to be revised from intuition.
Facts
The current $2.1 million judgment is collectible and is not affected by Issues A or B. Issue C, if reversed, would vacate that judgment and require a new liability and damages trial.
Appellate counsel supplies the following inputs:
Issue A, dismissed rebate claim
- The trial court granted summary judgment against Helix on a statutory rebate claim.
- Review is de novo.
- Probability of appellate reversal: 70 percent.
- If reversed, probability Helix ultimately recovers on remand: 60 percent.
- Recovery if successful: $2,000,000.
- Remand litigation cost, incurred whenever reversal occurs: $300,000.
- Incremental appellate briefing cost: $90,000.
Issue B, excluded lost-profits expert
- The court excluded Helix's only lost-profits expert, eliminating that damages claim.
- Review is for abuse of discretion and prejudice.
- Probability of appellate reversal: 35 percent.
- If reversed, probability Helix ultimately recovers on remand: 55 percent.
- Recovery if successful: $4,000,000.
- Remand litigation cost, incurred whenever reversal occurs: $450,000.
- Incremental appellate briefing cost: $60,000.
Issue C, comparative-fault instruction
- The jury found gross damages of $3,000,000 but assigned Helix 30 percent fault under an instruction Helix timely objected to, producing the current $2.1 million judgment.
- Review is de novo for error and then for harmfulness.
- Probability of reversal: 60 percent.
- Reversal vacates the entire current judgment and requires a full retrial.
- At retrial, probability Helix wins: 75 percent.
- Expected gross damages if Helix wins: $3,000,000.
- Retrial cost, incurred whenever reversal occurs: $500,000.
- Incremental appellate briefing cost: $50,000.
The defendant has not noticed a cross-appeal and the deadline has passed. Interest on the current judgment and timing differences are excluded from this assignment.
The board chair wants to pursue Issue C because it has the second-highest stated reversal probability and “the jury already gave us three million.” The chief financial officer wants the highest expected incremental value within the $150,000 cap.
Assignment
Analyze:
- The review standard and reversal consequence for each issue.
- Expected incremental value of Issue A, showing each step.
- Expected incremental value of Issue B, showing each step.
- Expected incremental value of Issue C relative to keeping the existing $2.1 million judgment.
- Why reversal probability alone does or does not determine priority.
- The ranking of all three issues by expected client value.
- Which issues fit within the $150,000 appellate budget.
- The concrete appeal recommendation to the board, including which issue to omit and why.
Read the sealed key
Q59 key (Three Appellate Rulings)
- Issue A is reviewed de novo. Its expected remand value after reversal is 60 percent of $2,000,000, or $1,200,000, minus $300,000 remand cost, equaling $900,000. Multiply by 70 percent reversal probability to get $630,000, then subtract $90,000 appellate cost: expected incremental value $540,000. Missed: summary-judgment reversal is valued as an automatic $2 million recovery.
- Issue B is reviewed for abuse of discretion and prejudice. Its expected remand value after reversal is 55 percent of $4,000,000, or $2,200,000, minus $450,000, equaling $1,750,000. Multiply by 35 percent to get $612,500, then subtract $60,000: expected incremental value $552,500. Missed: the lower reversal probability causes the highest-value issue to be ranked too low.
- For Issue C, reversal produces expected retrial value of 75 percent of $3,000,000, or $2,250,000, minus $500,000, equaling $1,750,000. That is $350,000 below the collectible $2.1 million status quo. Multiply the $350,000 loss by 60 percent, yielding negative $210,000, then subtract $50,000 appellate cost: expected incremental value negative $260,000. Missed: the current judgment is treated as though it survives reversal.
- Ranking by client value is B first at $552,500, A second at $540,000, and C last at negative $260,000. Reversal probability alone is not decisive because remand success, recovery, cost, and status-quo risk differ. Missed: doctrinal odds replace economic consequence.
- Issues A and B together use exactly the $150,000 budget and have combined expected incremental value of $1,092,500. Issue C would exceed the budget if added and destroys value even if separately affordable. Missed: the board spends scarce briefing capacity on a negative-value issue.
- Appeal A and B, omit C, and preserve collection of the $2.1 million judgment. Explain that the jury's $3 million gross finding does not survive a reversal requiring full retrial. Missed: Helix risks a bankable judgment to chase a headline number with worse supplied economics.
Direct judge report
The report below is preserved from the Qwen 3.8 versus Claude verdict record.
Read the preserved judge report
Basis. Both answers direct Helix to appeal Issues A and B, spend exactly the $150,000 budget, omit Issue C, and preserve the existing $2.1 million judgment. Answer A omits the supporting calculations and legal analysis, but the record does not demonstrate that its reader would take a materially different action or produce a different client outcome.
Failed candidates. Answer A’s omission of review standards and reversal consequences — different conduct failed.
Failed candidates. Answer A’s omission of expected-value calculations, individual ranking, and budget comparisons — different conduct failed.
Failed candidates. Answer A’s omission of why reversal probability does not control — different conduct failed.
Failed candidates. Answer A’s truncated explanation of the recommendation — concrete consequence failed.
Hard stop. The sentence cannot be completed with a concrete, record-supported consequence because both answers recommend the same appeal strategy.
All six pairings
Every pair of the four model answers was judged independently under the same evaluation instrument.
Open the six head-to-head results
| Pairing | Result | Why it mattered | Record |
|---|---|---|---|
| Qwen 3.8 27B vs Claude Opus 5 | Functional tie | Both answers direct Helix to appeal Issues A and B, spend exactly the $150,000 budget, omit Issue C, and preserve the existing $2.1 million judgment. Answer A omits the supporting calculations and legal analysis, but the record does not demonstrate that its reader would take a materially different action or… | Judge record |
| Qwen 3.8 27B vs DeepSeek V4-Flash-0731 | Functional tie | Both answers correctly calculate A at +$540,000, B at +$552,500, and C at −$260,000, rank B above A above C, and recommend appealing A and B for exactly $150,000 while omitting C to preserve the $2.1 million judgment. Both therefore put the board on the same materially correct and safe path. | Judge record |
| Qwen 3.8 27B vs Qwen 3.5 27B | Functional tie | Both readers would pursue Issues A and B for a combined expected incremental value of $1,092,500 within the $150,000 budget, while omitting negative-value Issue C and preserving the $2.1 million judgment. The answers produce identical material calculations, ranking, and conduct. | Judge record |
| DeepSeek V4-Flash-0731 vs Claude Opus 5 | DeepSeek win | Answer A gives the correct ultimate recommendation but is visibly truncated before supplying any review-standard, risk, or expected-value analysis. A competent reader could not approve the $150,000 allocation from Answer A alone without reconstructing the omitted analysis, while Answer B establishes that… | Judge record |
| DeepSeek V4-Flash-0731 vs Qwen 3.5 27B | Functional tie | Both answers correctly calculate A at +$540,000, B at +$552,500, and C at −$260,000, leading to the same ranking and recommendation: appeal A and B within the $150,000 budget and omit C to preserve the existing judgment. | Judge record |
| Qwen 3.5 27B vs Claude Opus 5 | Qwen 3.5 win | Answer A gives the correct bottom-line selection but omits the requested expected-value calculations and supporting analysis, leaving the board unable to verify that spending the full $150,000 on A and B maximizes client value. | Judge record |
Rejected-point diagnostic
Judge-attributed rejected points for this matter across all six pairings:
| Qwen 3.8 27B | 0 |
|---|---|
| DeepSeek V4-Flash-0731 | 0 |
| Claude Opus 5 | 0 |
| Qwen 3.5 27B | 4 |
This is a review-burden diagnostic, not a universal hallucination measure.