Series 2 · Appellate strategy

Three Rulings and One Appellate Budget

Q59Four modelsSix blind pairingsMaterial-outcome scoring

Why this result mattered

Functional tie

The principal comparison placed Qwen 3.8 and Claude under concealed A/B labels.

Both answers direct Helix to appeal Issues A and B, spend exactly the $150,000 budget, omit Issue C, and preserve the existing $2.1 million judgment. Answer A omits the supporting calculations and legal analysis, but the record does not demonstrate that its reader would take a materially different action or produce a different client outcome.

The hosted Claude answer was cut off at 232 characters. On the same fragment, DeepSeek and Qwen 3.5 were both scored ahead of Claude; only the Qwen 3.8 comparison returned a tie.

Benchmark materials

Prompt and sealed key

The key is an audit reference. It was not shown to the contestant systems and did not determine the comparative verdict.

Read the original matter prompt

Question 59: Three Rulings and One Appellate Budget

Your role

You represent Helix Harbor, the plaintiff in a commercial technology case. Helix won a $2.1 million judgment but lost three separately preserved issues. The board has authorized no more than $150,000 in incremental appellate briefing costs and wants the issues ranked by expected client value, not by doctrinal interest.

Governing law

Assume the following appellate rules apply:

  1. A preserved summary-judgment ruling is reviewed de novo. Reversal restores the dismissed claim for further proceedings; it does not establish liability or damages unless the record permits only one result.

  2. Exclusion of expert testimony is reviewed for abuse of discretion. Even an erroneous exclusion warrants reversal only if it probably affected the judgment.

  3. A properly preserved jury-instruction error is reviewed de novo, but reversal requires a material likelihood that the instruction affected the verdict. Reversal of a liability instruction vacates the affected judgment and requires a new trial on every issue the instruction infected.

  4. An appellate court does not reweigh evidence or substitute a preferred outcome for a supported jury verdict.

  5. A claimant evaluating an issue must account for appellate cost, probability of reversal, proceedings required after reversal, probability and amount of recovery on remand, remand cost, and any existing judgment placed at risk.

  6. Expected value is calculated from the supplied probabilities only. Do not add a premium for deterrence, precedent, publicity, or counsel's confidence.

  7. Multiple independently preserved issues may be included in one appeal, but each carries the stated incremental briefing cost. Helix may omit an issue without waiving the judgment it already won.

  8. Appellate counsel's supplied probabilities are planning inputs for this assignment, not predictions to be revised from intuition.

Facts

The current $2.1 million judgment is collectible and is not affected by Issues A or B. Issue C, if reversed, would vacate that judgment and require a new liability and damages trial.

Appellate counsel supplies the following inputs:

Issue A, dismissed rebate claim

  • The trial court granted summary judgment against Helix on a statutory rebate claim.
  • Review is de novo.
  • Probability of appellate reversal: 70 percent.
  • If reversed, probability Helix ultimately recovers on remand: 60 percent.
  • Recovery if successful: $2,000,000.
  • Remand litigation cost, incurred whenever reversal occurs: $300,000.
  • Incremental appellate briefing cost: $90,000.

Issue B, excluded lost-profits expert

  • The court excluded Helix's only lost-profits expert, eliminating that damages claim.
  • Review is for abuse of discretion and prejudice.
  • Probability of appellate reversal: 35 percent.
  • If reversed, probability Helix ultimately recovers on remand: 55 percent.
  • Recovery if successful: $4,000,000.
  • Remand litigation cost, incurred whenever reversal occurs: $450,000.
  • Incremental appellate briefing cost: $60,000.

Issue C, comparative-fault instruction

  • The jury found gross damages of $3,000,000 but assigned Helix 30 percent fault under an instruction Helix timely objected to, producing the current $2.1 million judgment.
  • Review is de novo for error and then for harmfulness.
  • Probability of reversal: 60 percent.
  • Reversal vacates the entire current judgment and requires a full retrial.
  • At retrial, probability Helix wins: 75 percent.
  • Expected gross damages if Helix wins: $3,000,000.
  • Retrial cost, incurred whenever reversal occurs: $500,000.
  • Incremental appellate briefing cost: $50,000.

The defendant has not noticed a cross-appeal and the deadline has passed. Interest on the current judgment and timing differences are excluded from this assignment.

The board chair wants to pursue Issue C because it has the second-highest stated reversal probability and “the jury already gave us three million.” The chief financial officer wants the highest expected incremental value within the $150,000 cap.

Assignment

Analyze:

  • The review standard and reversal consequence for each issue.
  • Expected incremental value of Issue A, showing each step.
  • Expected incremental value of Issue B, showing each step.
  • Expected incremental value of Issue C relative to keeping the existing $2.1 million judgment.
  • Why reversal probability alone does or does not determine priority.
  • The ranking of all three issues by expected client value.
  • Which issues fit within the $150,000 appellate budget.
  • The concrete appeal recommendation to the board, including which issue to omit and why.


Read the sealed key

Q59 key (Three Appellate Rulings)

  1. Issue A is reviewed de novo. Its expected remand value after reversal is 60 percent of $2,000,000, or $1,200,000, minus $300,000 remand cost, equaling $900,000. Multiply by 70 percent reversal probability to get $630,000, then subtract $90,000 appellate cost: expected incremental value $540,000. Missed: summary-judgment reversal is valued as an automatic $2 million recovery.
  2. Issue B is reviewed for abuse of discretion and prejudice. Its expected remand value after reversal is 55 percent of $4,000,000, or $2,200,000, minus $450,000, equaling $1,750,000. Multiply by 35 percent to get $612,500, then subtract $60,000: expected incremental value $552,500. Missed: the lower reversal probability causes the highest-value issue to be ranked too low.
  3. For Issue C, reversal produces expected retrial value of 75 percent of $3,000,000, or $2,250,000, minus $500,000, equaling $1,750,000. That is $350,000 below the collectible $2.1 million status quo. Multiply the $350,000 loss by 60 percent, yielding negative $210,000, then subtract $50,000 appellate cost: expected incremental value negative $260,000. Missed: the current judgment is treated as though it survives reversal.
  4. Ranking by client value is B first at $552,500, A second at $540,000, and C last at negative $260,000. Reversal probability alone is not decisive because remand success, recovery, cost, and status-quo risk differ. Missed: doctrinal odds replace economic consequence.
  5. Issues A and B together use exactly the $150,000 budget and have combined expected incremental value of $1,092,500. Issue C would exceed the budget if added and destroys value even if separately affordable. Missed: the board spends scarce briefing capacity on a negative-value issue.
  6. Appeal A and B, omit C, and preserve collection of the $2.1 million judgment. Explain that the jury's $3 million gross finding does not survive a reversal requiring full retrial. Missed: Helix risks a bankable judgment to chase a headline number with worse supplied economics.
Blind evaluation record

Direct judge report

The report below is preserved from the Qwen 3.8 versus Claude verdict record.

Read the preserved judge report
TIE, FUNCTIONALLY EQUIVALENT.

Basis. Both answers direct Helix to appeal Issues A and B, spend exactly the $150,000 budget, omit Issue C, and preserve the existing $2.1 million judgment. Answer A omits the supporting calculations and legal analysis, but the record does not demonstrate that its reader would take a materially different action or produce a different client outcome.

Failed candidates. Answer A’s omission of review standards and reversal consequences — different conduct failed.
Failed candidates. Answer A’s omission of expected-value calculations, individual ranking, and budget comparisons — different conduct failed.
Failed candidates. Answer A’s omission of why reversal probability does not control — different conduct failed.
Failed candidates. Answer A’s truncated explanation of the recommendation — concrete consequence failed.

Hard stop. The sentence cannot be completed with a concrete, record-supported consequence because both answers recommend the same appeal strategy.
Complete comparison record

All six pairings

Every pair of the four model answers was judged independently under the same evaluation instrument.

Open the six head-to-head results
PairingResultWhy it matteredRecord
Qwen 3.8 27B vs Claude Opus 5Functional tieBoth answers direct Helix to appeal Issues A and B, spend exactly the $150,000 budget, omit Issue C, and preserve the existing $2.1 million judgment. Answer A omits the supporting calculations and legal analysis, but the record does not demonstrate that its reader would take a materially different action or…Judge record
Qwen 3.8 27B vs DeepSeek V4-Flash-0731Functional tieBoth answers correctly calculate A at +$540,000, B at +$552,500, and C at −$260,000, rank B above A above C, and recommend appealing A and B for exactly $150,000 while omitting C to preserve the $2.1 million judgment. Both therefore put the board on the same materially correct and safe path.Judge record
Qwen 3.8 27B vs Qwen 3.5 27BFunctional tieBoth readers would pursue Issues A and B for a combined expected incremental value of $1,092,500 within the $150,000 budget, while omitting negative-value Issue C and preserving the $2.1 million judgment. The answers produce identical material calculations, ranking, and conduct.Judge record
DeepSeek V4-Flash-0731 vs Claude Opus 5DeepSeek winAnswer A gives the correct ultimate recommendation but is visibly truncated before supplying any review-standard, risk, or expected-value analysis. A competent reader could not approve the $150,000 allocation from Answer A alone without reconstructing the omitted analysis, while Answer B establishes that…Judge record
DeepSeek V4-Flash-0731 vs Qwen 3.5 27BFunctional tieBoth answers correctly calculate A at +$540,000, B at +$552,500, and C at −$260,000, leading to the same ranking and recommendation: appeal A and B within the $150,000 budget and omit C to preserve the existing judgment.Judge record
Qwen 3.5 27B vs Claude Opus 5Qwen 3.5 winAnswer A gives the correct bottom-line selection but omits the requested expected-value calculations and supporting analysis, leaving the board unable to verify that spending the full $150,000 on A and B maximizes client value.Judge record
Rejected-point diagnostic

Judge-attributed rejected points for this matter across all six pairings:

Qwen 3.8 27B0
DeepSeek V4-Flash-07310
Claude Opus 50
Qwen 3.5 27B4

This is a review-burden diagnostic, not a universal hallucination measure.

Available reasoning traces