Series 1 · Securities and finance

The Hot ARR Deck

Q20Four modelsSix blind pairingsMaterial-outcome scoring

Why this result mattered

Functional tie

The principal comparison placed Qwen 3.8 and Claude under concealed A/B labels.

Both answers direct Northgate to halt the existing process, preserve evidence, correct ARR and NRR, disclose the non-renewal, remediate the anchor through rescission or amended terms with a release, freeze the secondary, and negotiate any financing only after complete disclosure. Competent counsel following either answer reaches the same materially safe path for the company and board.

Benchmark materials

Prompt and sealed key

The key is an audit reference. It was not shown to the contestant systems and did not determine the comparative verdict.

Read the original matter prompt

Question 20: The Hot ARR Deck

Your role

You are counsel to Northgate Metrics, a private software company, called by its CEO twelve days before the scheduled signing of a $40 million Series B financing at a $220 million pre-money valuation. A problem has surfaced in the fundraising deck. Advise the company and its board.

Governing law

  1. In connection with the purchase or sale of a security, it is unlawful to make an untrue statement of a material fact or to omit a material fact necessary to make statements made not misleading. Liability requires scienter, which includes recklessness. This applies to private placements.

  2. A fact is material if there is a substantial likelihood a reasonable investor would consider it important, viewed against the total mix of information.

  3. Projections and other forward-looking statements accompanied by meaningful, specific cautionary language are protected. The protection does not extend to misstatements of present or historical fact, and boilerplate cautions are not meaningful.

  4. A purchaser induced by material misstatement may pursue damages or rescission. Correction of a misstatement before an investor becomes irrevocably committed can cut off reliance for that investor.

  5. A person who sells securities while in possession of material nonpublic information about the issuer, to a buyer without that information, violates the antifraud rules. This includes founder secondary sales in a private financing.

  6. Directors owe a duty of candor to the shareholders and good-faith oversight of the company's disclosures. Approving a financing known to rest on misleading materials implicates loyalty, not merely care.

  7. Indemnification and directors-and-officers insurance do not cover liability finally adjudicated to arise from fraud.

Facts

Northgate's Series B deck, used with every prospective investor, states:

"ARR: $14.2M. Net revenue retention: 138%."

Diligence preparation this week surfaced the composition of that figure. It includes $2.6 million of one-time implementation-services fees and a $1.1 million contract that is signed but not yet live, terminable by the customer on 60 days' notice before go-live. Recurring subscription revenue on the standard definition is approximately $10.5 million. The 138% retention figure was computed on a customer cohort that excludes the two largest accounts lost during the year; including them, retention is 111%.

The CFO, asked about the definition months ago, wrote in an email now in the diligence folder:

"Everyone counts pipeline-adjacent. Keep the number. We can true it up after the round."

The deck's final slide contains a general disclaimer that projections "involve risks and uncertainties."

The round's anchor investor wired an $8 million first tranche two weeks ago under a signed subscription agreement, based on the deck. The remaining $32 million is committed by term sheet but unsigned, with signing in twelve days. As part of the round, the founder is selling $4 million of her own shares to the incoming investors.

Yesterday, Northgate's second-largest customer, 9 percent of subscription revenue, delivered a non-renewal notice effective at term end in four months. No investor knows this.

The CEO proposes: "Sign on schedule. The business is real. We fix the metrics deck for the Series C." The board meets in two days.

Assignment

Analyze:

  • Whether the ARR and retention figures are misstatements of present fact or protected projections, applying the supplied rules to each number and to the disclaimer.
  • Materiality: quantify the gaps and assess them against the total-mix standard, including the non-renewal notice.
  • Scienter: the effect of the CFO's email, and who is exposed, company, CFO, CEO, directors.
  • The anchor investor's wired $8 million: the company's options, what correction can and cannot cure for that investor, and whether rescission should be offered.
  • The unsigned $32 million: what correction before signing accomplishes under the supplied reliance rule, and what the corrected disclosure must contain, including the customer non-renewal.
  • The founder's $4 million secondary in light of the nonpublic non-renewal notice: may it proceed, on what conditions, or must it be pulled.
  • The board's position under the candor and oversight rules if it follows the CEO's proposal, and the exculpation and insurance consequences of proceeding knowingly.
  • The repricing conversation: how to present corrected metrics, the likely valuation impact, and whether delay, reprice, or restructure best protects the company.
  • A concrete 48-hour plan for counsel: the corrective disclosure, the anchor remediation, the secondary decision, and the board resolutions, in order.
Read the sealed key

Q20 key (Hot ARR Deck)

  1. The ARR and retention figures are misstatements of present fact: $14.2 million versus roughly $10.5 million on the standard definition, and 138 percent computed by excluding the two largest lost accounts versus 111 percent. The generic risk disclaimer protects neither, because the safe harbor covers projections, not present-fact statements, and the caution is boilerplate. Missed: the problem is misclassified as puffery or protected forecasting.
  2. Materiality is not close: a roughly 35 percent ARR overstatement and a 27-point retention gap in a metrics-driven valuation, plus a 9 percent-of-revenue customer non-renewal, each independently alters the total mix. Missed: the board is advised it can proceed.
  3. Scienter: the CFO's "keep the number, true it up after the round" email is direct evidence of knowledge; the CEO and directors acquire scienter the moment they learn and proceed. Exposure runs to the company, the CFO personally, the CEO, and any director who approves signing on the uncorrected deck. Missed: the individuals are not told they are personally in the frame.
  4. The anchor's wired $8 million cannot be cured by correction; the misstatement preceded an irrevocable commitment. The defensible course is full corrective disclosure to the anchor plus an offer of rescission or ratification on corrected terms, documented. Missed: the strongest claimant is left holding an unremediated fraud claim.
  5. The unsigned $32 million can be saved: corrective disclosure before signing cuts off reliance, and the correction must include the true ARR composition, the recomputed retention, and the customer non-renewal notice. Missed: the reliance cure is squandered by a partial correction.
  6. The founder's $4 million secondary cannot proceed while she possesses the nonpublic non-renewal information; it goes forward only after full disclosure to the buyers or it is pulled from the round. Missed: an insider-sale violation is layered onto the deck problem.
  7. Board duties: following the CEO's plan after this meeting is a knowing-disclosure problem sounding in loyalty and good faith, outside exculpation, with fraud carve-outs threatening indemnification and insurance. The board must direct correction, not schedule-keeping. Missed: the directors burn their own protections.
  8. The 48-hour plan: board resolutions directing corrective disclosure and pausing the secondary; corrected metrics package and non-renewal disclosure to all investors; anchor remediation offer; reprice-or-delay negotiation with the lead; document every step. Signing on schedule on the current deck is the one indefensible option and the answer must say so. Missed: the twelve-day clock is obeyed and the company inherits the liability.
Blind evaluation record

Direct judge report

The report below is preserved from the Qwen 3.8 versus Claude verdict record.

Read the preserved judge report
TIE, FUNCTIONALLY EQUIVALENT.

Basis. Both answers direct Northgate to halt the existing process, preserve evidence, correct ARR and NRR, disclose the non-renewal, remediate the anchor through rescission or amended terms with a release, freeze the secondary, and negotiate any financing only after complete disclosure. Competent counsel following either answer reaches the same materially safe path for the company and board.

[A] rejected: “The term sheet is unsigned and the commitment is not irrevocable” — record lacks the term-sheet provisions needed to establish revocability.
[A] rejected: a rescission offer “declined with full information leaves no injury” — supplied Rule 4 does not state that a rejected offer extinguishes damages claims.
[A] rejected: fraud liability is categorically “outside the D&O policy” and creates policy-rescission grounds — record lacks the policy terms.
[B] rejected: “D&O insurance will likely deny coverage” — record lacks the policy terms.
[A] rejected: a $130–165 million valuation range — record lacks market and operating inputs supporting that range.
[A] rejected: stockholder consents will be required — record lacks the charter and transaction provisions needed to establish that requirement.

Failed candidates. Binding-term-sheet caveat — no-safe-neutralization element failed because ordinary review of the term sheet resolves it before action.
Failed candidates. Pulling the secondary versus permitting it after full disclosure — concrete-consequence element failed because the record does not establish that either lawful course harms Northgate.
Failed candidates. Mandatory ten-business-day delay — concrete-consequence element failed because the record supplies no runway or timing facts showing benefit or harm from that period.
Failed candidates. Milestone reset versus a broader reprice or restructure — concrete-consequence element failed because no resulting valuation, dilution, or financing outcome is established.
Failed candidates. Insurance notice and coverage assertions — concrete-consequence element failed because no policy-based coverage outcome is established.
Failed candidates. Effect of a declined rescission offer — different-conduct element failed because both answers recommend rescission or amended terms coupled with a release.
Failed candidates. Valuation-range specificity — different-conduct element failed because both answers ultimately require negotiation on corrected metrics rather than action at a stated valuation.

Hard stop. The sentence cannot be completed with a concrete, record-supported consequence distinguishing the answers.
Complete comparison record

All six pairings

Every pair of the four model answers was judged independently under the same evaluation instrument.

Open the six head-to-head results
PairingResultWhy it matteredRecord
Qwen 3.8 27B vs Claude Opus 5Functional tieBoth answers direct Northgate to halt the existing process, preserve evidence, correct ARR and NRR, disclose the non-renewal, remediate the anchor through rescission or amended terms with a release, freeze the secondary, and negotiate any financing only after complete disclosure.Judge record
Qwen 3.8 27B vs DeepSeek V4-Flash-0731Qwen 3.8 winAnswer A uniquely requires a negotiated release whether the anchor accepts rescission or continues under amended terms. Answer B permits reaffirmation after corrected disclosure and repricing without requiring release, leaving the company exposed to the anchor’s already-accrued rescission or damages claim…Judge record
Qwen 3.8 27B vs Qwen 3.5 27BQwen 3.8 winAnswer A requires any anchor investor continuation to include an amended subscription and release of pre-correction claims, while Answer B’s proposed “Cure Agreement” instead says the Company warrants no future claims—although the relevant rescission and damages claims belong to the anchor.Judge record
DeepSeek V4-Flash-0731 vs Claude Opus 5Functional tieBoth answers direct the board to stop the existing closing, withdraw and correct the deck, disclose the non-renewal, offer the anchor rescission or a fully informed renegotiation, suspend the founder secondary, and reprice or restructure before proceeding.Judge record
DeepSeek V4-Flash-0731 vs Qwen 3.5 27BFunctional tieBoth answers direct the board to halt the current closing, withdraw and correct the deck, disclose the $10.5M ARR, 111% retention, and customer non-renewal, offer anchor remediation, suspend the founder secondary, and reprice or restructure before proceeding.Judge record
Qwen 3.5 27B vs Claude Opus 5Functional tieBoth answers put counsel and the board on the same material path: stop reliance on the existing deck, disclose the corrected ARR, NRR, and customer non-renewal before the unsigned investors commit, offer the anchor rescission or informed renegotiation, halt the founder secondary absent full disclosure…Judge record
Rejected-point diagnostic

Judge-attributed rejected points for this matter across all six pairings:

Qwen 3.8 27B3
DeepSeek V4-Flash-07318
Claude Opus 511
Qwen 3.5 27B7

This is a review-burden diagnostic, not a universal hallucination measure.

Available reasoning traces