Series 1 · Discovery and privilege

The Deleted Messages and Privileged Memorandum

Q09Four modelsSix blind pairingsMaterial-outcome scoring

Why this result mattered

Functional tie

The principal comparison placed Qwen 3.8 and Claude under concealed A/B labels.

Both answers put Northstar on the same material path: concede the preservation gaps, undertake and fund targeted recovery, oppose a case-wide adverse inference and blanket phone imaging, permit limited preservation discovery, and enforce the memorandum’s clawback. Their differing assessments of prejudice, the vice president’s intent, and the precise curative package do not establish a record-supported difference in Northstar’s ultimate legal, financial, or evidentiary position.

Benchmark materials

Prompt and sealed key

The key is an audit reference. It was not shown to the contestant systems and did not determine the comparative verdict.

Read the original matter prompt

Question 9: The Deleted Messages and Privileged Memorandum

Your role

You represent Northstar Components in a federal antitrust case. The plaintiff has moved for sanctions, broad forensic imaging, and use of a privileged memorandum that Northstar inadvertently produced.

Governing law

  1. A duty to preserve arises when litigation is reasonably anticipated.

  2. A party must take reasonable and proportional steps to preserve potentially relevant evidence. Perfection is not required.

  3. Sanctions for lost electronically stored information require that:

    • The information should have been preserved.
    • It was lost because reasonable steps were not taken.
    • It cannot be restored or replaced through additional discovery.
  4. Upon a finding of prejudice, a court may impose measures no greater than necessary to cure the prejudice.

  5. An adverse inference, dismissal, or default requires a finding that the party acted with intent to deprive another party of the information’s use in litigation.

  6. Routine deletion is not automatically sanctionable after a preservation duty arises. The question is whether the party reasonably suspended or modified relevant systems.

  7. A corporation is responsible for preservation failures by employees within its possession, custody, or control. A senior employee’s deliberate deletion may be attributed to the corporation depending on authority, purpose, and the company’s response.

  8. Forensic imaging is not automatic. It must be proportional, targeted, and supported by evidence that ordinary discovery is inadequate.

  9. Attorney-client privilege protects confidential communications made for obtaining or providing legal advice. Business advice does not become privileged merely because a lawyer participates.

  10. Work product protects material prepared primarily because of anticipated litigation.

  11. Under the court’s clawback order, inadvertent production does not waive privilege when the producing party took reasonable precautions and promptly sought return after discovering the production.

  12. The receiving party must stop using a document after receiving a valid clawback demand until the court resolves the privilege claim.

Facts

On January 5, Northstar received a demand letter from a former distributor. The letter accused Northstar and several competitors of coordinating prices and stated that suit would be filed unless the dispute was resolved.

On January 8, Northstar’s general counsel issued a written hold to twelve executives. The hold suspended ordinary email deletion. It instructed recipients to preserve documents concerning distributor pricing, competitor communications, and “Project Lighthouse,” Northstar’s internal name for a pricing initiative.

The hold did not mention Slack, text messages, personal phones, or two outside pricing consultants.

Northstar’s Slack system automatically deleted messages after 30 days. On January 9, IT told the general counsel that suspending deletion would require an $18,000 account upgrade. The general counsel responded:

“Do the minimum until they actually sue. Email is the official record.”

Suit was filed March 1. Slack messages from January were already gone.

Some relevant Slack conversations survive because employees took screenshots and because one participant manually exported a channel.

On February 2, Northstar’s vice president of sales sent managers a text stating:

“Clean up jokes and speculation. Keep real business records.”

He then deleted approximately 200 text messages. He says the messages consisted of personal jokes, political comments, and embarrassing but irrelevant speculation. A whistleblower preserved screenshots of two deleted messages:

“Lighthouse only works if the others hold the umbrella.”

“Do not undercut the signal before Q3.”

The vice president says “umbrella” referred to maintaining Northstar’s own pricing discipline and that “signal” referred to published commodity prices.

Northstar’s mobile provider cannot restore the deleted text content. Some recipients may still possess copies.

Northstar’s Slack vendor maintained disaster-recovery data for 60 days. Those backups were overwritten on March 15 under the vendor’s default schedule because Northstar did not request restoration after suit was filed.

During document review, Northstar searched for competitor names and pricing terminology. The initial search did not include “Project Lighthouse,” even though the legal hold used that term. Reviewers discovered the omission after producing 400,000 documents. Northstar paused production and added the term.

Northstar eventually produced approximately 1.2 million documents. The production included a memorandum from outside counsel to the board analyzing antitrust exposure. The memorandum was correctly marked privileged in Northstar’s review platform but was included because of a production-script error.

Northstar discovered the error twelve days later and immediately sent a clawback notice. Before receiving the notice, plaintiff’s counsel questioned a witness about the memorandum and quoted from it in a deposition exhibit. After receiving the notice, counsel continued referring to its “admissions” in correspondence.

The plaintiff seeks:

  • An adverse inference that deleted messages showed price coordination.
  • Monetary sanctions.
  • Depositions of the general counsel and outside counsel.
  • Forensic images of all executives’ personal phones.
  • Denial of the clawback request.
  • Permission to use the memorandum as evidence of Northstar’s knowledge and intent.

Assignment

Analyze:

  • When Northstar’s preservation duty arose.
  • The reasonableness of the January 8 hold.
  • Loss of Slack data, vendor backups, and text messages.
  • Whether any lost information can be replaced.
  • Prejudice and intent to deprive.
  • Whether the vice president’s conduct should be attributed to Northstar.
  • Appropriate curative measures and whether an adverse inference is justified.
  • Whether forensic imaging of personal phones is proportional.
  • Whether the general counsel or outside counsel may be deposed.
  • Whether the memorandum is privileged, whether privilege was waived, and whether it must be returned.
  • The receiving lawyer’s continued use after the clawback notice.
  • Northstar’s duty to disclose additional preservation failures or search defects.
  • Immediate remediation steps.
  • The most defensible position Northstar should take with the court rather than merely opposing every request.


Read the sealed key

Q9 key (Deleted Messages and Memo)

  1. The preservation duty arose January 5 at the demand letter, not at filing. Missed: the whole defense timeline is wrong.
  2. The hold was unreasonably narrow: Slack, texts, personal phones, and the two consultants were omitted, and the general counsel's "do the minimum until they actually sue" is a conscious-decision document. Missed: defending the hold as written destroys credibility.
  3. Sanctions ladder: Slack and backup losses read as unreasonable-steps negligence with partial replaceability (screenshots, export, recipients), supporting curative measures and fees, not an adverse inference; the vice president's targeted deletions after his "clean up" text, with the two damning screenshots, are the genuine intent-to-deprive exposure, attributable to the company as a contested question. Missed: the client fights the wrong sanction or concedes the worst one.
  4. The March 15 backup overwrite, after suit, is a second, independent failure (restoration never requested). Missed: an aggravating fact surfaces unmanaged.
  5. Forensic imaging of all executives' personal phones is disproportionate; the defensible counter is targeted collection from the vice president and identified custodians. Missed: an overbroad intrusion or a credibility-costing blanket refusal.
  6. Depositions: a narrow, non-privileged preservation-facts deposition of the general counsel is a live risk to manage; outside counsel is off limits. Missed: privilege overclaimed into sanctions.
  7. The memorandum stays privileged: correct marking, script error, prompt clawback satisfy the order's standard; opposing counsel's post-notice use violates the stop-use rule and supports relief against them. Missed: a board-level antitrust analysis becomes an exhibit.
  8. Posture: self-disclose the search-term defect and preservation gaps, propose a concrete remediation and collection plan, pay reasonable fees, and fight only the adverse inference and the phone imaging. Missed: reflexive opposition converts a survivable record into case-ending sanctions.
Blind evaluation record

Direct judge report

The report below is preserved from the Qwen 3.8 versus Claude verdict record.

Read the preserved judge report
TIE, FUNCTIONALLY EQUIVALENT.

Basis. Both answers put Northstar on the same material path: concede the preservation gaps, undertake and fund targeted recovery, oppose a case-wide adverse inference and blanket phone imaging, permit limited preservation discovery, and enforce the memorandum’s clawback. Their differing assessments of prejudice, the vice president’s intent, and the precise curative package do not establish a record-supported difference in Northstar’s ultimate legal, financial, or evidentiary position.

[A] rejected: “The vice president’s deletion may have been a routine cleanup” — internally inconsistent with the directed deletion of approximately 200 messages after the preservation duty arose.
[A] rejected: “The core pricing and competitor-communication documents appear to have been produced” — record lacks facts establishing completeness of those productions.
[B] rejected: the vice president deleted messages “after receiving the hold” — record lacks identification of the twelve hold recipients.
[B] rejected: preserved email “overlap[s] substantially with Slack content” — record lacks evidence establishing that overlap.
[B] rejected: Project Lighthouse “was active” during the lost Slack month — record lacks its operative dates.

Failed candidates. Different assessments of the vice president’s intent — concrete consequence failed; both oppose the requested coordination inference and accept targeted relief.
Failed candidates. B’s prejudice concession and fee-shifting offer versus A’s conditional position — concrete consequence failed; the record does not establish that either posture produces a better net result.
Failed candidates. Full forensic acquisition of the vice president’s phone versus targeted imaging or extraction — material magnitude failed; both confine review through a protected, targeted protocol.
Failed candidates. Different proposed jury stipulations or instructions — concrete consequence failed; neither answer accepts the plaintiff’s requested case-wide inference.
Failed candidates. Different sequencing and privilege treatment for the general counsel’s deposition — different conduct failed; both permit limited non-privileged preservation discovery.
Failed candidates. Different sanctions requests for post-clawback use — likelihood failed; A’s more severe possibilities are qualified, while both seek cessation, return, and curative relief.
Failed candidates. B’s additional employment and internal-investigation measures — record support failed; no supplied fact establishes that those additions alter the litigation outcome.
Failed candidates. A’s suggestion of a possible adverse inference against plaintiff — likelihood failed; A expressly identifies it as potentially excessive.

Hard stop. The sentence cannot be completed with a concrete, record-supported consequence distinguishing the answers.
Complete comparison record

All six pairings

Every pair of the four model answers was judged independently under the same evaluation instrument.

Open the six head-to-head results
PairingResultWhy it matteredRecord
Qwen 3.8 27B vs Claude Opus 5Functional tieBoth answers put Northstar on the same material path: concede the preservation gaps, undertake and fund targeted recovery, oppose a case-wide adverse inference and blanket phone imaging, permit limited preservation discovery, and enforce the memorandum’s clawback.Judge record
Qwen 3.8 27B vs DeepSeek V4-Flash-0731Functional tieBoth answers place Northstar on the same material path: concede the preservation failures, investigate replacement sources, offer targeted remediation and reasonable curative costs, oppose an adverse inference absent intent, resist blanket phone imaging and outside-counsel testimony, and enforce the…Judge record
Qwen 3.8 27B vs Qwen 3.5 27BQwen 3.8 winAnswer A directs Northstar to issue a comprehensive supplemental hold covering Slack, texts, personal devices, consultants, and all relevant custodians, while Answer B recommends only limited preservation of selected devices and recipients.Judge record
DeepSeek V4-Flash-0731 vs Claude Opus 5Functional tieBoth answers direct Northstar to concede the January 5 preservation trigger and preservation failures, pursue replacement discovery and targeted device examination, accept proportionate monetary relief, oppose a broad adverse inference and blanket imaging, limit counsel depositions, and enforce the…Judge record
DeepSeek V4-Flash-0731 vs Qwen 3.5 27BFunctional tieBoth answers place Northstar on the same material path: acknowledge the January 5 preservation duty and subsequent failures, pursue replacement evidence, oppose severe sanctions and blanket imaging absent the required findings, use targeted discovery, and enforce the clawback order.Judge record
Qwen 3.5 27B vs Claude Opus 5Claude winAnswer B directs immediate, Northstar-funded forensic and recipient-side collection to replace the deleted Slack and text evidence, while Answer A primarily proposes preservation and possible later imaging if ordered.Judge record
Rejected-point diagnostic

Judge-attributed rejected points for this matter across all six pairings:

Qwen 3.8 27B4
DeepSeek V4-Flash-07312
Claude Opus 55
Qwen 3.5 27B11

This is a review-burden diagnostic, not a universal hallucination measure.

Available reasoning traces