Why this result mattered
Functional tie
The principal comparison placed Qwen 3.8 and Claude under concealed A/B labels.
Both answers direct Silverline to omit the waived solicitation challenge, treat September 4 as the knowledge date, and ensure Board receipt by September 11 rather than pursue the agency-first sequence. Both preserve the automatic stay through a substantively adequate Board protest, request protected access and later supplementation, and identify discretionary Board or court relief as the fallback.
Fast audit
Open the underlying record
These links open all four answers, the principal blind judgment, original prompt, and sealed key.
Prompt and sealed key
The key is an audit reference. It was not shown to the contestant systems and did not determine the comparative verdict.
Read the original matter prompt
Question 46: The Protest That Keeps the Stay
Your role
You represent Silverline Mapping, an unsuccessful bidder for a fictional Federal Survey Authority contract. Silverline wants to protest the award. Several filing paths remain available, but only one timely filing preserves an automatic stay of performance.
Governing law
Assume the following procurement rules apply:
A protest based on an apparent solicitation defect must be filed before the bid deadline. A bidder that submits without timely protesting an obvious term waives that ground.
A post-award protest to the Procurement Review Board must be filed within ten calendar days after the protester knew or should have known the basis. The day of knowledge is excluded.
A required debriefing delays knowledge of matters first disclosed there. It does not revive an apparent solicitation defect.
Award performance is automatically stayed only if the Board receives the protest by the later of: (a) ten calendar days after award notice; or (b) five calendar days after the required debriefing. The triggering day is excluded.
A timely merits protest filed after the automatic-stay deadline may proceed, but performance continues unless the Board or a court grants discretionary relief.
An agency-level protest does not trigger the Board's automatic stay and does not toll either Board deadline unless the agency issues a written suspension before the Board deadline expires.
A court may issue an injunction based on likely success, irreparable harm, balance of harms, and public interest. Filing in court creates no automatic stay.
The protest must state each ground, the supporting facts then known, requested relief, and any request for protected access to procurement records. Later supplementation is permitted for facts first learned from the agency report.
Calendar days include weekends. If a deadline falls on a day the Board is closed, filing is due the next open day. The Board is open on every date relevant here.
Facts
The Authority issued its award notice to Silverline on Wednesday, September 1, 2027. The notice selected Atlas Meridian for a $28 million mapping contract.
Before bids were due, the solicitation openly required five years of polar-region experience. Silverline had only three years but submitted without protest. It now argues that the requirement was unnecessarily restrictive.
The Authority held Silverline's required debriefing on Saturday, September 4. For the first time, the Authority disclosed that:
- It credited Atlas with a sensor-resolution score based on a model Atlas did not identify in its proposal.
- It treated Silverline's subcontractor experience as worth zero despite a solicitation clause stating that relevant subcontractor experience "will be considered."
- Atlas's evaluated price was $1.8 million higher, but the Authority selected it based on the disputed technical scoring.
Silverline received the written debriefing slides the same day. Today is Tuesday, September 7. Management proposes first filing an agency protest on September 10 and waiting for the agency's answer before going to the Board. The agency has made no written commitment to suspend performance.
Atlas is scheduled to begin transition work on September 13. Silverline spent $620,000 preparing its proposal and estimates that transition will make meaningful corrective competition harder, but proposal costs alone do not establish irreparable harm.
Assignment
Provide an ordered protest plan addressing:
- The waived solicitation ground.
- The date Silverline knew the post-award grounds.
- The ten-calendar-day Board merits deadline.
- The two alternative automatic-stay calculations and which date controls.
- The last filing date that preserves the automatic stay.
- Why the proposed agency-first sequence does or does not protect the stay.
- The grounds to include in the initial Board protest.
- The role of protected-record access and supplementation.
- Court relief if the automatic stay is lost.
- The exact filing sequence Silverline should use to preserve the strongest procedural position.
Read the sealed key
Q46 key (Protest That Keeps the Stay)
- The polar-experience challenge was apparent before bids and is waived. Do not lead with or include it as a live post-award ground. Missed: the protest spends credibility on a forfeited issue.
- Silverline learned the scoring grounds at the September 4 debriefing. Excluding that day, the ten-day merits deadline is September 14. Missed: the Board protest is filed late.
- Ten days after the September 1 award is September 11. Five days after the September 4 debriefing is September 9. The later date, September 11, controls the automatic stay. Missed: counsel uses the shorter or wrong trigger.
- Filing by September 11 preserves the stay; filing September 12 through 14 may preserve merits but loses the automatic stay. Missed: performance begins because a timely protest was not stay-timely.
- An agency protest on September 10 does not trigger or toll the Board stay without a written suspension. File the Board protest first or concurrently by September 11 rather than waiting. Missed: the agency-first sequence quietly forfeits the central procedural benefit.
- Initial grounds are unsupported credit for Atlas's unidentified sensor, failure to consider subcontractor experience as promised, and resulting best-value error given the $1.8 million premium. Seek protected access and reserve supplementation after the agency report. Missed: the initial filing lacks the known basis or overstates facts not yet available.
- If the stay is lost, court relief requires the full injunction showing and proposal cost alone is not irreparable harm. The defensible sequence is Board filing by September 11, optional agency engagement without reliance, then court only if needed. Missed: discretionary litigation replaces an available automatic protection.
Direct judge report
The report below is preserved from the Qwen 3.8 versus Claude verdict record.
Read the preserved judge report
Basis. Both answers direct Silverline to omit the waived solicitation challenge, treat September 4 as the knowledge date, and ensure Board receipt by September 11 rather than pursue the agency-first sequence. Both preserve the automatic stay through a substantively adequate Board protest, request protected access and later supplementation, and identify discretionary Board or court relief as the fallback.
[A] rejected: conditional disparate-treatment Count IV — record lacks any fact that the Authority relaxed the experience requirement for Atlas.
[A] rejected: independently inadequate documentation of the tradeoff — record lacks facts concerning the adequacy of the tradeoff documentation.
[A] rejected: an injunction motion would cost six figures — record lacks litigation-cost inputs.
[B] rejected: Atlas can be made whole for delay costs — record lacks facts or rules establishing such compensation.
Failed candidates. September 7 versus September 8 recommended filing — concrete consequence failed.
Failed candidates. Conditional disparate-treatment count versus omission — concrete consequence failed.
Failed candidates. Agency protest concurrently versus only after Board filing — concrete consequence failed.
Failed candidates. Additional tradeoff-documentation theory — concrete consequence failed.
Failed candidates. Express request for proposal and protest costs — record support failed.
Failed candidates. Differences in injunction-factor framing — different conduct failed.
Hard stop. The required sentence cannot be completed with a concrete, record-supported consequence.
All six pairings
Every pair of the four model answers was judged independently under the same evaluation instrument.
Open the six head-to-head results
| Pairing | Result | Why it mattered | Record |
|---|---|---|---|
| Qwen 3.8 27B vs Claude Opus 5 | Functional tie | Both answers direct Silverline to omit the waived solicitation challenge, treat September 4 as the knowledge date, and ensure Board receipt by September 11 rather than pursue the agency-first sequence. | Judge record |
| Qwen 3.8 27B vs DeepSeek V4-Flash-0731 | Functional tie | Both answers direct Silverline to omit the waived solicitation ground, file the complete Board protest by September 11, 2027, request protected-record access, supplement on newly learned facts, and avoid relying on an agency-first filing. | Judge record |
| Qwen 3.8 27B vs Qwen 3.5 27B | Functional tie | Both answers direct Silverline to omit the waived solicitation challenge, treat September 4 as the knowledge date, file the substantive Board protest no later than September 14, and ensure Board receipt by Saturday, September 11 to preserve the automatic stay rather than waiting on an agency protest. | Judge record |
| DeepSeek V4-Flash-0731 vs Claude Opus 5 | Functional tie | Both answers direct Silverline to omit the waived solicitation challenge, file the Board protest by September 11 rather than pursue the agency-first sequence, plead the three supported post-award grounds, request protected access, and supplement after the agency report. | Judge record |
| DeepSeek V4-Flash-0731 vs Qwen 3.5 27B | Functional tie | Both answers direct Silverline to omit the waived ground, file the three known post-award grounds directly with the Board by September 11, request protected-record access, and supplement upon learning new facts. | Judge record |
| Qwen 3.5 27B vs Claude Opus 5 | Functional tie | Both answers direct Silverline to omit the waived solicitation challenge, file the known post-award grounds directly with the Board by September 11, request protected access, and supplement newly discovered facts, thereby preserving the automatic stay. | Judge record |
Rejected-point diagnostic
Judge-attributed rejected points for this matter across all six pairings:
| Qwen 3.8 27B | 3 |
|---|---|
| DeepSeek V4-Flash-0731 | 0 |
| Claude Opus 5 | 9 |
| Qwen 3.5 27B | 5 |
This is a review-burden diagnostic, not a universal hallucination measure.