Qwen 3.8 across three opponents
98%59 of 60 equal or better
The percentage combines 15 wins and 44 functional ties within this series.
Evidence ยท Series 1
Series 1 source collection
20 legal matters, Q01 through Q20, with prompts, keys, four model outputs, six pairwise judge records per matter, code, and supporting documentation.
Outcome distribution
Qwen 3.8 judgments against all three opponents.
Protocol status: The original Series 1 protocol document is a design artifact and was not executed in every described detail.
Q12 disclosure: The completed hosted rerun was scored after the first attempt was blocked. The first blocked output and full routing record are not distributed.
A/B assignment authority: Stored assignments in the verdict records are authoritative. The bundled deterministic judge helper is historical reference tooling. Read the disclosure.
Benchmark materials
- Matter prompts20 files
- Sealed keys20 files
- Model outputs80 answers
- Blind judge records120 pairings
Code and documentation
- Local runner reference codePython
- Hosted runner reference codePython
- Historical/reference judge helperStored assignments are authoritative
- Battery splitter reference codePython
- Exact judge prompt usedByte-identical across both series
- Current evidence hash recordSHA-256