Operator-controlled path
Matter + prompts→Local model→Answer
No external router or cloud provider was required during generation.
Evidence · Technical Record
Systems and reproducibility
Exact public identifiers, local hardware, configuration, routing, completion status, and reproduction requirements.
System boundary
What the benchmark actually compared
The model name is not the system. The operational unit was the configured delivery path that produced the text shown to the judge.
The three local models ran on operator-controlled hardware. The documented hosted Claude path included OpenRouter, the upstream provider recorded as Amazon Bedrock, the model service, and intervening policy layers. The records identify the route but do not establish which hosted layer triggered a content filter.
Multi-provider delivery chain
Matter + prompts→OpenRouter→Amazon Bedrock, model service, and policy layers→Answer
The hosted result reflects the complete delivery chain, not the model name alone.
Privacy, custody, and operational control
Why local control matters in legal work
Local generation can keep sensitive matter data inside infrastructure controlled by the lawyer, firm, or client instead of sending it through outside gateways, cloud hosts, and model providers.
What the architecture changes
- Fewer outside custodians: the matter and prompts do not need to be copied to independent AI providers.
- No required internet transmission: an isolated local deployment can remove the external generation path.
- Direct control: the organization controls storage, access, retention, deletion, and audit records for the local system.
- Operational independence: an outside router or policy layer cannot silently stop local generation.
Systems under test
Four contestants
Public model identifiers and deployment roles used in the completed record.
Qwen 3.8 27B
Qwen3.8-27B-UD-Q6_K_XL
Dense 27B open-weight model, Q6_K_XL file, loaded entirely on one GPU through the recorded local server.
DeepSeek V4-Flash-0731
DeepSeek-V4-Flash-0731-UD-IQ4_XS
The prior local leader, preserved unchanged from the previous release.
Claude Opus 5
claude-opus-5
Reached through the documented hosted delivery path.
Qwen 3.5 27B
Qwen3.5-27B-UD-Q6_K_XL
Qwen 3.8's direct predecessor, evaluated on the same 80-matter battery.
Local execution environment
One operator-controlled workstation
Qwen 3.8 completed all 80 matters locally on the same workstation class used for the prior local systems.
| GPU | NVIDIA RTX 5090, 32 GB VRAM |
|---|---|
| System memory | 128 GB RAM |
| Runtime | llama.cpp compatible local server |
| Qwen 3.8 context | 131,072 tokens recorded in the server properties |
| Tools | None |
| Network dependency during local generation | None required |
Blind judge
OpenAI GPT-5.6 Sol
A rival-lab judge evaluated all 480 pairings under the exact same published prompt.
openai/gpt-5.6-sol
480
Six pairings for each of 80 matters.
One SHA-256
7e58b951cb86e4de…
Recorded configuration
Qwen 3.8 run record
The published machine records preserve the model identifier, context size, sampling settings, answer hashes, and finish reason for every Qwen 3.8 matter.
Configuration files
- Run conditions summarySettings and totals
- Server propertiesJSON
- Sanitized generation log80 records
- Matter-level run recordsQ01–Q80
Supplied raw package
- Four-model result packageUntouched ZIP
- Package checksumSHA-256
Delivery record
Qwen 3.8 completed all 80 matters
Every Qwen 3.8 run recorded finish_reason: stop, with no failed or truncated run.
020406080
Qwen 3.8complete80of 80
The three recorded hosted content_filter interruptions remain Q59, Q61, and Q75. In the Qwen 3.8 versus Claude record, those matters produced two Qwen wins and one tie.
Reproduction
What another researcher needs
Use the published matter, answers, exact judge prompt, and stored A/B assignment to reproduce a completed judgment.