Inspect the measurement.

Qwen/Qwen3.5-0.8B-Base · BoolQ (full validation, raw likelihood accuracy) · 74.3% accuracy

Full benchmark split · 3270 scored examples. Full pinned split; primary raw summed conditional answer-token log likelihood accuracy; character-normalized accuracy is a separate diagnostic. Explicit pinned prompt schema, no chat/BOS. Left context truncated only if prompt plus answer exceeds 1024 tokens. Generic text diagnostic does not validate specialist declared tasks.

Run ID / completed

43c0538c00e25eee8470f774
17 Oct 2026, 15:02 UTC

Model revision

dc7cdfe2ee4154fa7e30f5b51ca41bfa40174e68

Comparison scope

087725fb28ea8991a3f6bacbedcecfe584a0305a5e4d876496d753c3718af752

Evaluator SHA-256

eb01abebdaa64a9d8003d37b2006b198582b99f61377a6fe6d0645db15934307

Observed resources

126s elapsed · 4.35 GiB peak torch allocation
Dollar cost: not measured

Download evidence

result.json samples.jsonl environment.json evaluate.py manifest.json

Inspect individual examples

Read the recorded prediction, reference and outcome for each example.

Scored examples: 3,270
Passed: 2,428
Failed: 842
Metric: accuracy

Examples are downloaded only when requested. Search covers all loaded records; 25 examples are displayed per page. Download all examples · JSONL

Example evidence has not been loaded.