Inspect the measurement.

Qwen/Qwen2.5-VL-3B-Instruct · OCRBench v1 / full 1,000 questions — experimental batch-4 · 60.7% accuracy

Full benchmark split · 1000 scored examples. All 1,000 OCRBench v1 questions. Upstream answer-containment scoring; response verbosity can game this metric. Separate visual-text QA protocol, not document parsing/layout or table reconstruction. Image/decoding budgets and model chat template are CodeSOTA-pinned, not a claim of vendor inference parity.

Run ID / completed

b25ac4100084fad49c9f3938
18 Oct 2026, 18:17 UTC

Model revision

66285546d2b821cf421d4f5eb2576359d3770cd3

Comparison scope

d5b62220da83f9ebb4c271f820fa35b964ad4a0400651e30e94f5727f4a14007

Evaluator SHA-256

bd9caa01cf3bb32f2b5f470f953d191efed4632fd9263d386b21939ba5dc3eb1

Observed resources

369s elapsed · 7.84 GiB peak torch allocation
Dollar cost: not measured

Download evidence

result.json samples.jsonl environment.json evaluate.py manifest.json

Inspect individual examples

Read the recorded prediction, reference and outcome for each example.

Scored examples: 1,000
Passed: 607
Failed: 393
Metric: accuracy

Examples are downloaded only when requested. Search covers all loaded records; 25 examples are displayed per page. Download all examples · JSONL

Example evidence has not been loaded.