Inspect the measurement.

Qwen/Qwen2-VL-2B-Instruct · OCRBench v1 / full 1,000 questions — experimental batch-4 · 49.0% accuracy

Full benchmark split · 1000 scored examples. All 1,000 OCRBench v1 questions. Upstream answer-containment scoring; response verbosity can game this metric. Separate visual-text QA protocol, not document parsing/layout or table reconstruction. Image/decoding budgets and model chat template are CodeSOTA-pinned, not a claim of vendor inference parity.

Run ID / completed

c6ad7d2fef906cfb7173ab6f
18 Oct 2026, 18:15 UTC

Model revision

895c3a49bc3fa70a340399125c650a463535e71c

Comparison scope

d5b62220da83f9ebb4c271f820fa35b964ad4a0400651e30e94f5727f4a14007

Evaluator SHA-256

bd9caa01cf3bb32f2b5f470f953d191efed4632fd9263d386b21939ba5dc3eb1

Observed resources

265s elapsed · 5.49 GiB peak torch allocation
Dollar cost: not measured

Download evidence

result.json samples.jsonl environment.json evaluate.py manifest.json

Inspect individual examples

Read the recorded prediction, reference and outcome for each example.

Scored examples: 1,000
Passed: 490
Failed: 510
Metric: accuracy

Examples are downloaded only when requested. Search covers all loaded records; 25 examples are displayed per page. Download all examples · JSONL

Example evidence has not been loaded.