Documents / OCRBench v1 / full serial reference
Visual text understanding / serial OCRBench reference
Image + question → answer
Model size vs score
Static, watermarked figures. SlayerLab models are red. The close-up expands the crowded small-model cluster without changing any coordinates.
Measured results
Full pinned test split · 1,000 examples · accuracy higher is better. No global SOTA claim.
| System | Score | Samples | Elapsed¹ | Evidence |
|---|---|---|---|---|
| Qwen/Qwen2.5-VL-3B-Instruct 66285546d2b8 | 81.3% | 1000 | 680s | Inspect run → 18 Oct 2026, 18:32 UTC |
| Qwen/Qwen2-VL-2B-Instruct 895c3a49bc3f | 78.5% | 1000 | 768s | Inspect run → 18 Oct 2026, 18:33 UTC |
¹ Evaluation elapsed time includes model loading after dataset selection. It is not serving latency or a fair speed comparison.
What this measures
Benchmark origin
Dataset / split
echo840/OCRBench / test
Immutable dataset revision
92a54bd1384387c178d5a07140a2d85e0a3d12e1Upstream license metadata
Upstream benchmark code MIT; component data retain their source terms · consult upstream terms
Protocol
codesota-ocrbench-v1-full-native-resolution-serial-v1 / ocrbench
Selection
All 1000 examples in the pinned test split. SNLI rows without a valid label are excluded.
Configuration
Seed 42. Greedy generation for math and code. Mean answer-token log likelihood for text classification, completion and declarative multiple-choice recipes. No model judge.
Planned runs
0 qualified visits in 30 days · priority 0.0. Interest changes execution order, within the approved baseline list.
| System | Status |
|---|---|
| Qwen/Qwen2.5-VL-3B-Instruct | completed |
| Qwen/Qwen2-VL-2B-Instruct | completed |