66285546d2b8 · unreviewed
Intended task has not been reviewed for this revision.
unreviewed
- Review this exact revision, intended task and model interface.
Independent measured model card
Qwen/Qwen2.5-VL-3B-Instruct
Hugging Face source ↗ · JSON for agents
3,754.6M unique parameters (66285546d2b8)
Every result links to its execution evidence. Compare scores only within matching protocols; no overall best-model claim. GPU-slot time includes evaluator setup and CPU work.
All current benchmark results
| Benchmark / coverage | Result | Version / protocol | GPU-slot time | Evidence |
|---|---|---|---|---|
| OCRBench v1 / full 1,000 questions — experimental batch-4 full · 1,000 items | 60.70% accuracy · max | 66285546d2b8codesota-ocrbench-v1-full-bf16-qwen-batch4-v1 | 372.2s | Inspect run · Manifest |
| OCRBench v1 / full serial reference full · 1,000 items | 81.30% accuracy · max | 66285546d2b8codesota-ocrbench-v1-full-native-resolution-serial-v1 | 683.3s | Inspect run · Manifest |
The batch-4 OCRBench protocol failed singleton agreement checks. Its scores are experimental; use the separate full serial reference for model comparison.
Does this benchmark fit this model?
A runnable model is not automatically a valid measurement of its intended job. Specialist task quality and generic language diagnostics are separate.
Where this model sits
Position 1 of 2 measured system versions · 1000 items · full.
Small comparison cohort: only 1 other system versions. This is not a distribution of all available models.
Other assigned model versions with the same current scope, metric and item count; latest completed run per version. All revisions of the selected model are excluded from peers. Ranks are descriptive, not significance tests or population percentiles.