Reasoning / GSM8K
Math word problems
Word problem → numerical answer
Model size vs score
Green uses cohort medians, not a universal quality threshold. Frontier points have no measured alternative with both smaller model size and higher score. Lines connect observed points; they do not predict intermediate models. Uncertainty is not included in frontier membership.
2 compatible measurements. Parameter counts come from the architecture instantiated from each pinned model configuration; no weights are loaded for this count. Click a point to inspect its evidence.
Measured results
Pilot comparison · first 32 valid examples · exact_match higher is better. No global SOTA claim.
| System | Score | Samples | Elapsed¹ | Evidence |
|---|---|---|---|---|
| Qwen/Qwen2.5-1.5B 8faed761d45a | 9.4% | 32 | 15s | Inspect run → 17 Oct 2026, 00:47 UTC |
| Qwen/Qwen2.5-0.5B 060db6499f32 | 3.1% | 32 | 12s | Inspect run → 17 Oct 2026, 00:47 UTC |
¹ Evaluation elapsed time includes model loading after dataset selection. It is not serving latency or a fair speed comparison.
What this measures
740312add88f781978c0658806c59bc2815b9866Planned runs
1 qualified visits in 30 days · priority 34.7. Interest changes execution order, within the approved baseline list.
| System | Status |
|---|---|
| Qwen/Qwen2.5-0.5B | completed |
| Qwen/Qwen2.5-1.5B | completed |