Reasoning / HellaSwag
Commonsense completion
Scenario + four endings → most plausible ending
Model size vs score
Green uses cohort medians, not a universal quality threshold. Frontier points have no measured alternative with both smaller model size and higher score. Lines connect observed points; they do not predict intermediate models. Uncertainty is not included in frontier membership.
2 compatible measurements. Parameter counts come from the architecture instantiated from each pinned model configuration; no weights are loaded for this count. Click a point to inspect its evidence.
Measured results
Pilot comparison · first 32 valid examples · accuracy higher is better. No global SOTA claim.
| System | Score | Samples | Elapsed¹ | Evidence |
|---|---|---|---|---|
| Qwen/Qwen2.5-1.5B 8faed761d45a | 50.0% | 32 | 2s | Inspect run → 17 Oct 2026, 00:48 UTC |
| Qwen/Qwen2.5-0.5B 060db6499f32 | 37.5% | 32 | 1s | Inspect run → 17 Oct 2026, 00:47 UTC |
¹ Evaluation elapsed time includes model loading after dataset selection. It is not serving latency or a fair speed comparison.
What this measures
218ec52e09a7e7462a5400043bb9a69a41d06b76Planned runs
1 qualified visits in 30 days · priority 6.9. Interest changes execution order, within the approved baseline list.
| System | Status |
|---|---|
| Qwen/Qwen2.5-0.5B | completed |
| Qwen/Qwen2.5-1.5B | completed |