Language / BoolQ
BoolQ / sub-1B cohort
Passage + question → yes / no
Model size vs score
Green uses cohort medians, not a universal quality threshold. Frontier points have no measured alternative with both smaller model size and higher score. Lines connect observed points; they do not predict intermediate models. Uncertainty is not included in frontier membership.
11 compatible measurements. Parameter counts come from the architecture instantiated from each pinned model configuration; no weights are loaded for this count. Click a point to inspect its evidence.
Measured results
Pilot comparison · first 32 valid examples · accuracy higher is better. No global SOTA claim.
| System | Score | Samples | Elapsed¹ | Evidence |
|---|---|---|---|---|
| SlayerLab/Slayer149-Balanced-ARC-E-ft-45500 e7d4b8dea441 | 78.1% | 32 | 2s | Inspect run → 17 Oct 2026, 01:10 UTC |
| SlayerLab/Slayer139-1.01b 283ffb6d620b | 75.0% | 32 | 2s | Inspect run → 17 Oct 2026, 01:13 UTC |
| Qwen/Qwen2.5-0.5B-Instruct 7ae557604adf | 71.9% | 32 | 2s | Inspect run → 17 Oct 2026, 01:08 UTC |
| SlayerLab/Slayer149 7857dd28ec05 | 68.8% | 32 | 2s | Inspect run → 17 Oct 2026, 01:11 UTC |
| SlayerLab/Slayer-149M-Decision-v1 ccfd7d3d91f3 | 68.8% | 32 | 2s | Inspect run → 17 Oct 2026, 01:12 UTC |
| SlayerLab/Slayer149-Balanced-ARC-E-ft-v2 7d11e8fd6376 | 62.5% | 32 | 2s | Inspect run → 17 Oct 2026, 01:09 UTC |
| SlayerLab/Slayer139-1.01 2e72c696d7c5 | 56.3% | 32 | 2s | Inspect run → 17 Oct 2026, 01:12 UTC |
| SlayerLab/Slayer139-1.02 c4298d757ba3 | 56.3% | 32 | 2s | Inspect run → 17 Oct 2026, 01:08 UTC |
| SlayerLab/Slayer149-ARC 0e6793ecfb85 | 50.0% | 32 | 2s | Inspect run → 17 Oct 2026, 01:12 UTC |
| HuggingFaceTB/SmolLM2-360M-Instruct a10cc1512eab | 34.4% | 32 | 2s | Inspect run → 17 Oct 2026, 01:12 UTC |
| HuggingFaceTB/SmolLM2-135M-Instruct 12fd25f77366 | 21.9% | 32 | 1s | Inspect run → 17 Oct 2026, 01:08 UTC |
¹ Evaluation elapsed time includes model loading after dataset selection. It is not serving latency or a fair speed comparison.
What this measures
35b264d03638db9f4ce671b711558bf7ff0f80d5Planned runs
1 qualified visits in 30 days · priority 1007.0. Interest changes execution order, within the approved baseline list.
| System | Status |
|---|---|
| Qwen/Qwen2.5-0.5B-Instruct | completed |
| SlayerLab/Slayer139-1.02 | completed |
| HuggingFaceTB/SmolLM2-135M-Instruct | completed |
| SlayerLab/Slayer149-Balanced-ARC-E-ft-v2 | completed |
| SlayerLab/Slayer149-Balanced-ARC-E-ft-45500 | completed |
| SlayerLab/Slayer149 | completed |
| SlayerLab/Slayer139-1.01 | completed |
| SlayerLab/Slayer-149M-Decision-v1 | completed |
| SlayerLab/Slayer149-ARC | completed |
| HuggingFaceTB/SmolLM2-360M-Instruct | completed |
| SlayerLab/Slayer139-1.01b | completed |