Language / HellaSwag (full validation, character-normalized accuracy)
HellaSwag / full sub-1B common core (qwen35)
Provided context/question + candidate answers → selected answer
Model size vs score
Static, watermarked figures. SlayerLab models are red. The close-up expands the crowded small-model cluster without changing any coordinates.
Measured results
Full pinned test split · 10,042 examples · accuracy_char_normalized higher is better. No global SOTA claim.
| System | Score | Samples | Elapsed¹ | Evidence |
|---|---|---|---|---|
| Mapika/decider-0.8b a0a01d6f8135 | 55.7% | 10042 | 592s | Inspect run → 17 Oct 2026, 15:55 UTC |
| Qwen/Qwen3.5-0.8B-Base dc7cdfe2ee41 | 54.8% | 10042 | 585s | Inspect run → 17 Oct 2026, 16:05 UTC |
| SeerRay-Lab/Xiaomi-OCR-0 ff990be318c4 | 41.2% | 10042 | 512s | Inspect run → 17 Oct 2026, 15:54 UTC |
| nineninesix/qwen3_5-full-attn-only-14 a26dff91d90e | 26.5% | 10042 | 259s | Inspect run → 17 Oct 2026, 15:59 UTC |
¹ Evaluation elapsed time includes model loading after dataset selection. It is not serving latency or a fair speed comparison.
What this measures
Benchmark origin
Dataset / split
Rowan/hellaswag / validation
Immutable dataset revision
218ec52e09a7e7462a5400043bb9a69a41d06b76Upstream license metadata
unspecified · consult upstream terms
Protocol
codesota-common-core-qwen35-hellaswag-lm-eval-v2 / choice
Selection
All 10042 examples in the pinned test split. SNLI rows without a valid label are excluded.
Configuration
Seed 42. Greedy generation for math and code. Mean answer-token log likelihood for text classification, completion and declarative multiple-choice recipes. No model judge.
Planned runs
0 qualified visits in 30 days · priority 1006.6. Interest changes execution order, within the approved baseline list.
| System | Status |
|---|---|
| Qwen/Qwen3.5-0.8B | running |
| unsloth/Qwen3.5-0.8B-Base | running |
| McGill-NLP/AfriqueQwen3.5-0.8B-50Langs | queued |
| mithulaartigala/reasoning-summarizer-qwen3.5-0.8b-preview | queued |
| sh0ck0r/Qwen3.5-0.8B-heretic | queued |
| ahmedeltawel/Qwen3.5-0.8B-hsb-dsb | queued |
| OmniJev/PlayJev-0.8B | queued |
| Mapika/decider-0.8b | completed |
| SeerRay-Lab/Xiaomi-OCR-0 | completed |
| nineninesix/qwen3_5-full-attn-only-14 | completed |
| Qwen/Qwen3.5-0.8B-Base | completed |