Language / ARC-Easy / Qwen3.5 same-base cohort

Qwen3.5 0.8B cohort / full ARC-Easy

Question + candidate answers → selected answer

Measurements Evaluation protocol Planned runs Cost & information

Model size vs score

Static, watermarked figures. SlayerLab models are red. The close-up expands the crowded small-model cluster without changing any coordinates.

ARC-Easy / Qwen3.5 same-base cohort: all measured models, with a dotted Pareto frontier and peach favourable region — Overview · Download PNG · Download SVG

ARC-Easy / Qwen3.5 same-base cohort: small-model close-up, with a dotted Pareto frontier and peach favourable region — Small-model close-up · Download PNG · Download SVG

Measured results

Full pinned test split · 2,376 examples · accuracy higher is better. No global SOTA claim.

System	Score	Samples	Elapsed¹	Evidence
Mapika/decider-0.8b a0a01d6f8135	75.8%	2376	87s	Inspect run → 17 Oct 2026, 03:47 UTC
Qwen/Qwen3.5-0.8B-Base dc7cdfe2ee41	70.5%	2376	53s	Inspect run → 17 Oct 2026, 03:45 UTC
unsloth/Qwen3.5-0.8B-Base 44bc982576bf	70.5%	2376	75s	Inspect run → 17 Oct 2026, 03:46 UTC
NamanAgnih0tri/AlphaRoute-0.8B-v1.5 ef87d71f1105	69.6%	2376	80s	Inspect run → 17 Oct 2026, 03:54 UTC
McGill-NLP/AfriqueQwen3.5-0.8B-50Langs f49099d79e51	67.7%	2376	119s	Inspect run → 17 Oct 2026, 03:49 UTC
mithulaartigala/reasoning-summarizer-qwen3.5-0.8b-preview 558fc651bf01	66.9%	2376	91s	Inspect run → 17 Oct 2026, 03:53 UTC
Qwen/Qwen3.5-0.8B 2fc06364715b	65.9%	2376	76s	Inspect run → 17 Oct 2026, 03:57 UTC
Kewk/Heretical-Qwen3.5-0.8B c16807de35d1	65.7%	2376	82s	Inspect run → 17 Oct 2026, 03:55 UTC
sh0ck0r/Qwen3.5-0.8B-heretic fb35b60374fc	65.2%	2376	75s	Inspect run → 17 Oct 2026, 03:48 UTC
ahmedeltawel/Qwen3.5-0.8B-hsb-dsb 7e781a3e9704	63.6%	2376	83s	Inspect run → 17 Oct 2026, 03:53 UTC
cooperdk/BooruNL-0.8B 14dbcf2e009f	63.1%	2376	102s	Inspect run → 17 Oct 2026, 03:52 UTC
SeerRay-Lab/Xiaomi-OCR-0 ff990be318c4	56.5%	2376	74s	Inspect run → 17 Oct 2026, 03:50 UTC
mghafiri/qwen3.5-0.8B-decision-model 4a9939034a27	49.0%	2376	87s	Inspect run → 17 Oct 2026, 03:51 UTC
OmniJev/PlayJev-0.8B a7348002b1e1	27.6%	2376	91s	Inspect run → 17 Oct 2026, 03:45 UTC
nineninesix/qwen3_5-full-attn-only-14 a26dff91d90e	25.0%	2376	44s	Inspect run → 17 Oct 2026, 03:56 UTC

¹ Evaluation elapsed time includes model loading after dataset selection. It is not serving latency or a fair speed comparison.

What this measures

Benchmark origin

ARC-Easy / Qwen3.5 same-base cohort ↗

Dataset / split

allenai/ai2_arc / test

Immutable dataset revision

210d026faf9955653af8916fad021475a3f00453

Upstream license metadata

cc-by-sa-4.0 · consult upstream terms

Protocol

codesota-qwen35-arc-easy-v1 / choice

Selection

All 2376 examples in the pinned test split. SNLI rows without a valid label are excluded.

Configuration

Seed 42. Greedy generation for math and code. Summed conditional answer-token log likelihood; raw accuracy. Character-normalized accuracy is recorded separately. No model judge.

Read scoring and comparison rules →

Planned runs

0 qualified visits in 30 days · priority 0.0. Interest changes execution order, within the approved baseline list.

System	Status
Qwen/Qwen3.5-0.8B-Base	completed
OmniJev/PlayJev-0.8B	completed
unsloth/Qwen3.5-0.8B-Base	completed
Mapika/decider-0.8b	completed
sh0ck0r/Qwen3.5-0.8B-heretic	completed
McGill-NLP/AfriqueQwen3.5-0.8B-50Langs	completed
SeerRay-Lab/Xiaomi-OCR-0	completed
mghafiri/qwen3.5-0.8B-decision-model	completed
cooperdk/BooruNL-0.8B	completed
ahmedeltawel/Qwen3.5-0.8B-hsb-dsb	completed
mithulaartigala/reasoning-summarizer-qwen3.5-0.8b-preview	completed
NamanAgnih0tri/AlphaRoute-0.8B-v1.5	completed
Kewk/Heretical-Qwen3.5-0.8B	completed
nineninesix/qwen3_5-full-attn-only-14	completed
Qwen/Qwen3.5-0.8B	completed