Independent model directory
Choose a model by its results.
Start with an evaluation. Compare measured performance, model size and execution cost, then inspect the evidence.
Selected evaluation
ARC-Easy (full test split, raw accuracy)
Full pinned split · accuracy · higher is better
Best measured66.58%11 pinned systems
Protocol and scope
- Protocol
codesota-sub1b-arc-easy-standard-v2- Scope
cf50d4b44de811db4aed984c74f66a1eab461af452494ae6c905f9c80d3ffc25- Dataset
- allenai/ai2_arc · test · revision
210d026faf9955653af8916fad021475a3f00453 - Selection
- Full pinned split; configured limit 2376. Actual example counts appear beside every score.
Rows share the current evaluation scope. Different splits, prompts, scoring rules and older scopes are excluded. A subset score describes that subset; it is not a full-benchmark estimate.
| # | Model / pinned source | accuracy | Parameters | GPU-slot time | Evidence |
|---|---|---|---|---|---|
| 1 | SlayerLab/Slayer149-ARCCodeSOTA measurement · 0e6793ecfb85 | 66.58%2376 examples | 149.3MCounted from pinned model | 1.4 min | Inspect run →Manifest |
| 2 | SlayerLab/Slayer139-1.02CodeSOTA measurement · c4298d757ba3 | 65.91%2376 examples | 139.3MCounted from pinned model | 1.2 min | Inspect run →Manifest |
| 3 | Qwen/Qwen2.5-0.5B-InstructCodeSOTA measurement · 7ae557604adf | 65.57%2376 examples | 494.0MCounted from pinned model | 33.4s | Inspect run →Manifest |
| 4 | SlayerLab/Slayer139-1.01CodeSOTA measurement · 2e72c696d7c5 | 63.09%2376 examples | 139.3MCounted from pinned model | 1.2 min | Inspect run →Manifest |
| 5 | SlayerLab/Slayer149-Balanced-ARC-E-ft-v2CodeSOTA measurement · 7d11e8fd6376 | 62.16%2376 examples | 149.3MCounted from pinned model | 1.4 min | Inspect run →Manifest |
| 6 | SlayerLab/Slayer149-Balanced-ARC-E-ft-45500CodeSOTA measurement · e7d4b8dea441 | 59.34%2376 examples | 149.3MCounted from pinned model | 1.5 min | Inspect run →Manifest |
| 7 | SlayerLab/Slayer149CodeSOTA measurement · 7857dd28ec05 | 56.78%2376 examples | 148.9MCounted from pinned model | 1.4 min | Inspect run →Manifest |
| 8 | HuggingFaceTB/SmolLM2-360M-InstructCodeSOTA measurement · a10cc1512eab | 56.52%2376 examples | 361.8MCounted from pinned model | 26.6s | Inspect run →Manifest |
| 9 | SlayerLab/Slayer-149M-Decision-v1CodeSOTA measurement · ccfd7d3d91f3 | 56.36%2376 examples | 149.3MCounted from pinned model | 1.4 min | Inspect run →Manifest |
| 10 | SlayerLab/Slayer139-1.01bCodeSOTA measurement · 283ffb6d620b | 54.71%2376 examples | 139.3MCounted from pinned model | 1.3 min | Inspect run →Manifest |
| 11 | HuggingFaceTB/SmolLM2-135M-InstructCodeSOTA measurement · 12fd25f77366 | 54.00%2376 examples | 134.5MCounted from pinned model | 16.1s | Inspect run →Manifest |
GPU-slot time is reserved evaluator wall time, including setup and CPU work; it is not kernel-active GPU time. Missing cost or parameter measurements stay unknown. Model-card claims and download counts do not become scores.