Independent measured model card
SlayerLab/Slayer149-ARC
Hugging Face source ↗ · JSON for agents
149.3M unique parameters (0e6793ecfb85)
Every result links to its execution evidence. Compare scores only within matching protocols; no overall best-model claim. GPU-slot time includes evaluator setup and CPU work.
All current benchmark results
| Benchmark / coverage | Result | Version / protocol | GPU-slot time | Evidence |
|---|---|---|---|---|
| ARC-Challenge subset · 32 items | 40.62% accuracy · max | 0e6793ecfb85codesota-sub1b-choice-v1 | 5.0s | Inspect run · Manifest |
| ARC-Easy subset · 32 items | 78.12% accuracy · max | 0e6793ecfb85codesota-sub1b-choice-v1 | 5.1s | Inspect run · Manifest |
| ARC-Easy (full test split, raw accuracy) full · 2,376 items | 66.58% accuracy · max | 0e6793ecfb85codesota-sub1b-arc-easy-standard-v2 | 86.8s | Inspect run · Manifest |
| BoolQ subset · 32 items | 50.00% accuracy · max | 0e6793ecfb85codesota-sub1b-choice-v1 | 7.2s | Inspect run · Manifest |
| HellaSwag subset · 32 items | 34.38% accuracy · max | 0e6793ecfb85codesota-sub1b-choice-v1 | 5.1s | Inspect run · Manifest |
| HumanEval (full 164, serial greedy pass@1) full · 164 items | 0.61% pass@1 · max | 0e6793ecfb85codesota-humaneval-full-v2 | 589.4s | Inspect run · Manifest |
| OpenBookQA subset · 32 items | 37.50% accuracy · max | 0e6793ecfb85codesota-sub1b-choice-v1 | 5.0s | Inspect run · Manifest |