Completed / CodeSOTA execution on White
Inspect the measurement.
SlayerLab/Slayer149-Balanced-ARC-E-ft-45500 · ARC-Easy (full test split, raw accuracy) · 59.3% accuracy
Full benchmark split · 2376 scored examples. Full ARC-Easy test split (2376 examples); FP32, 1024-token input ceiling, raw summed answer-token log likelihood, zero-shot Question/Answer prompt. Character-normalized accuracy reported separately as acc_norm. Native masked-padding adapter uses unpadded row forwards. Distinct from the earlier token-normalized pilot.
Run ID / completed
7e4ed737ed0c83699e490690
17 Oct 2026, 02:59 UTC
17 Oct 2026, 02:59 UTC
Model revision
e7d4b8dea44125264077c2ce6d2f141379f999cdComparison scope
cf50d4b44de811db4aed984c74f66a1eab461af452494ae6c905f9c80d3ffc25Evaluator SHA-256
ea8141a818424f6e010d35f8e237d2f8682640b6d123ed87438ac01655492c8aObserved resources
84s elapsed · 0.78 GiB peak torch allocation
Dollar cost: not measured
Dollar cost: not measured
Download evidence
Inspect individual examples
Read the recorded prediction, reference and outcome for each example.
- Scored examples
- 2,376
- Passed
- 1,410
- Failed
- 966
- Metric
- accuracy
Examples are downloaded only when requested. Search covers all loaded records; 25 examples are displayed per page. Download all examples · JSONL
Example evidence has not been loaded.