Completed / CodeSOTA execution on White
Inspect the measurement.
HuggingFaceTB/SmolLM2-135M-Instruct · HellaSwag (full validation, character-normalized accuracy) · 42.9% accuracy_char_normalized
Full benchmark split · 10042 scored examples. Full pinned split; primary character-normalized accuracy: argmax summed conditional answer-token log likelihood divided by Python len(choice), excluding the leading delimiter; raw accuracy is a separate diagnostic. Explicit pinned prompt schema, no chat/BOS. Left context truncated only if prompt plus answer exceeds 1024 tokens. Generic text diagnostic does not validate specialist declared tasks. Recovered saved result after oversized-upload failure; no repeated inference. Resource slot includes original failed upload/retries.
Run ID / completed
11885f99269d479ce0c75faf
17 Oct 2026, 15:10 UTC
17 Oct 2026, 15:10 UTC
Model revision
12fd25f77366fa6b3b4b768ec3050bf629380bacComparison scope
9ba01dfd8954831947fc9d9e821f5c27a78509f522b87d05d07b1c4f172ae0e6Evaluator SHA-256
8ba81403e69d93b8121bd05e0ea4f9c55e0dc5c721449f50adfda136d7d976f1Observed resources
103s elapsed · 1.00 GiB peak torch allocation
Dollar cost: not measured
Dollar cost: not measured
Download evidence
Inspect individual examples
Read the recorded prediction, reference and outcome for each example.
- Scored examples
- 10,042
- Passed
- 4,308
- Failed
- 5,734
- Metric
- accuracy_char_normalized
Examples are downloaded only when requested. Search covers all loaded records; 25 examples are displayed per page. Download all examples · JSONL
Example evidence has not been loaded.