Inspect the measurement.

SlayerLab/Slayer139-1.01b · HellaSwag (full validation, character-normalized accuracy) · 36.3% accuracy_char_normalized

Full benchmark split · 10042 scored examples. Full pinned split; primary character-normalized accuracy: argmax summed conditional answer-token log likelihood divided by Python len(choice), excluding the leading delimiter; raw accuracy is a separate diagnostic. Explicit pinned prompt schema, no chat/BOS. Left context truncated only if prompt plus answer exceeds 1024 tokens. Generic text diagnostic does not validate specialist declared tasks.

Run ID / completed

cf6f66ed8c32a855cdf827cc
17 Oct 2026, 15:37 UTC

Model revision

283ffb6d620be2d6ea1f2429d721962dc8e73633

Comparison scope

9ba01dfd8954831947fc9d9e821f5c27a78509f522b87d05d07b1c4f172ae0e6

Evaluator SHA-256

8ba81403e69d93b8121bd05e0ea4f9c55e0dc5c721449f50adfda136d7d976f1

Observed resources

323s elapsed · 0.98 GiB peak torch allocation
Dollar cost: not measured

Download evidence

result.json samples.jsonl environment.json evaluate.py manifest.json

Inspect individual examples

Read the recorded prediction, reference and outcome for each example.

Scored examples: 10,042
Passed: 3,647
Failed: 6,395
Metric: accuracy_char_normalized

Examples are downloaded only when requested. Search covers all loaded records; 25 examples are displayed per page. Download all examples · JSONL

Example evidence has not been loaded.