Inspect the measurement.

SlayerLab/Slayer149-Balanced-ARC-E-ft-v2 · BoolQ (full validation, raw likelihood accuracy) · 57.6% accuracy

Full benchmark split · 3270 scored examples. Full pinned split; primary raw summed conditional answer-token log likelihood accuracy; character-normalized accuracy is a separate diagnostic. Explicit pinned prompt schema, no chat/BOS. Left context truncated only if prompt plus answer exceeds 1024 tokens. Generic text diagnostic does not validate specialist declared tasks.

Run ID / completed

e8e7f16c555401bee6232c40
17 Oct 2026, 16:02 UTC

Model revision

7d11e8fd63764baf00c276f586b2aa776648ed33

Comparison scope

7f2a4b46461d9e7fe0763f55ef1a046476cd093351d877ad5515e16812e1bdca

Evaluator SHA-256

8ba81403e69d93b8121bd05e0ea4f9c55e0dc5c721449f50adfda136d7d976f1

Observed resources

71s elapsed · 1.29 GiB peak torch allocation
Dollar cost: not measured

Download evidence

result.json samples.jsonl environment.json evaluate.py manifest.json

Inspect individual examples

Read the recorded prediction, reference and outcome for each example.

Scored examples: 3,270
Passed: 1,883
Failed: 1,387
Metric: accuracy

Examples are downloaded only when requested. Search covers all loaded records; 25 examples are displayed per page. Download all examples · JSONL

Example evidence has not been loaded.