Inspect the measurement.

mithulaartigala/reasoning-summarizer-qwen3.5-0.8b-preview · ARC-Challenge / Qwen3.5 same-base cohort · 34.9% accuracy

Full benchmark split · 1172 scored examples. Full raw-accuracy text-only diagnostic for a same-base Qwen3.5 cohort. Stock trusted model classes, no chat or images; generic text scores do not validate specialist OCR/classification/retrieval uses. Unique total/text/vision parameters measured after loading; source-reported metrics are not imported.

Run ID / completed

4d3599a2d9c1a839e8fb80e2
17 Oct 2026, 04:44 UTC

Model revision

558fc651bf01bbe7cf306ebf2fb32032d7b4e818

Comparison scope

8877f1f73ee0a3c138937e07b7f6ca917f2c1d914d223e6a8ffa42b9ac8f0a47

Evaluator SHA-256

bd025b3fd12a0b84102a8713ca2af2d08bf6f16178dcaac6fce4a1e720bf1f4c

Observed resources

32s elapsed · 3.83 GiB peak torch allocation
Dollar cost: not measured

Download evidence

result.json samples.jsonl environment.json evaluate.py manifest.json

Inspect individual examples

Read the recorded prediction, reference and outcome for each example.

Scored examples: 1,172
Passed: 409
Failed: 763
Metric: accuracy

Examples are downloaded only when requested. Search covers all loaded records; 25 examples are displayed per page. Download all examples · JSONL

Example evidence has not been loaded.