Completed / CodeSOTA execution on White
Inspect the measurement.
Qwen/Qwen2.5-Coder-1.5B · HumanEval (full 164, serial greedy pass@1) · 37.2% pass@1
Full benchmark split · 164 scored examples. Full 164-problem HumanEval; one greedy 256-token completion per problem; pass@1 is fraction passing original isolated unit tests. FP32; no chat template/BOS. Fixed top-level completion extraction; no retries or repair. Native generation recomputes context per token; HF uses a serial KV cache. Not HumanEval+ or an unrestricted-generation score.
Run ID / completed
2d80929f0ec5f26de33e4a98
17 Oct 2026, 03:35 UTC
17 Oct 2026, 03:35 UTC
Model revision
df3ce67c0e24480f20468b6ef2894622d69eb73bComparison scope
7388be4229d45cb22b346201154469558e2015772bcb2abde7f2fb7b0b7ad294Evaluator SHA-256
496a63534a2dcc2f9bf860166acaaeb0f15f41abd137c89881b8c7674bb321d6Observed resources
526s elapsed · 5.83 GiB peak torch allocation
Dollar cost: not measured
Dollar cost: not measured
Download evidence
Inspect individual examples
Read the recorded prediction, reference and outcome for each example. Code is shown with its original indentation; the reference is the unit-test criterion, not a reference implementation.
- Scored examples
- 164
- Passed
- 61
- Failed
- 103
- Metric
- pass@1
Examples are downloaded only when requested. Search covers all loaded records; 25 examples are displayed per page. Download all examples · JSONL
Example evidence has not been loaded.