Completed / CodeSOTA execution on White
Inspect the measurement.
Qwen/Qwen2.5-Coder-0.5B · HumanEval (full 164, serial greedy pass@1) · 27.4% pass@1
Full benchmark split · 164 scored examples. Full 164-problem HumanEval; one greedy 256-token completion per problem; pass@1 is fraction passing original isolated unit tests. FP32; no chat template/BOS. Fixed top-level completion extraction; no retries or repair. Native generation recomputes context per token; HF uses a serial KV cache. Not HumanEval+ or an unrestricted-generation score.
Run ID / completed
3a1b18f9e6b4ff4550f198af
17 Oct 2026, 03:26 UTC
17 Oct 2026, 03:26 UTC
Model revision
8123ea2e9354afb7ffcc6c8641d1b2f5ecf18301Comparison scope
7388be4229d45cb22b346201154469558e2015772bcb2abde7f2fb7b0b7ad294Evaluator SHA-256
496a63534a2dcc2f9bf860166acaaeb0f15f41abd137c89881b8c7674bb321d6Observed resources
437s elapsed · 1.89 GiB peak torch allocation
Dollar cost: not measured
Dollar cost: not measured
Download evidence
Inspect individual examples
Read the recorded prediction, reference and outcome for each example. Code is shown with its original indentation; the reference is the unit-test criterion, not a reference implementation.
- Scored examples
- 164
- Passed
- 45
- Failed
- 119
- Metric
- pass@1
Examples are downloaded only when requested. Search covers all loaded records; 25 examples are displayed per page. Download all examples · JSONL
Example evidence has not been loaded.