Completed / CodeSOTA execution on White
Inspect the measurement.
Qwen/Qwen2.5-0.5B · HumanEval (full 164, serial greedy pass@1) · 28.7% pass@1
Full benchmark split · 164 scored examples. Full 164-problem HumanEval; one greedy 256-token completion per problem; pass@1 is fraction passing original isolated unit tests. FP32; no chat template/BOS. Fixed top-level completion extraction; no retries or repair. Native generation recomputes context per token; HF uses a serial KV cache. Not HumanEval+ or an unrestricted-generation score.
Run ID / completed
227324969d59c1a426063471
17 Oct 2026, 03:18 UTC
17 Oct 2026, 03:18 UTC
Model revision
060db6499f32faf8b98477b0a26969ef7d8b9987Comparison scope
7388be4229d45cb22b346201154469558e2015772bcb2abde7f2fb7b0b7ad294Evaluator SHA-256
496a63534a2dcc2f9bf860166acaaeb0f15f41abd137c89881b8c7674bb321d6Observed resources
342s elapsed · 1.89 GiB peak torch allocation
Dollar cost: not measured
Dollar cost: not measured
Download evidence
Inspect individual examples
Read the recorded prediction, reference and outcome for each example. Code is shown with its original indentation; the reference is the unit-test criterion, not a reference implementation.
- Scored examples
- 164
- Passed
- 47
- Failed
- 117
- Metric
- pass@1
Examples are downloaded only when requested. Search covers all loaded records; 25 examples are displayed per page. Download all examples · JSONL
Example evidence has not been loaded.