Completed / CodeSOTA execution on White
Inspect the measurement.
Qwen/Qwen2.5-1.5B · HumanEval (full 164, serial greedy pass@1) · 36.0% pass@1
Full benchmark split · 164 scored examples. Full 164-problem HumanEval; one greedy 256-token completion per problem; pass@1 is fraction passing original isolated unit tests. FP32; no chat template/BOS. Fixed top-level completion extraction; no retries or repair. Native generation recomputes context per token; HF uses a serial KV cache. Not HumanEval+ or an unrestricted-generation score.
Run ID / completed
599a2e85e8b3a5c55f01d349
17 Oct 2026, 03:31 UTC
17 Oct 2026, 03:31 UTC
Model revision
8faed761d45a263340a0528343f099c05c9a4323Comparison scope
7388be4229d45cb22b346201154469558e2015772bcb2abde7f2fb7b0b7ad294Evaluator SHA-256
496a63534a2dcc2f9bf860166acaaeb0f15f41abd137c89881b8c7674bb321d6Observed resources
517s elapsed · 5.83 GiB peak torch allocation
Dollar cost: not measured
Dollar cost: not measured
Download evidence
Inspect individual examples
Read the recorded prediction, reference and outcome for each example. Code is shown with its original indentation; the reference is the unit-test criterion, not a reference implementation.
- Scored examples
- 164
- Passed
- 59
- Failed
- 105
- Metric
- pass@1
Examples are downloaded only when requested. Search covers all loaded records; 25 examples are displayed per page. Download all examples · JSONL
Example evidence has not been loaded.