CodeSOTAβOriginal site ↗
← All tasks

Speech / LibriSpeech test-clean

Speech transcription

English speech recording → transcript

Model size vs score

Model size versus weropenai/whisper-tiny.en: 37.76 million parameters; score 5.1%. All points use the same pinned protocol.00%921%1742%2663%3584%43105%openai/whisper-tiny.en · 37.76M parameters · 5.1%Model parameters (millions)wer · lower is better

1 compatible measurements. Parameter counts come from the architecture instantiated from each pinned model configuration; no weights are loaded for this count. Click a point to inspect its evidence.

Measured results

Pilot comparison · first 32 valid examples · wer lower is better. No global SOTA claim.

SystemScoreSamplesElapsed¹Evidence
openai/whisper-tiny.en
87c7102498dc
5.1%325sInspect run →
16 Oct 2026, 23:28 UTC

¹ Evaluation elapsed time includes model loading after dataset selection. It is not serving latency or a fair speed comparison.

What this measures

Dataset / split
Immutable dataset revision
71cacbfb7e2354c4226d01e70d77d5fca3d04ba1
Upstream license metadata
cc-by-4.0 · consult upstream terms
Protocol
codesota-pilot-v1 / asr
Selection
First 32 valid examples in the pinned split order. SNLI rows without a valid label are excluded.
Configuration
Seed 42. Greedy generation for math and code. Mean answer-token log likelihood for text classification, completion and declarative multiple-choice recipes. No model judge.
Read scoring and comparison rules →

Planned runs

1 qualified visits in 30 days · priority 42.0. Interest changes execution order, within the approved baseline list.

SystemStatus
openai/whisper-tiny.encompleted