Olign Ranks First on TIMIT in the Nyra Speech Timing Benchmark
Olign v1 ranks first of 22 systems on TIMIT in nyra labs' own table, and Olewave's run of its current release, v1.1.3, has a word timing error of 14.2 ms on TIMIT and 23.6 ms on Buckeye.
Olign ยท Nyra Speech Timing Benchmark
Summary
- First on TIMIT. Olign v1 ranks first of the 22 systems in nyra labs' own TIMIT table. It has the lowest word timing error there, 15.9 ms, and the most words within 50 ms of the hand labels.
- Closer still today. Olign v1 is now at release v1.1.3. In Olewave's run of nyra's public protocol, it has a word timing error of 14.2 ms on TIMIT and 23.6 ms on conversational speech from Buckeye.
- Check it yourself. You can rerun both tests with nyra's tool and an Olign API key. We have asked nyra to re-run Olign v1, and their number will be the authoritative one.
The benchmark
Nyra labs published the Nyra Speech Timing Benchmark in October 2026, with a paper and code on GitHub. It measures how close each system's word times come to hand labels, and every system goes through the same protocol. This post covers its read speech and its conversation.
TIMIT is read speech, 1,626 sentences. Buckeye is conversation, 2,008 chunks of up to 20 seconds each, with 80,507 words.
Each word gets a timing error, the average of how far its start and its end land from the hand labels. Word MAE is that error averaged over all the words, in milliseconds. Lower is better. Acc@50 ms is the share of words whose start and end both land within 50 ms of the labels. Higher is better. Nyra treats this share as its headline number.
How we measured
Every row on the two pictures in this post is nyra's own run, except Olign v1, which carries a star. Olign v1 is Olewave's run of release v1.1.3 on the same public protocol, and it holds no place on nyra's leaderboard.
The v1.1.3 numbers were run by Olewave on the released v1.1.3 configuration. We cut nyra's exact chunks from our licensed copies of TIMIT and Buckeye, sent the reference words as the transcript, and scored the output with nyra's public scorer at commit 8acf80f.
Nyra keeps its reference times private, since neither corpus may be
redistributed. So we rebuilt them from the corpus files, the TIMIT .WRD files
and the Buckeye .words files. Our reference covers 1,625 of nyra's 1,626
TIMIT sentences and 79,871 of its 80,507 Buckeye words.
Nyra's table lists Olign v1 at 15.9 ms on TIMIT and 35.7 ms on Buckeye. Nyra ran an earlier release in October. Scored against our reference, those same predictions give 15.3 ms and 35.3 ms. So our reference reads about half a millisecond below nyra's, and you can read Olign v1's bars with that margin in mind.
Olign v1 at release v1.1.3 on nyra's benchmark
Olewave's runs, TIMIT on November 15, 2026 and Buckeye on November 11, 2026
| TIMIT | Buckeye | |
|---|---|---|
| Word MAE | 14.2 ms | 23.6 ms |
| Median word timing error | 7.9 ms | 8.7 ms |
| Acc@50 ms | 86.2 % | 86.8 % |
| Acc@100 ms | 96.9 % | 94.7 % |
Words that come back with no length are scored where they were placed.
TIMIT, where Olign ranks first
In nyra's TIMIT table, Olign v1 ranks first of 22 systems. Its word MAE is 15.9 ms. Nyra's own aligner follows at 21.3 ms, and MFA with a wide beam at 23.2 ms. Olign v1 also has the most words within 25, 50 and 100 ms of the hand labels, and it ties for first at 200 ms.
On release v1.1.3, Olewave's run brings the word MAE to 14.2 ms.
The picture puts that run, starred, next to every other system in nyra's TIMIT table, as nyra published it on November 10, 2026. Systems tagged ASR are timed only on the words they recognized, so nyra asks you to read them together with word recall. You will find it in the table under the picture.
All the numbers
| System | Nyra's name | Type | Word recall | Word MAE | Acc@50 ms |
|---|---|---|---|---|---|
| Olign v1 โ Olewave | forced aligner | 14.2 ms | 86.2 % | ||
| nyra-FA nyra labs | nyra_fa | forced aligner | 99.9 % | 21.3 ms | 82.2 % |
| MFA, wide beam โ | mfa (wide beam) | forced aligner | 99.9 % | 23.2 ms | 77.7 % |
| MFA, default beam โ | mfa (default beam) | forced aligner | 99.9 % | 23.5 ms | 77.7 % |
| CrisperWhisper 2.0 large nyra labs | crisperwhisper_2.0_large | ASR system | 98.8 % | 29.6 ms | 70.1 % |
| Qwen3-FA Qwen | qwen_fa | forced aligner | 100.0 % | 30.1 ms | 70.8 % |
| Fish Transcribe 1 Pro Fish Audio | fish_transcribe_1_pro | ASR system | 98.9 % | 30.4 ms | 70.2 % |
| Grok STT xAI | xai_grok_stt | ASR system | 98.3 % | 37.3 ms | 51.3 % |
| MMS-FA Meta | mms_fa | forced aligner | 100.0 % | 37.3 ms | 51.3 % |
| ElevenLabs Cue | Cue (ElevenLabs) | forced aligner | 100.0 % | 44.6 ms | 32.5 % |
| MWA, TIMIT model MLSpeech | mwa_timit | forced aligner | 100.0 % | 47.2 ms | 53.5 % |
| CTC segmentation | ctc_segmentation | forced aligner | 100.0 % | 49.3 ms | 38.9 % |
| Universal 3.5 Pro AssemblyAI | assemblyai_u35_pro | ASR system | 97.9 % | 51.3 ms | 33.6 % |
| Universal 2, disfluencies AssemblyAI | assemblyai_u2_disfl | ASR system | 98.5 % | 53.0 ms | 27.5 % |
| MAI-Transcribe-2 Microsoft | mai_transcribe_2 | ASR system | 99.3 % | 54.9 ms | 27.6 % |
| MWA, Buckeye model MLSpeech | mwa_buckeye | forced aligner | 100.0 % | 58.9 ms | 49.8 % |
| Nemotron Transcribe NVIDIA | nemotron_transcribe | ASR system | 98.9 % | 60.0 ms | 23.4 % |
| NeMo-FA NVIDIA | nemo_fa | forced aligner | 100.0 % | 60.0 ms | 19.3 % |
| Nova-3 Deepgram | deepgram_nova3 | ASR system | 98.2 % | 63.4 ms | 20.5 % |
| WhisperX | whisperx | ASR system | 98.2 % | 64.8 ms | 13.5 % |
| Canary NVIDIA | canary | ASR system | 97.6 % | 85.5 ms | 13.2 % |
| Ink-Whisper Cartesia | cartesia_ink_whisper | ASR system | 98.2 % | 88.3 ms | 20.7 % |
โ Olign v1 is Olewave's run of release v1.1.3 on November 15, 2026, with nyra's public scorer on nyra's chunks. It holds no place on nyra's leaderboard. โ MFA 3.3.9 with the english_mfa 3.1.0 model, as nyra runs it.
Buckeye
Buckeye is the harder test. Nyra cuts each interview into chunks of up to 20 seconds, exactly at the first and the last word. Each chunk keeps its laughter and its long pauses, and often the interviewer's voice. Earlier Olign releases made most of their Buckeye error at the chunk edges and next to pauses.
In nyra's Buckeye table, nyra's own aligner has the lowest word MAE, 20.2 ms. On release v1.1.3, Olewave's run of Olign v1 gives 23.6 ms. Chunks of 15 seconds or less score 22.9 ms, and longer chunks score 23.8 ms. The picture shows nyra's Buckeye table the same way, with Olign v1 starred.
All the numbers
| System | Nyra's name | Type | Word recall | Word MAE | Acc@50 ms |
|---|---|---|---|---|---|
| nyra-FA nyra labs | nyra_fa | forced aligner | 100.0 % | 20.2 ms | 88.7 % |
| Olign v1 โ Olewave | forced aligner | 23.6 ms | 86.8 % | ||
| Fish Transcribe 1 Pro Fish Audio | fish_transcribe_1_pro | ASR system | 93.2 % | 35.5 ms | 75.8 % |
| MFA, wide beam โ | mfa (wide beam) | forced aligner | 99.9 % | 35.7 ms | 79.1 % |
| CrisperWhisper 2.0 large nyra labs | crisperwhisper_2.0_large | ASR system | 93.2 % | 38.8 ms | 66.1 % |
| Qwen3-FA Qwen | qwen_fa | forced aligner | 99.9 % | 39.4 ms | 75.0 % |
| Grok STT xAI | xai_grok_stt | ASR system | 92.9 % | 48.8 ms | 50.5 % |
| MFA, default beam โ | mfa (default beam) | forced aligner | 98.7 % | 48.9 ms | 78.3 % |
| ElevenLabs Cue | Cue (ElevenLabs) | forced aligner | 100.0 % | 50.8 ms | 32.4 % |
| MMS-FA Meta | mms_fa | forced aligner | 100.0 % | 52.7 ms | 49.9 % |
| MAI-Transcribe-2 Microsoft | mai_transcribe_2 | ASR system | 91.7 % | 54.4 ms | 31.0 % |
| Nemotron Transcribe NVIDIA | nemotron_transcribe | ASR system | 93.8 % | 60.1 ms | 33.4 % |
| Universal 3.5 Pro AssemblyAI | assemblyai_u35_pro | ASR system | 88.7 % | 68.1 ms | 30.9 % |
| NeMo-FA NVIDIA | nemo_fa | forced aligner | 100.0 % | 72.6 ms | 20.4 % |
| CTC segmentation | ctc_segmentation | forced aligner | 100.0 % | 81.0 ms | 48.8 % |
| Nova-3 Deepgram | deepgram_nova3 | ASR system | 92.7 % | 87.6 ms | 14.8 % |
| WhisperX | whisperx | ASR system | 88.1 % | 89.8 ms | 8.7 % |
| Canary NVIDIA | canary | ASR system | 88.2 % | 107.2 ms | 10.0 % |
| MWA, Buckeye model MLSpeech | mwa_buckeye | forced aligner | 100.0 % | 108.7 ms | 58.9 % |
| MWA, TIMIT model MLSpeech | mwa_timit | forced aligner | 100.0 % | 143.2 ms | 51.3 % |
| Ink-Whisper Cartesia | cartesia_ink_whisper | ASR system | 88.6 % | 144.8 ms | 35.3 % |
โ Olign v1 is Olewave's run of release v1.1.3 on November 11, 2026, with nyra's public scorer on nyra's chunks. It holds no place on nyra's leaderboard. โ MFA 3.3.9 with the english_mfa 3.1.0 model, as nyra runs it.
What is in release v1.1.3
Since nyra's run, Olign v1 has moved through releases v1.1.0 to v1.1.3. The API and the response format are unchanged.
- Words outside Olign's dictionary get their pronunciation from a pronunciation model.
- Recordings longer than 15 seconds, and recordings of any length sent without a transcript, are handled differently at their start and end.
- A word next to a pause ends or begins in a new place. The word before a pause no longer runs into the pause, so pauses come back closer to their full length.
The last two changes address the chunk edges and the pauses, the two places where Buckeye cost earlier releases the most. A recording of 15 seconds or less sent with a transcript is aligned in one pass.
Check it yourself
You can rerun both tests through the hosted service with nyra's own tool. You
need an Olign API key, which comes with an account at
tycho.olewave.com, and your own licensed
copies of TIMIT and Buckeye. Point --data-root at them if they are not in
nyra's default folder. The two runs send about 8.6 hours of audio, 1.4 hours of
TIMIT and 7.2 hours of Buckeye.
Run nyra's benchmark on Olign v1
git clone https://github.com/nyrahealth/nyra_speech_timing_benchmark
cd nyra_speech_timing_benchmark
pip install -e .
export OLIGN_KEY=your-olign-key
nyra-timing run --dataset timit --model olign_v1 \
--out results_api --save-predictions predictions
nyra-timing run --dataset buckeye --model olign_v1 \
--out results_api --save-predictions predictions
Each run prints its scores and saves the service's raw responses. Every Olign
response carries its release in the X-Olign-Version header, so you can see
which release answered.
Check which release answered
python - <<'EOF'
import gzip, json
for name in ("timit", "buckeye"):
raw = json.load(gzip.open(f"predictions/raw/olign_v1_{name}.json.gz"))
print(name, {v for job in raw.values()
for k, v in job.get("_versions", {}).items()
if k.lower() == "x-olign-version"})
EOF
It prints {'v1.1.3'} for each test set when v1.1.3 aligned every clip. Your
scores use your own copy of the reference, so expect them to differ from ours
in the decimals.
Next steps
-
Our request for nyra's re-run
Our pull request asks nyra to re-run Olign v1 and to record which release answered each hosted row. When nyra publishes its run, that is the number to use, and we will link it here. We have also opened an issue on nyra's MFA rows. -
Try Olign on your own recording
The Speech Timestamps Playground aligns up to 30 seconds of English speech, with no account. -
Start with the Olign API
Sign up and get free credits to align your own recordings. -
See FA-Bench
Olewave's own public benchmark of forced alignment and ASR with timestamps.