IBM Granite Speech 4.1 Plus Ties for First on FA-Bench ASR with Timestamps
IBM's open model Granite Speech 4.1 Plus scores a word timestamp F1 of 0.489 on FA-Bench, level with Google Chirp 2 at 0.486. IBM contributed the adapter, and FA-Bench ran every test cell again on its own GPUs.
FA-Bench Leaderboard · github.com/olewave/fa-bench · Paper on arXiv
IBM's Granite Speech 4.1 Plus joined the FA-Bench leaderboard for ASR with word timestamps on November 5, 2026. It scores a word timestamp F1 at 20 ms of 0.489, with a 95% interval of 0.477 to 0.500. Google Chirp 2 scores 0.486, from 0.475 to 0.498. The two intervals overlap and both rank ranges run from 1 to 2, so the two share first place among 17 systems.
Granite Speech 4.1 Plus is an open model under the Apache License 2.0. You run it on your own GPU, so the board shows no price for it. The board lists Google Chirp 2 at $0.96 per audio hour.
Where each one scores higher
Granite Speech scores higher on TIMIT read speech, 0.597 against 0.488 on clean audio and 0.531 against 0.497 with noise added. Chirp 2 scores higher on Buckeye conversation, 0.492 against 0.450 clean and 0.469 against 0.379 with noise added.
All the numbers
| TIMIT clean | TIMIT noisy | Buckeye clean | Buckeye noisy | Mean of 4 | |
|---|---|---|---|---|---|
| Word timestamp F1, Granite Speech 4.1 Plus | 0.597 | 0.531 | 0.450 | 0.379 | 0.489 |
| Word timestamp F1, Google Chirp 2 | 0.488 | 0.497 | 0.492 | 0.469 | 0.486 |
| Word boundary error, Granite Speech 4.1 Plus | 26.1 ms | 31.4 ms | 37.7 ms | 47.3 ms | |
| Word boundary error, Google Chirp 2 | 32.7 ms | 32.6 ms | 26.9 ms | 28.1 ms |
How it reached the board
IBM contributed an adapter for Granite Speech to FA-Bench's open-source code. It came in as pull request #23, with a commit by Zvi Kons that lists IBM's own scores on clean audio. FA-Bench then ran all 20 test cells again on its own GPUs, and every cell came within 0.012 F1 of IBM's numbers. The board shows FA-Bench's run.
According to the pull request, the model's timestamp training drew on TIMIT's training split only, and on no Buckeye audio. So FA-Bench's TIMIT and Buckeye test sets are held out from it. The model marks where each word ends on a 10 ms grid, and each word starts where the word before it ends.
How FA-Bench scores
Audio goes in and timed words come out. A word's start and its end each count as a hit when they land within 20 ms of the boundary a linguist labelled by hand, and a misheard word costs both. The test sets are TIMIT core test and Buckeye test, each clean and with reverb, noise, music and babble added.
Thank you to Zvi Kons and the IBM Granite Speech team for the contribution. Have an ASR system or an aligner? Send a pull request with an adapter. FA-Bench runs every test cell again and publishes the result.
Olewave makes both FA-Bench and Olign, so Olign is listed unranked at the bottom of the table.