Key Takeaways
- IBM’s 470M-parameter Granite Speech 5.0 Turbo CTC models transcribe 3.5 hours of audio per second on an H200 GPU.
- Encoder-only design with 16 Conformer blocks cuts token rate from 50 to 12.5 per second, delivering >20x throughput gain.
- Apache 2.0 and non-commercial variants lead far-field ASR speed while maintaining competitive accuracy.
Table of Contents
A 470M Parameter Model That Transcribes 3.5 Hours of Audio in One Second
IBM’s Granite speech team has shipped two compact English speech recognition models that deliver more than 12,600 RTFx on an NVIDIA H200 GPU. That translates to roughly 3.5 hours of transcribed audio per second under batched inference.
Hugging Face published the technical dossier on August 25, 2026, detailing the 470M-parameter Granite Speech 5.0 Turbo CTC models. The pair splits along licensing lines: one carries Apache 2.0 terms, while the other uses CC-BY-NC-SA-4.0 for non-commercial use.
The commercial-friendly variant is named granite-speech-5.0-470m-turboctc. The non-commercial sibling ends with an -nc suffix and trains on additional speech data.
Both models target high-volume speech-to-text workloads. They also include a WebGPU demo that runs streaming recognition in Chrome or Edge browsers, according to Hugging Face.
Encoder-Only Design, Conformer Stack, and a Radical Token Rate Cut
Benchmark Profile
On the OpenASR Leaderboard’s public English short-form test sets, the non-commercial variant posts an aggregate 4.85% word error rate. The Apache 2.0 model lands at 5.00%.
Those results are unofficial but were scored with the same tooling as the leaderboard. The figures are expected to match the official public results.
Far-field ASR results add more context. As of August 25, 2026, the non-commercial model placed fifth in accuracy and the Apache model ninth, while both ranked as the fastest entries on the FFASR Leaderboard.
Encoder Architecture
Granite Speech 5.0 Turbo CTC departs from earlier Granite Speech models by removing the language model and projector entirely. The encoder-only design keeps a 470M-parameter footprint while delivering more than 20x faster throughput than prior generations.
The stack uses 16 Conformer blocks with chunkwise attention to avoid quadratic sequence scaling. Self-conditioning is applied after the eighth block, and connectionist temporal classification loss drives training.
A key change is token rate. Previous encoders emitted 50 characters per second; the new models generate 12.5 tokens per second.
The non-commercial model uses SentencePiece tokenization. The Apache 2.0 model uses BPE tokenization.
Three stages of 2x subsampling bring the 100-frame-per-second log Mel spectrogram front end down to 12.5 tokens per second. The first stage stacks Mel vectors with a reshape operation.
The next two stages are built into the first two Conformer blocks as strided convolutions with stride 2 and mean-pooled residuals. This design preserves strong transcription performance while sharply lowering compute cost.
Training Data
Training data mixes natural and synthetic corpora. Shared natural sources include MLS, YODAS, CommonVoice-17, LibriSpeech, VoxPopuli, AMI, and Earnings-22.
The non-commercial variant adds GigaSpeech and SPGI Speech. Synthetic training includes 2,000 hours of multi-speaker concatenations from several base datasets.
Another 500 hours come from Earnings-22 segments. A further 240 hours cover numbers, currencies, web addresses, phone formats, and dotted values generated with the gpt-oss family and synthesized via StyleTTS2.
What Open-Weight Speed at This Size Means for Edge AI
The immediate strategic signal is that sub-billion-parameter speech recognition can now clear a threshold that previously justified larger, slower, and often closed models. By combining 20x throughput gains with competitive word error rates, IBM’s Granite team has made on-device, real-time transcription a practical default rather than an experimental edge case.
The licensing split is deliberate. Apache 2.0 removes friction for commercial integration, while the non-commercial variant trades a small WER improvement for restricted use. Product teams now have a clear commercial path without accepting meaningfully weaker accuracy.
The encoder-only tradeoff is equally important. These models drop speech translation and keyword biasing, capabilities that require the older language-model-equipped architecture. That narrows the use case to pure speech-to-text, but it also concentrates optimization on exactly the function most edge applications need.
For AI teams, the implication is that model size no longer predicts transcription latency in the same way. A 470M model running at 12,600 RTFx on an H200 can handle batch volumes that previously demanded far larger hardware footprints.
Integration bottlenecks now shift toward tokenization, front-end processing, and streaming pipeline design rather than raw model compute. That is a meaningful reset for edge speech products.
The far-field results also suggest that speed does not require sacrificing robustness in noisy or reverberant conditions. Both models remained competitive while leading in throughput, which strengthens the case for deployment in real-world voice interfaces and meeting transcription stacks.
Real-Time ASR Becomes a Baseline, Not a Benchmark
Granite Speech 5.0 Turbo CTC resets the performance floor: real-time factor is no longer a differentiator when an open 470M model transcribes hours of audio in a second. The next competitive gap is pipeline engineering, streaming latency, and licensing clarity.
For teams building AI content pipelines that need to scale, programmatic SEO AI automation is how Andres SEO Expert approaches scalable technical publishing — contact us to map your next move.
Frequently Asked Questions
What is Granite Speech 5.0 Turbo CTC and how fast is it?
Granite Speech 5.0 Turbo CTC is a pair of compact English speech recognition models from IBM with 470 million parameters. On an NVIDIA H200 GPU, they achieve more than 12,600 RTFx, transcribing approximately 3.5 hours of audio per second under batched inference.
What are the licensing differences between the two Granite Speech 5.0 models?
The models are identical in architecture but split along licensing: one uses Apache 2.0 for commercial use, while the other uses CC-BY-NC-SA-4.0 for non-commercial use and includes additional speech data, slightly improving accuracy.
What architectural changes allow Granite Speech 5.0 Turbo CTC to be so fast?
It removes the language model and projector entirely, using an encoder-only design with 16 Conformer blocks and chunkwise attention. It also cuts token rate from 50 characters per second to 12.5 tokens per second and uses three stages of 2x subsampling.
What training data was used for the non-commercial Granite Speech 5.0 model?
Training data includes natural speech sources such as MLS, YODAS, CommonVoice-17, LibriSpeech, VoxPopuli, AMI, Earnings-22, plus GigaSpeech and SPGI Speech. It also uses 2,000 hours of synthetic multi-speaker concatenations and additional synthetic segments.
What are the tradeoffs of the encoder-only design?
The encoder-only design drops speech translation and keyword biasing, narrowing the use case to pure speech-to-text. However, it concentrates optimization on transcription, delivering more than 20x faster throughput while maintaining competitive word error rates.
What does Granite Speech 5.0 Turbo CTC mean for edge AI and real-time ASR?
It resets the performance floor for real-time ASR, making on-device transcription a practical default. Real-time factor is no longer a differentiator; the competitive gap shifts to pipeline engineering, streaming latency, and licensing clarity.
