From OpenAI Whisper to Your In-House ASR Service: Recognizing Name Entities & Domain-Specific Terms
Whisper out of the box is impressive, but anyone who's actually shipped it into production knows the pain: it fumbles proper nouns, product names, medical term…
Olewave at ICASSP 2024
We were a patron and exhibitor at IEEE ICASSP 2024 in Seoul, and our Founder & CEO Wei Chu delivered a spotlight talk on training speech models from wild, web-…
Variational Autoencoder (VAE) and Reparameterization Trick - Revisiting the Classic Generative Model
Before diffusion, before normalizing flows, before latent diffusion models quietly built VAE encoders into every serious generative stack — there was Kingma an…
A Review of Microsoft+OpenAI, Google, Meta, and Nvidia's Open Source Large Speech Models for ASR
Every major AI lab has now shipped an open-source large speech model for ASR, and picking the right one for production has become a real decision.
Tycho:a tookit for building high-ROI in-house speech-related services (ASR/TTS/Translation):Overview
Most teams that want an in-house ASR, TTS, or speech translation service face the same messy choice: glue together fragments of ESPnet, NeMo, k2, and HuggingFa…
From OpenAI's Whisper Model to Your Own In-House ASR Service: Postprocessing and Language Modeling
Raw Whisper output is impressive out of the box, but anyone who has shipped it to real users knows the gap between a demo transcript and something a downstream…
From OpenAI's Whisper Model to Your Own In-House ASR Service: Long Audio and Streaming (Part 3)
Whisper was trained on 30-second chunks, and it shows the moment you feed it a two-hour podcast or try to wire it into a live captioning pipeline.
From OpenAI's Whisper Model to Your Own In-House ASR Service: ROI (Return-on-Investment) (Part 2)
The build-versus-buy question for ASR usually gets framed as accuracy versus convenience, but the real deciding factor is ROI over the lifetime of the product.
From OpenAI's Whisper Model to Your Own In-House ASR Service: Overview (Part 1)
When OpenAI open-sourced Whisper, it quietly reset the baseline for what an in-house ASR service looks like.
Why word timestamps generated by OpenAI Whisper are not accurate? How to make them accurate again?
Ask Whisper for word timestamps and you'll get numbers back — but if you've ever tried to line them up with the actual audio for karaoke, captioning, or dubbin…
Speech Generative AI: VoiceBox by Meta AI (also Flow Matching and Neural ODE)
For years, speech generation lagged text and image generation in both scale and generality — every task got its own bespoke model.
I-JEPA: Yannn LeCun's First 'World Model' for Computer Vision
Yann LeCun has been arguing for years that generative pretraining is the wrong bet for building machines that understand the world.
LoRA: allow a high school student to train Large Language Model (GPT-3) with a gaming graphics card
Full fine-tuning of a 175B-parameter GPT-3 is a non-starter for almost everyone — the storage alone rules it out, never mind the GPUs.
A Review of SpeechT5: Introducing Google's T5 into Speech (ASR, TTS, SID, ...) Tasks
T5 unified text-to-text tasks under a single encoder-decoder pretraining recipe and became a workhorse of NLP.
Review of HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis
For a long time, neural vocoders forced you to pick a side: autoregressive models like WaveNet gave you gorgeous audio but were painfully slow, while GAN-based…
In-depth Review of Google's SoundStream: An End-to-End Neural Audio Codec
Neural codecs quietly became the foundation layer of modern generative audio, and SoundStream is one of the papers that got the ball rolling.
In-depth Review of VALL-E: Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers
Zero-shot TTS from a three-second reference clip sounded like a stretch until VALL-E shipped, and this in-depth review unpacks exactly how Microsoft pulled it …
Non Collision Mispronunciation Addition (NCMA) for Accented ASR
Accented and non-native speech breaks pronunciation dictionaries in ways that quietly poison your acoustic model alignments, and this Interspeech 2021 work ask…
One-Edit-Distance FSA/Network-based (OEDN) in Mispronunciation Detection and Accented ASR
Non-native acoustic modeling has a chicken-and-egg problem: bad phone alignments produce bad models, and bad models produce bad alignments.
![[Olewave's Short Review] Xception: Deep Learning with Depthwise Separable Convolutions](/media/video/2022/07/olewaves-short-review-xception-deep-learning-with-depthwise-separable-convolutions/preview.jpg)
[Olewave's Short Review] Xception: Deep Learning with Depthwise Separable Convolutions
If you just want the core Xception idea without the long-form deep dive, this short review is the express version.
![[Short Review] Conformer: Convolution-augmented Transformer for Speech Recognition](/media/video/2022/06/short-review-conformer-convolution-augmented-transformer-for-speech-recognition/preview.jpg)
[Short Review] Conformer: Convolution-augmented Transformer for Speech Recognition
Conformer took the ASR world by storm by doing something almost boring: gluing a convolution module into each Transformer block so the model captures both glob…
博士大叔使用计算机作弊降维打击2022高考数学压轴大题 Ph.D. uses computer cheating to solve 2022 college entrance math exam
What happens when you turn a research-grade computer algebra toolkit loose on the hardest problem of the 2022 Chinese gaokao math exam?
![[Long Review] Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages](/media/video/2022/05/long-review-wav2seq-pre-training-speech-to-text-encoder-decoder-models-using-pseudo/preview.jpg)
[Long Review] Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages
Most speech pretraining has focused on the encoder side, wav2vec, HuBERT, WavLM, but sequence-to-sequence ASR needs a pretrained decoder too.
![[Short Review] Towards Zero-Label Language Learning](/media/video/2022/05/short-review-towards-zero-label-language-learning/preview.jpg)
[Short Review] Towards Zero-Label Language Learning
What if the labeled data you spent months annotating wasn't necessary at all?
![[Long Review] Deduplicating Training Data Makes Language Models Better](/media/video/2022/04/long-review-deduplicating-training-data-makes-language-models-better/preview.jpg)
[Long Review] Deduplicating Training Data Makes Language Models Better
Deduplication sounds boring until you learn that a single 61-word sentence appears more than 60,000 times inside C4, and that language models trained on such c…
Triplets-like Russian Figure Skaters: Can Kullback-Leibler Divergence be Used Tell Their Difference?
Speaker adaptation is one of those ASR problems that sounds solved until you actually try to adapt a sequence-to-sequence model without wrecking the encoder's …
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
wav2vec 2.0 changed the game with contrastive self-supervision, but HuBERT asked a sharper question: what if we skip the contrastive trick entirely and just do…
W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self Supervise
Why choose between contrastive learning and masked language modeling for speech pretraining when you can wire both into the same network?
Exploring Wav2vec 2.0 fine-tuning for improved speech emotion recognition
Speech emotion recognition has long been stuck with tiny labeled datasets and hand-crafted acoustic features.
Joint Unsupervised and Supervised Training for Multilingual ASR
Pretrain-then-finetune has become the default recipe for multilingual ASR, but it leaves a lot on the table: the two stages don't share losses, and the supervi…