Google Researcher's In-Depth Analysis on End-to-End Speech Recognition, Part 1: Overview & Modeling
A decade of deep learning cut ASR word error rates by more than 50% relative and, along the way, made classical HMM pipelines look like a relic.
Deduct OpenAI GPT-4o's Neural Network Architecture
OpenAI hasn't published a paper on GPT-4o, so this video does the next best thing: it reverse-engineers what the neural network architecture almost certainly l…
Google's Universal Speech Model for 100+ languages beats OpenAI's Whisper Model
Google's Universal Speech Model quietly did something remarkable: matched or beat OpenAI's Whisper on ASR across 100+ languages while using roughly one-seventh…
![[Detailed Paper Reading] Zipformer: A faster and better encoder for automatic speech recognition](/media/video/2024/02/detailed-paper-reading-zipformer-a-faster-and-better-encoder-for-automatic-speech/preview.jpg)
[Detailed Paper Reading] Zipformer: A faster and better encoder for automatic speech recognition
The Conformer has been the default ASR encoder for years, but the k2/icefall team's Zipformer quietly took the WER crown on LibriSpeech, Aishell-1, and WenetSp…
In-depth Review of Google's SoundStream: An End-to-End Neural Audio Codec
Neural codecs quietly became the foundation layer of modern generative audio, and SoundStream is one of the papers that got the ball rolling.
![[Olewave's Review] AudioLM: a Language Modeling Approach to Audio Generation](/media/video/2023/03/olewaves-review-audiolm-a-language-modeling-approach-to-audio-generation/preview.jpg)
[Olewave's Review] AudioLM: a Language Modeling Approach to Audio Generation
AudioLM is the paper that convinced a lot of people audio generation should look more like language modeling than signal processing.
Olewave's most detailed illustration of RNN-T: Sequence Transduction with Recurrent Neural Networks
Alex Graves' RNN-T paper is the quiet ancestor of nearly every streaming ASR system shipping today, from Google's on-device recognizer to countless open-source…
Google fired Blake Lemoine for saying AI bot is sentient? Does LaMDA or ChatGPT think like human?
When Google fired Blake Lemoine for going public with claims that LaMDA had become sentient, the story went viral, but the underlying question stuck around: do…
![[Olewave's Long Review] Xception: Deep Learning with Depthwise Separable Convolutions](/media/video/2022/07/olewaves-long-review-xception-deep-learning-with-depthwise-separable-convolutions/preview.jpg)
[Olewave's Long Review] Xception: Deep Learning with Depthwise Separable Convolutions
Xception took the Inception hypothesis to its logical extreme by pushing cross-channel and spatial correlations into fully separate operations, and the depthwi…
![[Olewave's Short Review] Xception: Deep Learning with Depthwise Separable Convolutions](/media/video/2022/07/olewaves-short-review-xception-deep-learning-with-depthwise-separable-convolutions/preview.jpg)
[Olewave's Short Review] Xception: Deep Learning with Depthwise Separable Convolutions
If you just want the core Xception idea without the long-form deep dive, this short review is the express version.
![[Long Review] CLAS: Deep context: end-to-end contextual speech recognition](/media/video/2022/04/long-review-clas-deep-context-end-to-end-contextual-speech-recognition/preview.jpg)
[Long Review] CLAS: Deep context: end-to-end contextual speech recognition
Anyone who has shipped an ASR product knows the pain of proper nouns: contact names, song titles, medication names, obscure place names.
![[Long Review] Hurdles to Progress in Long Form Question Answering](/media/video/2022/04/long-review-hurdles-to-progress-in-long-form-question-answering/preview.jpg)
[Long Review] Hurdles to Progress in Long Form Question Answering
Long-form question answering is one of those benchmarks that looks impressive on the leaderboard and disturbing under a microscope.
![[Long Review] Finetuned Language Models Are Zero-Shot Learners](/media/video/2022/03/long-review-finetuned-language-models-are-zero-shot-learners/preview.jpg)
[Long Review] Finetuned Language Models Are Zero-Shot Learners
Before InstructGPT made instruction tuning a household phrase, Google's FLAN paper made the case that a modest amount of multi-task fine-tuning on natural-lang…
![[Long Review] 'GShard': Scaling Giant Models with Conditional Computation and Automatic Sharding](/media/video/2022/02/long-review-gshard-scaling-giant-models-with-conditional-computation-and-automatic/preview.jpg)
[Long Review] 'GShard': Scaling Giant Models with Conditional Computation and Automatic Sharding
Scaling a Transformer past a hundred billion parameters gets philosophical fast: do you activate the whole network for every token, or route each token to the …