Self-supervised pretraining like wav2vec 2.0 gave us great speech encoders, but if you want a full encoder-decoder ASR system, the decoder was still starting from scratch. Wav2Seq fixes that by inventing a pseudo language, discrete cluster IDs derived from speech features, and pretraining the whole seq2seq Transformer to translate audio into it.

This short review distills the Wav2Seq recipe into the parts that matter, how the pseudo tokens are built, why the resulting pretraining task actually helps downstream ASR and spoken NLU, and how it fits alongside encoder-only pretraining approaches. A quick correction on seq2seq attention is included. Hit play for the fast version if you're evaluating pretraining strategies for low-resource speech-to-text.