Most speech pretraining has focused on the encoder side, wav2vec, HuBERT, WavLM, but sequence-to-sequence ASR needs a pretrained decoder too. Wav2Seq closes that gap with a clever trick: cluster speech representations into discrete pseudo-tokens, then treat those tokens as a synthetic "pseudo language" and pretrain a full encoder-decoder Transformer to translate speech into it.

This long-form review walks through the Wav2Seq pipeline in detail, how the pseudo-language is constructed, why treating it as a real translation target teaches the decoder something useful, and how the pretrained model transfers to downstream ASR and spoken language understanding tasks. A correction about attention in the seq2seq module is included. If you're working on low-resource ASR, cascaded speech-to-text, or trying to squeeze more out of self-supervised pretraining beyond the encoder, the deep-dive walks through every design choice worth understanding.