Most speech SSL models optimize for ASR and hope everything else works out. WavLM, from Microsoft, is built the other way around — pretrain once, do everything: recognition, speaker verification, diarization, separation, the full SUPERB benchmark. The trick is a gated relative-position bias in the transformer plus utterance mixing during pretraining, which forces the model to learn speaker-aware and noise-robust representations rather than just phonetic ones.

The result was a clean sweep of the SUPERB leaderboard when it landed, with especially strong gains on non-ASR tasks that HuBERT and wav2vec 2.0 tend to underserve. This session walks through what utterance mixing actually looks like at training time, why the position bias matters for longer contexts, and how the 94k-hour pretraining corpus was assembled. If you're tired of maintaining a separate encoder for every downstream speech task, WavLM is one of the strongest single-backbone bets available on HuggingFace today — the deep dive is worth queueing up.