Review DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generation ~ Seed-TTS

Tutorials

Review DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generation ~ Seed-TTS

The Seed-TTS team is back with DiTAR, and it's a clean answer to one of the more awkward tradeoffs in modern speech generation: how do you keep the flexibility of continuous speech representations without paying the comp

The Seed-TTS team is back with DiTAR, and it's a clean answer to one of the more awkward tradeoffs in modern speech generation: how do you keep the flexibility of continuous speech representations without paying the compute tax that hybrid diffusion-plus-autoregressive stacks usually demand, and without settling for the suboptimal outputs those combinations tend to produce? DiTAR sidesteps discrete speech tokens entirely, using a patch-based framework that pairs a language model with a diffusion transformer to generate continuous audio efficiently.

The review digs into the divide-and-conquer trick at the heart of the model โ€” the LM chews on aggregated patch embeddings and hands off to the diffusion transformer to produce the next patch conditioned on that output โ€” plus a neat inference-time knob that redefines temperature as the noise-injection point in the reverse diffusion ODE, giving you a clean dial between diversity and determinism. Extensive scaling analysis, zero-shot benchmarks on robustness, speaker similarity, and naturalness are all in scope, with DiTAR posting SOTA numbers across the board. If you're tracking where autoregressive speech generation is headed after VALL-E and Seed-TTS, this deep-dive is worth queueing up.