![[Long Review] Fully Sharded Data Parallel: faster AI training with fewer GPUs](/media/video/2022/04/long-review-fully-sharded-data-parallel-faster-ai-training-with-fewer-gpus/preview.jpg)
![[Short Review] Fully Sharded Data Parallel: faster AI training with fewer GPUs](/media/video/2022/04/short-review-fully-sharded-data-parallel-faster-ai-training-with-fewer-gpus/preview.jpg)
Tutorials
[短评] 完全分片数据并行：用更少的GPU进行更快的AI训练
完全分片数据并行是Meta对每个语音和语言团队最终都会提出的问题的答案：我们如何能够训练大一个数量级的模型，而无需配置大一个数量级的GPU？
![[Long Review] Deduplicating Training Data Makes Language Models Better](/media/video/2022/04/long-review-deduplicating-training-data-makes-language-models-better/preview.jpg)
Tutorials
[长篇评论] 数据去重使语言模型更好
数据去重听起来很无聊，直到你了解到一个61词的句子在C4中出现了超过60,000次，以及用这些语料库训练的语言模型会以令人不安的方式向用户吐出训练集中的文本
![[Short Review] Deduplicating Training Data Makes Language Models Better](/media/video/2022/04/short-review-deduplicating-training-data-makes-language-models-better/preview.jpg)
![[Long Review] CLAS: Deep context: end-to-end contextual speech recognition](/media/video/2022/04/long-review-clas-deep-context-end-to-end-contextual-speech-recognition/preview.jpg)
![[Long Review] Hurdles to Progress in Long Form Question Answering](/media/video/2022/04/long-review-hurdles-to-progress-in-long-form-question-answering/preview.jpg)