Papers of the day   All papers

Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition


Quoc Le: Pretty amazing progress on speech recognition thanks to pre-training and self-training with unlabeled data. Key ingredients: Large conformer architecture + wave2vec2.0 pretraining + Noisy Student Training Link:

4 replies, 539 likes

Loren Lugosch: New SOTA for LibriSpeech using Transducers (+ contrastive pre-training, self-training, and other tricks). Think I might write that Transducer explainer after all.

2 replies, 48 likes

Leo Boytsov: It's a WWW (wild-wild-wild) world! Google and Facebook report nearly simultaneously on success in using self-supervision and self-training in speech recognition. Ouch. 1. 2.

0 replies, 16 likes

Leo Boytsov: Great improvements in ASR thanks to self-supervised training.

1 replies, 13 likes

Ruoming Pang: 1.4%/2.6% on LibriSpeech with Conformer + Noisy Student + Wav2Vec:

0 replies, 9 likes

HoxoMaxwell! 🎃: Conformer 👇 Gulati, Convolution-augmented Transformer for Speech Recognition, 2020

0 replies, 6 likes

Jim Dowling: Amazing to @quocleix do ablation studies on a 1.1bn param network. In a talk with @sinash93 next month at the @Data_AI_Summit , we will explain how we parallelize Ablation studies using Maggy -

1 replies, 4 likes

Daisuke Okanohara: Noisy student training (generate labeled dataset using a trained model, and train a student w/ noise) is also effective in speech recognition tasks, using a giant Conformer (Self-attention+Conv.) model. Achieve new SOTA on the LibriSpeech data set.

0 replies, 4 likes

Takuya Yoshioka: Stunning.

0 replies, 1 likes


0 replies, 1 likes


Found on Oct 21 2020 at

PDF content of a computer science paper: Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition