Papers of the day   All papers

Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition

Comments

Quoc Le: Pretty amazing progress on speech recognition thanks to pre-training and self-training with unlabeled data. Key ingredients: Large conformer architecture + wave2vec2.0 pretraining + Noisy Student Training Link: https://arxiv.org/abs/2010.10504 https://t.co/vBIlQWx2Us

4 replies, 539 likes


Loren Lugosch: New SOTA for LibriSpeech using Transducers (+ contrastive pre-training, self-training, and other tricks). Think I might write that Transducer explainer after all. https://arxiv.org/pdf/2010.10504.pdf https://t.co/P3Rk4tzxfP

2 replies, 48 likes


Leo Boytsov: It's a WWW (wild-wild-wild) world! Google and Facebook report nearly simultaneously on success in using self-supervision and self-training in speech recognition. Ouch. 1. https://arxiv.org/abs/2010.10504 2. https://arxiv.org/abs/2010.11430

0 replies, 16 likes


Leo Boytsov: Great improvements in ASR thanks to self-supervised training.

1 replies, 13 likes


Ruoming Pang: 1.4%/2.6% on LibriSpeech with Conformer + Noisy Student + Wav2Vec: https://arxiv.org/abs/2010.10504

0 replies, 9 likes


HoxoMaxwell! 🎃: Conformer 👇 Gulati, Convolution-augmented Transformer for Speech Recognition, 2020 https://arxiv.org/abs/2005.08100 https://t.co/2MnqmqgC0K

0 replies, 6 likes


Jim Dowling: Amazing to @quocleix do ablation studies on a 1.1bn param network. In a talk with @sinash93 next month at the @Data_AI_Summit , we will explain how we parallelize Ablation studies using Maggy - https://databricks.com/session_eu20/parallel-ablation-studies-for-machine-learning-with-maggy-on-apache-spark https://t.co/O1QUyTGTmz

1 replies, 4 likes


Daisuke Okanohara: Noisy student training (generate labeled dataset using a trained model, and train a student w/ noise) is also effective in speech recognition tasks, using a giant Conformer (Self-attention+Conv.) model. Achieve new SOTA on the LibriSpeech data set. https://arxiv.org/abs/2010.10504

0 replies, 4 likes


Takuya Yoshioka: Stunning.

0 replies, 1 likes


DLビギナ: https://twitter.com/hillbig/status/1320510342036746240?12422

0 replies, 1 likes


Content

Found on Oct 21 2020 at https://arxiv.org/pdf/2010.10504.pdf

PDF content of a computer science paper: Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition