Papers of the day   All papers

The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives

Comments

Sep 16 2019 Lena Voita

Evolution of Representations in the Transformer: blog post on our @emnlp2019 paper is out! blog post: https://lena-voita.github.io/posts/emnlp19_evolution.html paper: https://arxiv.org/abs/1909.01380 @lena_voita, @RicoSennrich, @iatitov https://t.co/doWpZGdGzY
4 replies, 654 likes


Sep 16 2019 Thomas Wolf

A fascinating article by @lena_voita if you're interested in understanding what makes MLM models like BERT differents from LM models like GPT/GPT-2 (auto-regressive) and MT models. And conveyed in such a beautiful blog post, a master-piece of knowledge sharing!
3 replies, 390 likes


Nov 06 2019 Lena Voita

2nd day of @emnlp: Evolution of Representations in the Transformer! 16:30-18:00, hall 2A, poster P43 https://www.aclweb.org/anthology/D19-1448.pdf (another paper with my research parents @iatitov and @RicoSennrich )
0 replies, 44 likes


Sep 16 2019 Christian Szegedy

Super interesting analysis of the evolution of representation inside transformers. It uses information bottleneck and the methods are generic enough to analyze any network.
0 replies, 29 likes


Oct 11 2019 Sebastian Ruder

For more info and analyses, check out the excellent blog post by @lena_voita: https://lena-voita.github.io/posts/emnlp19_evolution.html Paper: https://arxiv.org/abs/1909.01380
0 replies, 28 likes


Sep 16 2019 Machine Learning and NLP

The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives https://arxiv.org/pdf/1909.01380.pdf #NLProc
0 replies, 12 likes


Sep 05 2019 arxiv

The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Trans... http://arxiv.org/abs/1909.01380 https://t.co/dlREiG4WP8
0 replies, 4 likes


Content