Papers of the day   All papers

Episodic Memory in Lifelong Language Learning

Comments

DeepMind: Our new work uses episodic memory to learn language tasks in a lifelong setting. The episodic memory is used in two ways: sparse experience replay during training and local adaptation during inference. Paper: http://arxiv.org/abs/1906.01076 Cyprien, @seb_ruder, @ikekong & @DaniYogatama https://t.co/ST3FjUfadG

2 replies, 692 likes


Sebastian Ruder: 1/ Our paper Episodic Memory in Lifelong Language Learning with Cyprien de Masson d'Autume, @ikekong, and @DaniYogatama was accepted to @NeurIPSConf. We go beyond MTL and tackle lifelong learning where models need to acquire new information continually: https://arxiv.org/abs/1906.01076 https://t.co/SCcz9hgGm8

3 replies, 413 likes


Sebastian Ruder: In our new paper (my first collaboration at DeepMind, yay!) with Cyprien, @ikekong, & @DaniYogatama, we leverage episodic memory during training (sparse replay) and inference (local adaptation) for continual learning (on QA and classification tasks). https://arxiv.org/abs/1906.01076 https://t.co/ZHdl3yAu72

2 replies, 260 likes


Julian Harris: "Long term" memory is a key research challenge in #NLProc Episodic Memory in Lifelong Language Learning: local adaptation, tested against 8 datasets and #BERT w/ one GPU by @DaniYogatama @ikekong d'Autume @seb_ruder at @DeepMindAI https://arxiv.org/abs/1906.01076 From @NeurIPSConf '19 https://t.co/jkSOTVNxbW

1 replies, 187 likes


Dani Yogatama: We think episodic memory is a key component of general linguistic intelligence models. In this work, we use it for sparse experience replay and local adaptation to mitigate catastrophic forgetting in textcat and QA: https://arxiv.org/abs/1906.01076 (with Cyprien, @seb_ruder , @ikekong)

0 replies, 149 likes


Thomas Scialom: Episodic Memory in Lifelong Language Learning https://arxiv.org/pdf/1906.01076.pdf Motivation: prevent catastrophic forgettingĀ in neural networks The authors proposed an episodic memory model that performs sparse experience replay and local adaptation with the following components:

3 replies, 102 likes


arXiv CS-CL: Episodic Memory in Lifelong Language Learning http://arxiv.org/abs/1906.01076

0 replies, 6 likes


IntuitionMachine: Paper on reducing catastrophic forgetting in language learning (from DeepMind) https://arxiv.org/abs/1906.01076 https://t.co/6RBu5rN91v

0 replies, 3 likes


Deepak Garasangi: And you all should definitely check his new paper on episodic memory here ! https://arxiv.org/abs/1906.01076

1 replies, 2 likes


Content

Found on Oct 24 2019 at https://arxiv.org/pdf/1906.01076.pdf

PDF content of a computer science paper: Episodic Memory in Lifelong Language Learning