Papers of the day   All papers

One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers

Comments

Jun 10 2019 Yuandong Tian

Two more arXiv papers regarding to lottery tickets (sparse weight patterns) from our group. 1. Lottery tickets transfer from one dataset to another. https://arxiv.org/abs/1906.02773. @arimorcos is the first author. 2. Lottery tickets also exists in RL and NLP. https://arxiv.org/abs/1906.02768
2 replies, 324 likes


Jun 12 2019 Soumith Chintala

Lottery Initializations: i thought they were overfitting to datasets. In "One ticket to win them all", @arimorcos Haonan Yu @WonderMicky @tydsh show that they generalize across datasets and optimizers. Surprising. More investigation pending as to why... https://twitter.com/tydsh/status/1138184223997587459
5 replies, 198 likes


Jun 10 2019 Michela Paganini

New paper on #LotteryTickets in deep nets & transfer across datasets and optimizers now out on @arxiv_org! "One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers". Work led by @arimorcos at @facebookai ➡️ https://arxiv.org/abs/1906.02773
0 replies, 57 likes


Dec 10 2019 Ari Morcos

Do lottery ticket initializations generalize, or are they overfit to the precise conditions used to generate them? If you're at #NeurIPS2019, come see our poster #170 happening right now! Paper: https://arxiv.org/abs/1906.02773 Blog: https://ai.facebook.com/blog/understanding-the-generalization-of-lottery-tickets-in-neural-networks https://t.co/s9u8Rhho2w
1 replies, 56 likes


Sep 03 2019 Yuandong Tian

4 papers are accepted in NeurIPS. Thanks for all the collaborators! https://arxiv.org/abs/1810.00337 https://arxiv.org/abs/1906.02773 https://arxiv.org/abs/1906.00744 https://arxiv.org/abs/1906.12029
0 replies, 46 likes


Sep 04 2019 Yuandong Tian

2/4: https://arxiv.org/abs/1906.02773 shows lottery tickets initialization generalizes across various optimizers and datasets (Fashion MNIST, SVHN, CIFAR-10/100, ImageNet, Places365) and can be used for training sparse (and small) models on other datasets for free. @arimorcos @WonderMicky
2 replies, 36 likes


Nov 25 2019 Facebook AI

2/4: Do lottery tickets contain generic inductive biases or are they overfit to the particular dataset and optimizer used to find them? Encouragingly, we found that lottery tickets generalize across related, but distinct datasets and across optimizers: https://arxiv.org/abs/1906.02773?fbclid=IwAR0exCzOb_rchjZBJVCWgSaTuJaXRrsE9AQcK4RpfXdEKEpGuUmxw7jcn7w
1 replies, 34 likes


Jun 11 2019 Adam Santoro

I'm loving this line of research. Great work!
1 replies, 15 likes


Nov 25 2019 Ari Morcos

This work was done with a number of collaborators: @tydsh, Haonan Yu, @WonderMicky, Sergey Edunov, Tina Jiang, and Qucheng Gong at @facebookai. Papers discussed below: https://arxiv.org/abs/1906.02773 https://arxiv.org/abs/1906.02768 https://arxiv.org/abs/1905.13405 https://arxiv.org/abs/1909.13458
0 replies, 3 likes


Nov 25 2019 小猫遊りょう(たかにゃし・りょう)

One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers https://arxiv.org/abs/1906.02773 Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP https://arxiv.org/abs/1906.02768
1 replies, 2 likes


Jun 12 2019 Jonathan Frankle

PS. We proposed rewinding in March as "late resetting" in a prevoius version of this paper. We're excited that @irregularized and @arimorcos + team have already found it to be useful in studying winning ticket tranfer learning! https://arxiv.org/abs/1905.07785 https://arxiv.org/abs/1906.02773 https://t.co/MqSazij1lo
0 replies, 2 likes


Dec 02 2019 akira

https://arxiv.org/abs/1906.02773 In the “lottery ticket hypothesis”,only good initial values ​​influence model performance, good initial values ​​can be transferred from one data set to another. They experimented with different models, datasets, and optimizers, but it can be transferred. https://t.co/HipCjEDZqM
0 replies, 1 likes


Content