Papers of the day   All papers

One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers

Comments

Jun 10 2019 Yuandong Tian

Two more arXiv papers regarding to lottery tickets (sparse weight patterns) from our group. 1. Lottery tickets transfer from one dataset to another. https://arxiv.org/abs/1906.02773. @arimorcos is the first author. 2. Lottery tickets also exists in RL and NLP. https://arxiv.org/abs/1906.02768
2 replies, 324 likes


Jun 12 2019 Soumith Chintala

Lottery Initializations: i thought they were overfitting to datasets. In "One ticket to win them all", @arimorcos Haonan Yu @WonderMicky @tydsh show that they generalize across datasets and optimizers. Surprising. More investigation pending as to why... https://twitter.com/tydsh/status/1138184223997587459
5 replies, 198 likes


Jun 10 2019 Michela Paganini

New paper on #LotteryTickets in deep nets & transfer across datasets and optimizers now out on @arxiv_org! "One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers". Work led by @arimorcos at @facebookai ➡️ https://arxiv.org/abs/1906.02773
0 replies, 57 likes


Sep 03 2019 Yuandong Tian

4 papers are accepted in NeurIPS. Thanks for all the collaborators! https://arxiv.org/abs/1810.00337 https://arxiv.org/abs/1906.02773 https://arxiv.org/abs/1906.00744 https://arxiv.org/abs/1906.12029
0 replies, 46 likes


Sep 04 2019 Yuandong Tian

2/4: https://arxiv.org/abs/1906.02773 shows lottery tickets initialization generalizes across various optimizers and datasets (Fashion MNIST, SVHN, CIFAR-10/100, ImageNet, Places365) and can be used for training sparse (and small) models on other datasets for free. @arimorcos @WonderMicky
2 replies, 36 likes


Jun 11 2019 Adam Santoro

I'm loving this line of research. Great work!
1 replies, 15 likes


Jun 12 2019 Jonathan Frankle

PS. We proposed rewinding in March as "late resetting" in a prevoius version of this paper. We're excited that @irregularized and @arimorcos + team have already found it to be useful in studying winning ticket tranfer learning! https://arxiv.org/abs/1905.07785 https://arxiv.org/abs/1906.02773 https://t.co/MqSazij1lo
0 replies, 2 likes


Content