Papers of the day   All papers

EXTREME LANGUAGE MODEL COMPRESSION WITH OPTIMAL SUBWORDS AND SHARED PROJECTIONS

Comments

Sep 28 2019 Miles Brundage

Totally crazy that you can squeeze BERT down to 7 MB without much drop in performance - "Extreme Language Model Compression with Optimal Subwords and Shared Projections," Zhao et al.: https://arxiv.org/abs/1909.11687
8 replies, 950 likes


Sep 27 2019 Machine Learning and NLP

Extreme language model compression with optimal subwords and shared projections https://arxiv.org/pdf/1909.11687.pdf #NLProc
0 replies, 9 likes


Sep 28 2019 arXiv CS-CL

Extreme Language Model Compression with Optimal Subwords and Shared Projections http://arxiv.org/abs/1909.11687
0 replies, 7 likes


Sep 28 2019 Jaidev Deshpande

This is super encouraging. An entire school of thought, starting from Shannon down to Gregory Chairing, proclaims that knowledge, and indeed, intelligence, should be compressible, and simpler than that data it tries to work with.
0 replies, 5 likes


Oct 02 2019 HubBucket | Healthcare and Medicine Technology

#HealthIT #NLP/#NLProc #NLU #NMT/#MT Pre-Trained Deep #NeuralNetwork Language Models such as #ELMo, #GPT, #BERT and #XLNet size makes them impractical on #mHealth and #HealthTech devices. 🖥️https://arxiv.org/abs/1909.11687?utm_source=Deep+Learning+Weekly&utm_campaign=06d703a367-EMAIL_CAMPAIGN_2019_04_24_03_18_COPY_01&utm_medium=email&utm_term=0_384567b42d-06d703a367-72970241 @HubBucket @HubIoMT @HubMobileApps @HubVoiceNLP https://t.co/BY5Q6ttcU9
0 replies, 4 likes


Sep 28 2019 David Praise Chukwuma Kalu

Extreme Language Model Compression with Optimal Subwords and Shares Projections. Zhao et al. : https://arxiv.org/abs/1909.11687 #NeuralNetworks #nlp #DataScience #ArtificialIntelligence #machinelearning https://t.co/kH7gaupF0A
0 replies, 2 likes


Sep 29 2019 John O'Malia

The student/teacher model applied here for language model compression https://arxiv.org/abs/1909.11687 with shared projection matrices reminds me so much of the interplay between hippocampus and cortex. Shout out to @pgmid and the BrainInspired podcast for prompting thoughts of crossover
0 replies, 1 likes


Content