Papers of the day   All papers

Predicting What You Already Know Helps: Provable Self-Supervised Learning

Comments

Jason Lee: Predicting What You Already Know Helps: Provable Self-Supervised Learning We analyze how predicting parts of the input from other parts (missing patch, missing word, etc.) helps to learn a representation that linearly separates the downstream task. https://arxiv.org/abs/2008.01064 1/2 https://t.co/ahjaONZlfI

1 replies, 540 likes


Stanford NLP Group: Wonderful to see some theory behind the great success of self-supervised learning. Still trying to get our slow brains around how strong the results are. Cameo for the Stanford Sentiment Treebank—can it become the MNIST of #NLProc? By @jasondeanlee & al. https://arxiv.org/abs/2008.01064 https://t.co/TZd6vL3E7K

1 replies, 191 likes


Phillip Isola: Nice to see more theory on this. Paraphrasing: the only way to correctly colorize pikachu yellow is to first implicitly recognize that you are looking at a picture of pikachu!

2 replies, 104 likes


Sham Kakade: Great to see some theory on self-supervised learning. Looking forward to reading this one!

1 replies, 55 likes


Qi Lei: When the label (pikachu) captures some of the joint information between the input image and the image patch, predicting the missing part (self-supervised learning) implicitly learns the label.

0 replies, 9 likes


MONTREAL.AI: Predicting What You Already Know Helps: Provable Self-Supervised Learning Lee et al.: https://arxiv.org/abs/2008.01064 #DeepLearning #MachineLearning #SelfSupervisedLearning https://t.co/glWcB6sz2r

0 replies, 8 likes


Kevin Yang 楊凱筌: Pretraining helps because of approximate independence between inputs and the pretext task given the labels for the downstream task. @jasondeanlee @QiLei45724485 Nikunj Saunshi, and Jiacheng Zhuo https://arxiv.org/abs/2008.01064 https://t.co/6mMux8pQyU

0 replies, 1 likes


Content

Found on Aug 19 2020 at https://arxiv.org/pdf/2008.01064.pdf

PDF content of a computer science paper: Predicting What You Already Know Helps: Provable Self-Supervised Learning