Papers of the day   All papers

Selfie: Self-supervised Pretraining for Image Embedding

Comments

Jun 10 2019 Thang Luong

Introducing Selfie, an unsupervised pretraining method that extends the concept of masked language model (e.g., BERT) towards continuous data, such as images. Overall challenging, but promising in low-data regime! Joint work with @thtrieu_ @quocleix https://arxiv.org/abs/1906.02940 https://t.co/A6ogEIFjtS
5 replies, 584 likes


Jun 21 2019 William Horton

It's cool to see cutting-edge ideas from NLP cross over to computer vision! This new technique (Selfie) adapts the masked language model concept from BERT to generate self-supervised image embeddings https://arxiv.org/abs/1906.02940
1 replies, 91 likes


Jun 10 2019 DataScienceNigeria

Like SELFIE? - the latest self-supervised pretraining technique that generalizes the concept of masked LM (like BERT) to continuous data (e.g images) It achieves significant gains when labeled set is small. Kudos to @thtrieu_ @quocleix @lmthang Read https://arxiv.org/pdf/1906.02940.pdf https://t.co/DZiPkkmZqS
0 replies, 19 likes


Jun 10 2019 Pranav Rajpurkar

Amazing!
0 replies, 9 likes


Jun 17 2019 niki parmar

Further studies show that self-attention is the most useful in later layers while convolutions better capture lower-level features. Combining their will be an interesting research direction.
1 replies, 5 likes


Jun 18 2019 Deep Learning Weekly

Selfie: Self-supervised Pretraining for Image Embedding https://arxiv.org/abs/1906.02940
0 replies, 3 likes


Jun 10 2019 Raym Geis

This looks promising for radiology in settings with sparse data [1906.02940] Selfie: Self-supervised Pretraining for Image Embedding https://arxiv.org/abs/1906.02940
0 replies, 2 likes


Jun 17 2019 Trieu H. Trinh

Great paper exploring attention architecture for images! We encounter similar results in our latest work https://arxiv.org/abs/1906.02940. Table 5 shows that keeping first layers conv, while using attention for last layers improve ResNet performance.
0 replies, 2 likes


Jun 10 2019 Statistics Papers

Selfie: Self-supervised Pretraining for Image Embedding. http://arxiv.org/abs/1906.02940
0 replies, 1 likes


Jun 10 2019 arXiv CS-CV

Selfie: Self-supervised Pretraining for Image Embedding http://arxiv.org/abs/1906.02940
0 replies, 1 likes


Jun 10 2019 cs.LG Papers

Selfie: Self-supervised Pretraining for Image Embedding. Trieu H. Trinh, Minh-Thang Luong, and Quoc V. Le http://arxiv.org/abs/1906.02940
1 replies, 0 likes


Content