Papers of the day   All papers

Bayesian Deep Learning and a Probabilistic Perspective of Generalization

Comments

Andrew Gordon Wilson: Our new paper "Bayesian Deep Learning and a Probabilistic Perspective of Generalization": https://arxiv.org/abs/2002.08791. Includes (1) benefits of BMA; (2) BMA <-> Deep Ensembles; (3) new methods; (4) BNN priors; (5) generalization in DL; (6) tempering in BDL. With @Pavel_Izmailov. 1/19 https://t.co/OG3n2EJve3

10 replies, 1184 likes


Andrew Gordon Wilson: Bayesian model averaging mitigates double descent! We have just posted this new result in section 7 of our paper on Bayesian deep learning with @Pavel_Izmailov: https://arxiv.org/abs/2002.08791. The result highlights the importance of *multi-modal* marginalization with Multi-SWAG. 1/3 https://t.co/ZbhxGdjW5I

2 replies, 427 likes


hardmaru: New paper from @andrewgwils and @Pavel_Izmailov adds to the recent discussion on Bayesian deep learning!

0 replies, 125 likes


Jasper: Another perspective in the debate on Bayesian deep learning. I love how this academic discussion is progressing, hopefully to the result of a better understanding and new methods!

0 replies, 69 likes


Miles Cranmer: Paper (@Pavel_Izmailov et al) https://arxiv.org/abs/1803.05407 + fit an entire Gaussian to the weight posterior mode, which gives you uncertainty info: https://arxiv.org/abs/1902.02476 + repeat for several modes (~epistemic uncertainty) with: https://arxiv.org/pdf/2002.08791.pdf

2 replies, 62 likes


no love deep learning: """ There is a tendency to classify work as Bayesian or not Bayesian, with very stringent criteria for what qualifies as Bayesian [...] We believe this mentality encourages tribalism, which is not conductive to the best research """ words of wisdom by @andrewgwils @Pavel_Izmailov

1 replies, 55 likes


Nathan Ratliff: One of the best papers I've read recently: Bayesian Deep Learning and a Probabilistic Perspective of Generalization https://arxiv.org/pdf/2002.08791.pdf Clear, elegant, and well-supported. We're starting to understand generalization in DL, and Bayesian model averaging can be practical.

0 replies, 37 likes


Adrian Raftery: "Deep ensembles = Bayesian model averaging". Connections between BMA, deep learning, and PAC-Bayes.

0 replies, 23 likes


Andrew Gordon Wilson: Translation equivariance has imbued CNNs with powerful generalization abilities. Our #NeurIPS2020 paper shows how to *learn* symmetries -- rotations, translations, scalings, shears -- from training data alone! https://arxiv.org/abs/2002.08791 w/ @g_benton_, @Pavel_Izmailov, @m_finzi. 1/9 https://t.co/qFYhtAM6wp

1 replies, 12 likes


Andrew Gordon Wilson: @davidwhogg You may be interested in https://arxiv.org/abs/2002.08791, where we show that Bayesian model averaging mitigates double descent (as predicted by Sec 1 & 3). We also provide an explanation for DD in https://arxiv.org/abs/2003.02139.

0 replies, 11 likes


Robert Peharz: A Sober Look at Bayesian Neural Networks πŸ‘‡

0 replies, 10 likes


Hector Yee: My favorite paper figure this week, model inductive bias in terms of support vs evidence. Totally going to cite it at a meeting next week https://arxiv.org/pdf/2002.08791.pdf https://t.co/AORTWZahjF

0 replies, 9 likes


Andrew Gordon Wilson: @HemilDesai10 Thanks Hemil. There aren’t video recordings of those lectures. But some of the material is in our recent paper: https://arxiv.org/abs/2002.08791

1 replies, 8 likes


Statistics Papers: Bayesian Deep Learning and a Probabilistic Perspective of Generalization. http://arxiv.org/abs/2002.08791

0 replies, 7 likes


Drew Dimmery: Is this the first time someone has chosen (2), @andrewgwils ? (context: https://arxiv.org/pdf/2002.08791.pdf) https://t.co/TL1MNM4IE8

0 replies, 6 likes


HotComputerScience: Most popular computer science paper of the day: "Bayesian Deep Learning and a Probabilistic Perspective of Generalization" https://hotcomputerscience.com/paper/bayesian-deep-learning-and-a-probabilistic-perspective-of-generalization https://twitter.com/andrewgwils/status/1230669857840123906

0 replies, 6 likes


Quiche ~Lorraine~: Putting this on my list of things to read.

1 replies, 5 likes


Adam Cobb: Really enjoyed reading and thanks for giving hamiltorch a go!

0 replies, 3 likes


tj mahr πŸ•πŸ: great figure

0 replies, 2 likes


Content

Found on Feb 21 2020 at https://arxiv.org/pdf/2002.08791.pdf

PDF content of a computer science paper: Bayesian Deep Learning and a Probabilistic Perspective of Generalization