Jascha: "Finite Versus Infinite Neural Networks: an Empirical Study." https://arxiv.org/abs/2007.15801 This paper contains everything you ever wanted to know about infinite width networks, but didn't have the computational capacity to ask! Like really a lot of content. Let's dive in. https://t.co/aKjgbCrcLU
7 replies, 2279 likes
Jeff Dean (@🏡): Really nice paper analyzing properties of infinite width networks by @GoogleAI researchers @hoonkp, @sschoenholz, Jeffrey Pennington, Ben Adlam, @Locchiu, @ARomanNovak and @jaschasd.
Jascha's thread here has lots of good discussion and a tour of some of the figures. ⬇️
3 replies, 320 likes
Zoubin Ghahramani: Looks like solid scholarly work . I love empirical comparisons that give insights, so looking forward to reading the paper. Also kudos for having 140 references spanning several decades!
1 replies, 107 likes
Grady Booch: The image on the right in the GIF is also a good visualization of me adjusting my pillow when I lay down and finally get to sleep.
2 replies, 56 likes
Daniel Roy: Where my popcorn?
1 replies, 55 likes
davilagrau: Neural Networks architectures are getting quite interesting...
Did you see this @thinbaker ? Finite Versus Infinite Neural Networks: an Empirical Study
1 replies, 11 likes
Sam Schoenholz: @hoonkp did an absolutely insane amount of computation to answer all the questions you didn't know you had about finite vs infinite neural networks. Check out the summary / paper:
0 replies, 6 likes
Simone Scardapane *is back at work* 🧑💻: *Finite Versus Infinite Neural Networks:
an Empirical Study*
Must-read paper on infinite-width NNs by @jaschasd @hoonkp @sschoenholz @Locchiu @ARomanNovak et al.
LOTS of hints on their performance (for CIFAR-10) & links to classical training practices.
0 replies, 5 likes
Aleksandar Ivanov: From ReLU to infinity 📈♾💻
0 replies, 3 likes
Found on Aug 08 2020 at https://arxiv.org/pdf/2007.15801.pdf