Papers of the day   All papers

Finite Versus Infinite Neural Networks: an Empirical Study

Comments

Jascha: "Finite Versus Infinite Neural Networks: an Empirical Study." https://arxiv.org/abs/2007.15801 This paper contains everything you ever wanted to know about infinite width networks, but didn't have the computational capacity to ask! Like really a lot of content. Let's dive in. https://t.co/aKjgbCrcLU

7 replies, 2279 likes


Jeff Dean (@🏑): Really nice paper analyzing properties of infinite width networks by @GoogleAI researchers @hoonkp, @sschoenholz, Jeffrey Pennington, Ben Adlam, @Locchiu, @ARomanNovak and @jaschasd. Jascha's thread here has lots of good discussion and a tour of some of the figures. ⬇️

3 replies, 320 likes


Zoubin Ghahramani: Looks like solid scholarly work . I love empirical comparisons that give insights, so looking forward to reading the paper. Also kudos for having 140 references spanning several decades!

1 replies, 107 likes


Grady Booch: The image on the right in the GIF is also a good visualization of me adjusting my pillow when I lay down and finally get to sleep.

2 replies, 56 likes


Daniel Roy: Where my popcorn?

1 replies, 55 likes


davilagrau: Neural Networks architectures are getting quite interesting... Did you see this[1] @thinbaker ? Finite Versus Infinite Neural Networks: an Empirical Study https://arxiv.org/abs/2007.15801 #machinelearning #trends

1 replies, 11 likes


Sam Schoenholz: @hoonkp did an absolutely insane amount of computation to answer all the questions you didn't know you had about finite vs infinite neural networks. Check out the summary / paper:

0 replies, 6 likes


Simone Scardapane *is back at work* πŸ§‘β€πŸ’»: *Finite Versus Infinite Neural Networks: an Empirical Study* Must-read paper on infinite-width NNs by @jaschasd @hoonkp @sschoenholz @Locchiu @ARomanNovak et al. LOTS of hints on their performance (for CIFAR-10) & links to classical training practices. https://arxiv.org/abs/2007.15801 https://t.co/qXyaGXHs6o

0 replies, 5 likes


Aleksandar Ivanov: From ReLU to infinity πŸ“ˆβ™ΎπŸ’»

0 replies, 3 likes


Content

Found on Aug 08 2020 at https://arxiv.org/pdf/2007.15801.pdf

PDF content of a computer science paper: Finite Versus Infinite Neural Networks: an Empirical Study