Papers of the day   All papers

EXASCALE DEEP LEARNING FOR SCIENTIFIC INVERSE PROBLEMS

Comments

Oct 03 2019 Nando de Freitas

27,600 GPUs, 1/2 PB data, and a neural net with 220,000,000 weights. More please! https://arxiv.org/pdf/1909.11150.pdf
25 replies, 637 likes


Oct 03 2019 François Chollet

TensorFlow scales pretty well. Work done on the Summit supercomputer at Oak Ridge. 27,600 V100 GPUs, near-linear scaling.
1 replies, 187 likes


Oct 03 2019 Kareem🔥10x statistician🔥Carr

If you ever feel like your model has "too many parameters", remember this deep learning network with 220,000,000 weights.
6 replies, 125 likes


Sep 30 2019 Reza Zadeh

Distributed training of 27,600 GPUs, with near-linear scaling (factor 0.93). Trains a network with 10^8 weights on 0.5 PB of data. Main idea: AllReduce via two new communication strategies, “Grouping” & “Bitvector”. https://arxiv.org/abs/1909.11150 https://t.co/GBUIHDdCuz
0 replies, 33 likes


Oct 03 2019 Peyman Milanfar

For inverse problems, a physical (even approximate) model of the process is ~always available. i.e. We have strong prior information that should result in fewer weights, not more. Over-paramaterizing is wasteful in the sense that we're learning some physics that's already known.
1 replies, 29 likes


Oct 03 2019 (((ل()(ل() 'yoav))))

I am not a material scientist so take with a grain of salt, but from my humble understanding of this paper, it seems that this SUPER MASSIVE compute didn't really produce any useful results. and on top of that the experimental design seem to be poor as well? 1/2
2 replies, 23 likes


Sep 26 2019 arxiv

Exascale Deep Learning for Scientific Inverse Problems. http://arxiv.org/abs/1909.11150 https://t.co/hVBqJAtMXt
0 replies, 17 likes


Oct 02 2019 Alison B Lowndes ✿

EXASCALE #deeplearning from @ORNL & @NvidiaAI for scientific inverse problems - new comms strategy in sync distributed (CNN) training resulting in near-linear scaling (0.93) on Summit (27,600 Volta #GPUs) harnessing Tensorcores on 0.5PB of data! https://arxiv.org/pdf/1909.11150.pdf https://t.co/1pef7kaZ9U
0 replies, 10 likes


Oct 03 2019 Laszlo Sragner

@mnet_user @NandoDF Exascale Deep Learning for Scientific Inverse Problems https://arxiv.org/abs/1909.11150
0 replies, 9 likes


Sep 30 2019 Alexander Sergeev

Horovod is used to scale the Summit again!
0 replies, 6 likes


Sep 30 2019 Stephen Pimentel

Exascale Deep Learning for Scientific Inverse Problems https://arxiv.org/abs/1909.11150
0 replies, 5 likes


Oct 06 2019 Daniel Klotz

omg: https://arxiv.org/abs/1909.11150
0 replies, 4 likes


Sep 28 2019 Underfox

Researchers introduced a new strategies in sync distributed #DeepLearning which produce an optimal overlap between computation and communication and result in near-linear scaling of distributed training up to 27600 @nvidia V100 GPUs on the Summit. #HPC https://arxiv.org/pdf/1909.11150.pdf https://t.co/T0UZKLs9AS
0 replies, 4 likes


Nov 08 2019 Timothy Liu

@PaulyAlcorn @HPC_Guru @nvidia @ISChpc @AMD @NVIDIADC If we’re *very* liberal with the way we measure, NVIDIA has one exascale machine today - ORNL Summit: https://arxiv.org/abs/1909.11150 (demonstrated 2.15 EFLOPs FP16 on real problem)
0 replies, 1 likes


Oct 04 2019 Flávio Clésio

Scientists: "We need to take care of the environment and the carbon footprint" Also the Scientists...
2 replies, 1 likes


Sep 30 2019 ଶୁଆ

Exascale Deep Learning for Scientific Inverse Problems. ବୈଜ୍ଞାନିକ ଓଲଟ ପ୍ରଶ୍ନ ସବୁ ପାଇଁ ଏକ୍ସ। ଅଙ୍କରେ ହେଉଥିବା ଗଭୀର ଶିକ୍ଷା । #ଶୁଆନୁବାଦ ଠିକ୍ ହେଲା @strytellrofpast ଆଜ୍ଞା ?🙏 This is Odia translation of your tweet @StephenPiment 🙏
0 replies, 1 likes


Oct 04 2019 liubov

And the answer was 42... "27,600 V100 GPUs, 0.5 PB data, and a neural net with 220,000,000 weights... If you wonder, it all was used to address scientific inverse problem in materials imaging." ArXiV: https://arxiv.org/pdf/1909.11150.pdf #ItIsNotAboutSize #nn #datascience #Telegram https://t.co/Al5zzyKaCd
0 replies, 1 likes


Oct 03 2019 Educofin

Don’t try this home: The latest news in #ArtificialIntelligence #carbonfootprint @SIAT_Italia
0 replies, 1 likes


Oct 03 2019 Lacra Provinciana

KE
1 replies, 0 likes


Content