Papers of the day   All papers

AutoML-Zero: Evolving Machine Learning Algorithms From Scratch

Comments

Quoc Le: Fun AutoML-Zero experiments: Evolutionary search discovers fundamental ML algorithms from scratch, e.g., small neural nets with backprop. Can evolution be the “Master Algorithm”? ;) Paper: https://arxiv.org/abs/2003.03384 Code: https://git.io/JvKrZ https://t.co/wZQJimrLid

22 replies, 1918 likes


Google AI: Extending research into evolutionary #AutoML, we present an approach that evolves algorithms from scratch—using only basic mathematical operations—rediscovering fundamental ML techniques & showing the potential to discover novel algorithms. Read more ↓ https://goo.gle/3edHPBc

6 replies, 710 likes


Jeff Dean (@🏡): AutoML-Zero: new research from @GoogleAI researchers Esteban Real, @crazydonkey200, David R. So, & @quocleix that that can rediscover fundamental ML techniques by searching a space of different ways of combining basic mathematical operations. Arxiv: https://arxiv.org/abs/2003.03384 https://t.co/cX3SDgfUOm

5 replies, 579 likes


hardmaru: Genetic programming learned operations reminiscent of dropout, normalized gradients, and weight averaging when trying to evolve better learning algorithms. Cool work! https://arxiv.org/abs/2003.03384

4 replies, 366 likes


Jeff Dean: Neat to see evolution discover some of the approaches that human ML experts have devised over the years!

4 replies, 255 likes


Ilya Sutskever: i like how it suggests that backprop may really be the best algorithm for training neural networks https://twitter.com/quocleix/status/1237528603564204033

7 replies, 249 likes


Christian Szegedy: Amazing progress from Quoc's group

1 replies, 71 likes


James Wang: GPT-3 generates perfectly correct web code from plain English descriptions. *crosses off web developer as future-proof job* https://twitter.com/sharifshameem/status/1282676454690451457

5 replies, 48 likes


Jeff Clune: AI-generating algorithms Pillars 1 and 2 take a big step forward (meta-learning the architecture and meta-learning the learning algorithm itself). Very exciting times!

0 replies, 46 likes


Per Kristian Lehre: Impressive results by Google Brain's AutoML-Zero: Evolving machine learning algorithms from scratch using genetic programming https://arxiv.org/abs/2003.03384

0 replies, 45 likes


News from Science: “This is one of those papers that could launch a lot of future research.” https://fcld.ly/5zdv7ii

1 replies, 28 likes


arxiv: AutoML-Zero: Evolving Machine Learning Algorithms From Scratch. http://arxiv.org/abs/2003.03384 https://t.co/yaKC1kHAYa

0 replies, 22 likes


Dmytro Mishkin: Tl;dr: 2-layer NN with SGD-training discovered by an evolutionary algorithm. https://t.co/tpf7nsfxMW

0 replies, 19 likes


Antonio Regalado: the snake bites its tail Google AI can independently discover AI methods then optimize them

1 replies, 16 likes


小猫遊りょう(たかにゃし・りょう): Can evolution be the “Master Algorithm”? https://twitter.com/quocleix/status/1237528603564204033?s=20

1 replies, 12 likes


Ben Sprecher: Wow. What a fascinating, complete system for generating learning ML code from the lowest level and evolving it over time. And fully open sourced, too! @jeffclune @kenneth0stanley FYI

0 replies, 9 likes


Peter Seibel: I mean, it made all the computer programmers, so yeah.

0 replies, 9 likes


ALife Papers: AutoML-Zero: Evolving Machine Learning Algorithms From Scratch "a novel framework that significantly reduces human bias through a generic search space. [...] evolutionary search can still discover two-layer neural networks trained by backpropagation." https://arxiv.org/abs/2003.03384 https://t.co/jjeATqEeyK

1 replies, 8 likes


Theophano Mitsa: #MachineLearning AutoML-Zero: Evolving Machine Learning Algorithms From Scratch https://arxiv.org/abs/2003.03384

0 replies, 8 likes


Aran Komatsuzaki: AutoML-Zero: Evolving Machine Learning Algorithms From Scratch. Genetic-programming-like method for finding a good ML algorithm code. While the performance is far from the sota, I'm glad this kind of fun idea has been attempted. https://arxiv.org/abs/2003.03384 https://t.co/C0EtY4i1MF

0 replies, 8 likes


ScienceAlert: Google Engineers 'Mutate' AI to Make It Evolve Systems Faster Than We Can Code Them http://www.sciencealert.com/coders-mutate-ai-systems-to-make-them-evolve-faster-than-we-can-program-them

0 replies, 7 likes


TedPavlic: New preprint from @GoogleAI uses genprog to evolve symbolic #ML algorithms than themselves setup and train inference engines. Evolves grad decent and backprop by itself. #ANN #GoogleBrain "AutoML-Zero: Evolving Machine Learning Algorithms from Scratch" https://arxiv.org/pdf/2003.03384.pdf

0 replies, 6 likes


Nam Le: AutoML-Zero by a group of Google Brain used Genetic Programming as a form of Program Synthesis, to automatically evolve ML algorithms. 2-layer Neural Nets solutions were discovered, starting from simple mathematical symbols. https://arxiv.org/abs/2003.03384

0 replies, 4 likes


John Grant: AutoML-Zero: Evolving Machine Learning Algorithms From Scratch https://arxiv.org/abs/2003.03384 ht @blopeur

0 replies, 3 likes


Pierre Gutierrez: I will soon be obsolete!

0 replies, 3 likes


Luigi Acerbi: Very interesting work on evolutionary ML starting from simple primitives.

0 replies, 3 likes


Simone Scardapane: *Evolving Machine Learning Algorithms From Scratch* by @quocleix @crazydonkey200 et al. A simple evolutionary procedure with basic math ops can learn non-trivial ML algorithms, including model + update rule. Nice neuroevolution-related read. Preprint: https://arxiv.org/abs/2003.03384 https://t.co/vhlHmAiqwX

0 replies, 3 likes


Germán Sierra: Meanwhile, in the real world: letting machine learning algorithms evolve (almost) by themselves Evolving Machine Learning Algorithms From Scratch https://arxiv.org/pdf/2003.03384.pdf

0 replies, 2 likes


Sayak Paul: Woah!

0 replies, 2 likes


OGAWA, Tadashi: => NAS (AutoML), Google Accelerator-aware NAS, Mar 5, 2020 https://arxiv.org/abs/2003.02838 BigNAS, Mar 24 https://arxiv.org/abs/2003.11142 EvoNorms: Evolving Normalization-Activation Layers, Apr 28 https://arxiv.org/abs/2004.02967 MobileDets, Apr 30 https://arxiv.org/abs/2004.14525 NAS 2020 https://twitter.com/ogawa_tter/status/1257367888668786691 https://t.co/aUnXSVLOsk

1 replies, 2 likes


NIDHAL SELMI - نضال السالمي: Genetic programming leads to solutions that have "intermediate states". Self-sufficient sub parts of the whole system. This is like going from design to evolution except with artificial neural networks.

0 replies, 2 likes


James Michael Dupont: Wow. Mind blown. The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence): Neural Architecture Search and Google’s New AutoML Zero with Quoc Le - #366 https://twimlai.com/twiml-talk-366-neural-architecture-search-and-googles-new-automl-zero-with-quoc-le

0 replies, 2 likes


Statistics Papers: AutoML-Zero: Evolving Machine Learning Algorithms From Scratch. http://arxiv.org/abs/2003.03384

0 replies, 2 likes


Amoebius Weinstein: For those worried that Skynet might become true. Here a recent paper that might have got us a big step closer. Not by Cyberdyne Systems but folks working for Google https://arxiv.org/pdf/2003.03384.pdf #AI #stopkillerobots https://t.co/BxvnrxFxcL

0 replies, 2 likes


BioDecoded: AutoML-Zero: Evolving Machine Learning Algorithms From Scratch | arXiv https://arxiv.org/abs/2003.03384 #AutoML https://t.co/qTyhRTxMEa

0 replies, 1 likes


Gapry: AutoML-Zero: Evolving Machine Learning Algorithms From Scratch #GapryPaperReadingList #Algorithms #MachineLearning #AutoML #AutoMLZero arXiv: https://arxiv.org/abs/2003.03384 repo: https://github.com/google-research/google-research/tree/master/automl_zero#automl-zero

0 replies, 1 likes


Guillermo Quintana: The only drawback that I see is overfitting since it's creating new algorithms with very high accuracy but designed specifically for the datasets used. But even then, it's an amazing work. Link to the paper: https://arxiv.org/abs/2003.03384

1 replies, 1 likes


ThreeMonkeys AndMe: Here is a link to the related research paper👇🏼 “AutoML-Zero: Evolving Machine Learning Algorithms From Scratch” https://arxiv.org/pdf/2003.03384.pdf (ML stands for “Machine Learning”) https://arxiv.org/pdf/2003.03384.pdf Note the “Google Brain” connection👇🏼 https://t.co/oKh9oXEV38

0 replies, 1 likes


@reiver ⊼ (Charles Iliya Krempeaux): 《AutoML-Zero: Evolving Machine Learning Algorithms From Scratch》 by Esteban Real, Chen Liang, David R. So, Quoc V. Le https://arxiv.org/abs/2003.03384 (machine learning) H/T @QEDanMazur

0 replies, 1 likes


OGAWA, Tadashi: => "Hardware Efficiency Aware Neural Architecture Search and Compression", Song Han, MIT, Invited, EMC^2, Jun 16, 2019 PDF (32MB) https://www.emc2-workshop.com/assets/docs/cvpr-19/han-talk.pdf https://songhan.mit.edu/ 35 Innovators Under 35 https://twitter.com/ogawa_tter/status/1143731943256707072 ProxylessNAS => AMC => HAQ (DQ) https://twitter.com/ogawa_tter/status/1141876641842360320 https://t.co/ocWYIddCVN

1 replies, 1 likes


Nikolai Rozanov: The future of AI just came into existence... https://arxiv.org/pdf/2003.03384.pdf

0 replies, 1 likes


amazin.ai: Do you think Data Scientists are indispensable? Google’s new Auto-ML Zero is a game changer - https://arxiv.org/pdf/2003.03384.pdf #ai #artificialintelligence #machinelearning https://www.instagram.com/p/CBiBZc0FKSA/?igshid=1tdykprhxd5o5

0 replies, 1 likes


vikrant: Master Algorithm @necronet @kapil__kathuria @CholericCleric

0 replies, 1 likes


Brijesh Singh: Can evolution be the Master Algorithm? Evolutionary search discovers fundamental ML algorithms from scratch, e.g., small neural nets with backprop. Genetic programming learned operations reminiscent of dropout, normalized gradients, & weight averaging! https://arxiv.org/abs/2003.03384

0 replies, 1 likes


Irenes (many): Whoa, cool.

0 replies, 1 likes


OGAWA, Tadashi: => "AutoML at Google and Future Directions", Quoc V. Le, Google, Invited, ICLR WS on Neural Architecture Search, Apr 26, 2020 https://slideslive.com/38926392/automl-at-google-and-future-directions AutoML-Zero, Mar 6 2020 https://arxiv.org/abs/2003.03384 Song Han, IEEE Micro, Jan/Feb 2020 https://ieeexplore.ieee.org/abstract/document/8897011 https://twitter.com/ogawa_tter/status/1144518821220212736 https://t.co/Qpm2WrMisK

1 replies, 0 likes


Content

Found on Mar 10 2020 at https://arxiv.org/pdf/2003.03384.pdf

PDF content of a computer science paper: AutoML-Zero: Evolving Machine Learning Algorithms From Scratch