Papers of the day   All papers

The Hardware Lottery

Comments

Mark O. Riedl: This paper is 🔥 https://arxiv.org/abs/2009.06489

18 replies, 1074 likes


Jeff Dean (@🏡): A delightful essay by @GoogleAI's @sarahookr on the notion that often the success of research idea depends on the idea and how well it suits the available HW and SW, rather than just the merits of the idea. It's an enjoyable and read, Sara!

6 replies, 477 likes


Leon Overweel: Really great history + position paper by @sarahookr: The Hardware Lottery (https://arxiv.org/abs/2009.06489) "This essay introduces the term HW lottery to describe when a research idea wins b/c it suits the available SW/HW, not because the idea is superior to alt. research directions." https://t.co/djxTkEZcpB

2 replies, 318 likes


hardmaru: The Hardware Lottery by @SaraHookr “When a research idea wins because it is suited to the available software and hardware, not because the idea is superior…The advent of specialized hardware makes it increasingly costly to stray off of the beaten path.” https://arxiv.org/abs/2009.06489

5 replies, 316 likes


Sara Hooker: Thank you for all the feedback! This came together very slowly over the last year, during odd moments I could find in the day to slowly build it out. Most of all, it gave me joy to think deeply about this topic. Website: https://hardwarelottery.github.io/ Paper: https://bit.ly/3ku2OmB https://t.co/oZC522uK8x

2 replies, 218 likes


Loren Lugosch: Great paper by @sarahookr on ML HW: https://arxiv.org/abs/2009.06489 "Our brain has over 85 billion neurons but runs on the energy equivalent of an electric shaver"; what's wrong with current architectures, such that we need supercomputers to run models that are a lot smaller than this?

1 replies, 103 likes


Kosta Derpanis: http://arxiv.org/abs/2009.06489 https://t.co/uvsYV5I18T

2 replies, 75 likes


elvis: Very cool and timely essay about the effect of hardware and software on the success of research ideas. By @sarahookr https://arxiv.org/abs/2009.06489 https://t.co/Xnlut8Fr41

1 replies, 50 likes


Brandon Rohrer: This fabulous essay by @sarahookr introduces the idea of a Hardware Lottery - that the success of an idea depends heavily on what hardware (and software) happens to be prevalent when it's proposed. https://arxiv.org/pdf/2009.06489.pdf h/t @mark_riedl

1 replies, 42 likes


Cory Doctorow #BLM: Google Brain researcher @sarahook's new paper "The Hardware Lottery," proposes that there is a secret, relentless force that bends the course of machine learning: processor architecture. https://arxiv.org/abs/2009.06489 1/ https://t.co/VSpTALSEjR

5 replies, 30 likes


Will Manidis: “When a research idea wins because it is suited to the available software and hardware, not because the idea is superior [but as] The advent of specialized hardware makes it increasingly costly to stray off of the beaten path.” Favorite preprint of 2020 https://arxiv.org/abs/2009.06489 https://t.co/1T2VTlVefr

1 replies, 24 likes


Charles 🎉 Frye: Imagine if consumer graphics had required efficient trees, instead of matmuls. I bet we'd all be talking about "deep tree learning" and GPTree for NLP with 175 bn branches! Really nice paper on the "Hardware Lottery" by @sarahookr, with historical egs from Babbage to Hinton. https://t.co/3Nvj0qK5WJ

3 replies, 20 likes


Jonathan Frankle: So thrilled this is finally out - great work @sarahookr!

4 replies, 16 likes


Nicola De Cao: The Hardware Lottery: how the success of algorithms is biased on the current hardware we have. A good example for ML: we see DL succeeding because we have GPUs. Other emerging ideas may die soon only because there is no specialized hardware to run them and not because are bad.

0 replies, 15 likes


Will Zeng: Highly recommend this new paper by @sarahookr on The Hardware Lottery to anyone working in quantum algorithms and applications today. https://hardwarelottery.github.io/ We can continue to learn from the history of classical computing

0 replies, 15 likes


Charles 🎉 Frye: Loved this paper by @SaraHookr Great news: I'll be talking with her next Tuesday at 5 Pacific at the @weights_biases Salon! We'll cover this paper and her recent work on outlier detection via gradient variance. Come thru and ask her some Qs, y'all! 🔗 http://tiny.cc/wb-salon

0 replies, 12 likes


Pradyumna Singh | प्रद्युम्न: The Hardware Lottery A thoroughly enjoyable essay by @sarahookr on the serendipitous intersection of compute hardware, software and ideas, which determines which ideas ultimately prevail (at least in the short term). https://arxiv.org/abs/2009.06489

0 replies, 11 likes


Erich Elsen: Nice essay by @sarahookr on the interaction between the hardware we have and the algorithms we choose to investigate. Another framing I like is that we (the community) are performing gradient descent on a (hardware, algorithms) landscape and we're trapped in a local minima.

1 replies, 11 likes


Ash Fontana: This paper by @sarahookr provides a good framework for predicting what machine learning systems might work in the medium term https://arxiv.org/pdf/2009.06489.pdf. Her other papers, e.g. https://arxiv.org/abs/1902.09574, are also good for those that want to understand the current effectiveness of ML.

0 replies, 9 likes


Igor Mordatch: A really great essay - thank you @sarahookr! For more thoughts on this topic, I would also suggest fantastic "Value-laden Disciplinary Shifts in Machine Learning" (http://arxiv.org/abs/1912.01172) by @DotanRavit and @SmithaMilli

0 replies, 8 likes


Chris Albon: Congrats to @sarahookr on a gamechanging paper

0 replies, 8 likes


@chris_wigley: Great piece by @sarahookr framing the "hardware lottery" - implications of available hardware+software on success of scientific research. (Spoiler: success increasingly super-linked to HW/SW choices). Massively true for us at @GenomicsEngland. https://arxiv.org/pdf/2009.06489.pdf https://t.co/cBb7bPspNv

1 replies, 7 likes


Neil Thompson: Insightful paper and coining of the term "hardware lottery" by @sarahookr to describe how certain types of computing are stymied if they don't fit with hardware https://arxiv.org/abs/2009.06489... (1/2)

1 replies, 6 likes


Libby Kinsey: "While deep neural networks have clear commercial use cases, there are early warning signs that the path to the next breakthrough in AI may require an entirely different combination of algorithm, hardware and software." The Hardware Lottery by @sarahookr https://arxiv.org/pdf/2009.06489.pdf

0 replies, 5 likes


Neal Khosla: This was a good paper. And also reads like an opportunity for all those deep learning doubters. https://arxiv.org/pdf/2009.06489.pdf

0 replies, 3 likes


Ash Jogalekar: Great essay. A particularly pernicious example of the intersection of hardware and ideas - the cotton gin and slavery.

0 replies, 3 likes


Eryk Salvaggio: Dig it: this paper “introduces the term hardware lottery to describe when a research idea wins because it is suited to the available software and hardware and not because the idea is superior to alternative research directions.”

0 replies, 2 likes


Javier Fuentes Alonso: Why did neural networks only became mainstream in the last 10 years if the backpropagation paper was written in the 80s? Because it took some time to win "The hardware lottery". https://arxiv.org/abs/2009.06489 Thread 👇👇👇 https://t.co/6a3bWrVipb

1 replies, 2 likes


Szabi Marka: Hardware matters. https://arxiv.org/pdf/2009.06489.pdf @perrymetzger any comments? https://t.co/0YycU4lqMv

0 replies, 1 likes


Christian Wolf: Essay by @sarahookr, why our AI research is guided by hardware ... that's why I begin my lecture on GPUs in my deep learning course (https://perso.liris.cnrs.fr/christian.wolf/teaching/deeplearning/cm-deeplearning-3-2-gpu-hardware.pdf) with the sentence "the history of GPUs is a history of economics as well as computer science". 1/2

0 replies, 1 likes


Nicolas Pinto: The Hardware Lottery by Sara Hooker — This essay introduces the term hardware lottery to describe when a research idea wins because it is suited to the available software and hardware and not because the idea is superior to alternative research directions.

0 replies, 1 likes


Cory Doctorow #BLM: The Hardware Lottery: Chip architecture is the secret legislator of machine learning. https://twitter.com/doctorow/status/1307022980543700995 6/ https://t.co/c4xZG04uxA

1 replies, 1 likes


davilagrau: Innovation incentives: a historical perspective https://arxiv.org/abs/2009.06489 #innovation #tech #history https://t.co/YhXAAdFUiT

1 replies, 0 likes


@reiver ⊼ (Charles Iliya Krempeaux): 《The Hardware Lottery》 by Sara Hooker https://arxiv.org/abs/2009.06489 (machine learning)

1 replies, 0 likes


Content

Found on Sep 17 2020 at https://arxiv.org/pdf/2009.06489.pdf

PDF content of a computer science paper: The Hardware Lottery