Papers of the day   All papers

Shortcut Learning in Deep Neural Networks

Comments

Melanie Mitchell: Love this example of "unintended cue learning" from https://arxiv.org/pdf/2004.07780.pdf https://t.co/mJIoV1KiNs

9 replies, 307 likes


Melanie Mitchell: Good mantra for AI: "We must not confuse performance on a dataset with the acquisition of an underlying ability." From Geiros et al., Shortcut Learning in Deep Neural Networks https://arxiv.org/pdf/2004.07780.pdf

7 replies, 300 likes


Sebastian Raschka: Interesting paper arguing that many issues in deep learning can be attributed to shortcut learning (= getting good performance on a prob without actually understanding it; e.g., the classic perceptron for classifying tanks issue Marvin Minsky talked about) https://arxiv.org/abs/2004.07780

7 replies, 273 likes


Denny Britz: This paper on shortcut learning shows why we must carefully inspect results before attributing super-human performance to models. Benchmarks are often solved through shortcuts found in the data or inductive biases, not the underlying ability. arXiv: https://arxiv.org/abs/2004.07780 https://t.co/rgj6EgEfuD

4 replies, 259 likes


hardmaru: Shortcut Learning in Deep Neural Networks Humans also take shortcuts and cheat in life when we can get away with it. Interesting they mention that ways to overcome shortcut learning for artificial agents might apply to closing loopholes in human systems. https://arxiv.org/abs/2004.07780

2 replies, 126 likes


Wieland Brendel: It is becoming ever more clear that machines use all kinds of shortcuts to solve given tasks, leading to a large range of unexpected behaviours. In https://arxiv.org/abs/2004.07780 we give a thorough overview over this phenomenon & current attempts to overcome shortcut learning.

0 replies, 45 likes


Ankur Handa: A fun read on various shortcomings in deep neural networks (perhaps well known by now) which they call as "shortcut learning". https://arxiv.org/abs/2004.07780 The example on rat navigation was interesting to me. https://t.co/nh8uwVClZi

2 replies, 27 likes


Catherine Olsson: This isn't a fundamentally different observation from "specification gaming" or "shortcut learning" (https://deepmind.com/blog/article/Specification-gaming-the-flip-side-of-AI-ingenuity, https://arxiv.org/abs/2004.07780) Any time you've specified "what you want", it's game-able/shortcut-able. It needs to adapt to avoid the latest shortcuts. 8/

1 replies, 7 likes


Melanie Mitchell: . @bethgelab is doing some of the most interesting work around in understanding deep learning systems.

0 replies, 7 likes


arXiv CS-CV: Shortcut Learning in Deep Neural Networks http://arxiv.org/abs/2004.07780

0 replies, 6 likes


Daisuke Okanohara: Shortcuts are decision rules that use unintended features/clues to solve problems. Current ML/DL (and animals) tend to learn shortcuts and cannot generalize to out-of-distribution data (generalize wrongly). o.o.d generalization test is required. https://arxiv.org/abs/2004.07780

0 replies, 4 likes


Carlos E. Perez: It's time to address the elephant in the room for inductive learning. 'Short Cut' learning is a manifestation of Goodhart's Law. This paper proposes a framework to explore this: https://arxiv.org/abs/2004.07780

0 replies, 4 likes


arXiv CS-CV: Shortcut Learning in Deep Neural Networks http://arxiv.org/abs/2004.07780

0 replies, 3 likes


Javier Fuentes: Are we testing our ML models the wrong way? When your test set is in the same distribution than your training set you are blindfolded against what in this new paper call 'Shortcut learning'. https://arxiv.org/abs/2004.07780

2 replies, 2 likes


Jason H. Moore, PhD: Shortcut Learning in Deep Neural Networks https://arxiv.org/abs/2004.07780 HT @rasbt #deeplearning #machinelearning #datascience

0 replies, 1 likes


Ken Brucker: Those tricky rats! Also shows how challenging it can be to test the target variable.

0 replies, 1 likes


Joel Lehman: Not a bad mantra even beyond AI -- the measure is not the goal: the goal is the goal. Weirdly easy for us to conflate measure/goal, in day-to-day life or as a society (e.g. good grades in school, twitter likes, salary $, GDP as end in itself)

0 replies, 1 likes


Today I read: #TIR Shortcut learning in deep networks (https://arxiv.org/pdf/2004.07780.pdf) - Super accessible read on how DNNs follow unintended "shortcut" strategies in learning a concept. Networks look for easy solutions. It also draws connections between learning in humans or animals with DNNs. (1/n)

1 replies, 0 likes


Biagio Mattia La Rosa: "Science aims for understanding. [..] deep learning as a scientific discipline is still lagging behind in terms of understanding the principles and limitations that govern how machines learn [...]" Great conclusion of the paper https://arxiv.org/pdf/2004.07780.pdf written 1/3

1 replies, 0 likes


Content

Found on Apr 17 2020 at https://arxiv.org/pdf/2004.07780.pdf

PDF content of a computer science paper: Shortcut Learning in Deep Neural Networks