Papers of the day   All papers

The Measure of Intelligence

Comments

Nov 06 2019 François Chollet

I've just released a fairly lengthy paper on defining & measuring intelligence, as well as a new AI evaluation dataset, the "Abstraction and Reasoning Corpus". I've been working on this for the past 2 years, on & off. Paper: https://arxiv.org/abs/1911.01547 ARC: https://github.com/fchollet/ARC https://t.co/bVrmgLAYEv
53 replies, 3091 likes


Nov 12 2019 Quoc Le

Want to improve accuracy and robustness of your model? Use unlabeled data! Our new work uses self-training on unlabeled data to achieve 87.4% top-1 on ImageNet, 1% better than SOTA. Huge gains are seen on harder benchmarks (ImageNet-A, C and P). Link: https://arxiv.org/abs/1911.04252 https://t.co/0umSnX7wui
22 replies, 1416 likes


Nov 11 2019 Emil Wallner

François Chollet’s core point: We can't measure an AI system's adaptability and flexibility by measuring a specific skill. With unlimited data, models memorize decisions. To advance AGI we need to quantify and measure ***skill-acquisition efficiency***. Let’s dig in👇 https://t.co/Y3GjoG9n4e
14 replies, 1089 likes


Nov 06 2019 Brandon Rohrer

If you like to think deeply about building and measuring intelligence, I highly recommend this read. It's a cogent review of intelligence measurement, it's challenges, past approaches, and their limitations. @fchollet also takes the brave step of proposing a path forward.
2 replies, 228 likes


Nov 11 2019 Emil Wallner

'The Measure of Intelligence' contradicts the compute-is-all-we-need narrative, adds historical context, and is accessible to a broad audience. Hats off to @fchollet for creating clear and actionable suggestions to advance AGI. It's a must-read! https://arxiv.org/pdf/1911.01547.pdf
4 replies, 117 likes


Nov 12 2019 Riva

just read @fchollet's new paper.. so cool... 1) a critique of past and current AI benchmarking 2) reexamines the definitions of intelligence 3) suggests how to quantify generalization difficulty 4) *and* provides a dataset to test for it (..& more!) https://arxiv.org/pdf/1911.01547.pdf
4 replies, 116 likes


Nov 07 2019 Suzana Ilić

Wow. Can't wait to read this.
0 replies, 66 likes


Nov 08 2019 David Holz

@SamoBurja @rivatez @jimkxa @fchollet just released a VERY important paper arguing against simple scaling in AI. It's the most convincing piece yet and huge step forward in the discussion of general intelligence https://twitter.com/fchollet/status/1192121587467784192
0 replies, 60 likes


Nov 08 2019 Abel.TM

Great seeing AI focusing on fundamental questions: New approaches for generalization and reasoning (@GaryMarcus) http://rebooting.ai/ The nature of understanding (@tdietterich) https://twimlai.com/twiml-talk-315-what-does-it-mean-for-a-machine-to-understand-with-thomas-dietterich/ Actionable definitions of intelligence (@fchollet) https://twitter.com/fchollet/status/1192121587467784192
2 replies, 41 likes


Nov 12 2019 Deb Raji

Lots of great work on evaluation lately, but it's always framed as if AGI is the goal - what if it wasn’t? What if human-like cognition was not the goal? What if what we use ML for requires a completely different computational process from the way we think?https://twitter.com/fchollet/status/1192121587467784192
1 replies, 34 likes


Nov 08 2019 Davor Jordacevic

I highly recommend this paper about building and measuring intelligence. Thanks @fchollet for such a nice work. 📄
1 replies, 30 likes


Nov 10 2019 Pierre Ferragu

Excellent paper, @fchollet (http://arxiv.org/abs/1911.01547) What I took from it: 1) The very idea of general intelligence is misleading. intelligence is specific to a context (e.g. human mind is orders of magnitude better at 2D and 3D than 4D)
1 replies, 20 likes


Nov 13 2019 Stanisław Jastrzębski

So do deep networks 'interpolate' or do they 'extrapolate'? :) For context see https://arxiv.org/abs/1911.01547 or @GaryMarcus critique of deep learning; I think most people would classify ImageNet-A as 'extrapolation', but also unclear what is the unlabeled dataset overlap with ImageNetA
0 replies, 15 likes


Nov 11 2019 Kyle McDonald

a new metric for measuring general intelligence. very curious to see the first baseline results on this, and also the deep criticisms of this framing of "intelligence".
2 replies, 15 likes


Nov 07 2019 SchoolOfAIOfficial

If one can measure the intelligence of students perhaps one can help a student to learn more effectively based on their precise needs and areas of difficulty. Read this new paper to learn about another approach to intelligence:
0 replies, 13 likes


Nov 06 2019 Aakash Kumar Nain

Next paper to read
0 replies, 8 likes


Nov 11 2019 MichelleRobbins

Excellent summary thread 👇 Grabbed the full @fchollet paper here -> https://arxiv.org/pdf/1911.01547.pdf for further exploration #AI #AGI
0 replies, 5 likes


Nov 13 2019 Karel Břinda

A very thoughtful paper about how to define and measure intelligence, with a great overview of the history and challenges as well as of the limitations of common benchmarks.
0 replies, 5 likes


Nov 11 2019 YorkU Philosophy

The Philosophy Department at York University is hiring an Assistant Professor in Ethics and AI. http://webapps.yorku.ca/academichiringviewer/viewposition.jsp?positionnumber=2028
0 replies, 5 likes


Nov 11 2019 Kristin Andrews

We're hiring! Ethics of AI (and other nonhuman systems) because types of intelligences abound. https://arxiv.org/abs/1911.01547?fbclid=IwAR1kJaH7Duihhv4Afo2HLYuDEJwfYHDYcgPAJvuaPyRA5SizTVrj4E65pkE and https://link.springer.com/article/10.1007/s11023-019-09506-6?fbclid=IwAR3AzRSO1hAWyUCHNgiIQ3T4p80zvfLta4B2AgDekGG6v4cyjydfMx4yQuw
1 replies, 5 likes


Nov 12 2019 Gema Parreño

Measuring Intelligence by skill-acquisition efficiency ; scope, generalization difficulty, priors, and experience as pieces to be accounted for in characterizing intelligent systems https://arxiv.org/pdf/1911.01547.pdf #AGI #Benchmarking #ML #MeasureAndProgress
0 replies, 3 likes


Nov 06 2019 Dagmar Monett

THIS THREAD 👇 #defining #intelligence #measuring #intelligence #AI
0 replies, 3 likes


Nov 07 2019 Pablo Casas

👌👌👌
0 replies, 2 likes


Nov 07 2019 msb.ai

• Intelligence is the efficiency with which a learning system turns experience and priors into skill at previously unknown tasks • measure of intelligence must account for priors, experience, & generalization difficulty
2 replies, 2 likes


Nov 06 2019 Ronald Richman

It’s a good day when there is something new to read from @fchollet. Now just need to wait for v2 of the DL with Python book!
0 replies, 2 likes


Nov 06 2019 Valentino Zocca 🇮🇹 🇪🇺

The measure of intelligence by @fchollet #AI #deeplearning #MachineLearning
0 replies, 1 likes


Nov 11 2019 Markus Wulfmeier

Exciting to see additional perspectives on intelligence as measured through adaptation and efficiency of transfer.
0 replies, 1 likes


Nov 14 2019 Edward Craig

The Measure of Intelligence (in AI) - Cornell / long paper #AI https://arxiv.org/abs/1911.01547
0 replies, 1 likes


Nov 07 2019 eagerWorks

If you want to go serious about what intelligence is, we highly recommend reading this amazing work by @fchollet. The dataset he provides it’s a really nice bonus to start testing some ideas 💡
0 replies, 1 likes


Nov 13 2019 Philipp Bayer

This is more a book than a paper, dense with thoughts and information on 64 pages: https://arxiv.org/abs/1911.01547 @fchollet on how people focused too much on measuring *specific skills* at the cost of generalization, culminating in a set of tasks similar to IQ-tests for machines
1 replies, 1 likes


Content