Papers of the day   All papers

The Measure of Intelligence


Nov 06 2019 François Chollet

I've just released a fairly lengthy paper on defining & measuring intelligence, as well as a new AI evaluation dataset, the "Abstraction and Reasoning Corpus". I've been working on this for the past 2 years, on & off. Paper: ARC:
53 replies, 3091 likes

Nov 12 2019 Quoc Le

Want to improve accuracy and robustness of your model? Use unlabeled data! Our new work uses self-training on unlabeled data to achieve 87.4% top-1 on ImageNet, 1% better than SOTA. Huge gains are seen on harder benchmarks (ImageNet-A, C and P). Link:
22 replies, 1416 likes

Nov 11 2019 Emil Wallner

François Chollet’s core point: We can't measure an AI system's adaptability and flexibility by measuring a specific skill. With unlimited data, models memorize decisions. To advance AGI we need to quantify and measure ***skill-acquisition efficiency***. Let’s dig in👇
14 replies, 1089 likes

Nov 06 2019 Brandon Rohrer

If you like to think deeply about building and measuring intelligence, I highly recommend this read. It's a cogent review of intelligence measurement, it's challenges, past approaches, and their limitations. @fchollet also takes the brave step of proposing a path forward.
2 replies, 228 likes

Nov 11 2019 Emil Wallner

'The Measure of Intelligence' contradicts the compute-is-all-we-need narrative, adds historical context, and is accessible to a broad audience. Hats off to @fchollet for creating clear and actionable suggestions to advance AGI. It's a must-read!
4 replies, 117 likes

Nov 12 2019 Riva

just read @fchollet's new paper.. so cool... 1) a critique of past and current AI benchmarking 2) reexamines the definitions of intelligence 3) suggests how to quantify generalization difficulty 4) *and* provides a dataset to test for it (..& more!)
4 replies, 116 likes

Nov 07 2019 Suzana Ilić

Wow. Can't wait to read this.
0 replies, 66 likes

Nov 08 2019 David Holz

@SamoBurja @rivatez @jimkxa @fchollet just released a VERY important paper arguing against simple scaling in AI. It's the most convincing piece yet and huge step forward in the discussion of general intelligence
0 replies, 60 likes

Nov 08 2019 Abel.TM

Great seeing AI focusing on fundamental questions: New approaches for generalization and reasoning (@GaryMarcus) The nature of understanding (@tdietterich) Actionable definitions of intelligence (@fchollet)
2 replies, 41 likes

Nov 12 2019 Deb Raji

Lots of great work on evaluation lately, but it's always framed as if AGI is the goal - what if it wasn’t? What if human-like cognition was not the goal? What if what we use ML for requires a completely different computational process from the way we think?
1 replies, 34 likes

Nov 08 2019 Davor Jordacevic

I highly recommend this paper about building and measuring intelligence. Thanks @fchollet for such a nice work. 📄
1 replies, 30 likes

Nov 10 2019 Pierre Ferragu

Excellent paper, @fchollet ( What I took from it: 1) The very idea of general intelligence is misleading. intelligence is specific to a context (e.g. human mind is orders of magnitude better at 2D and 3D than 4D)
1 replies, 20 likes

Nov 13 2019 Stanisław Jastrzębski

So do deep networks 'interpolate' or do they 'extrapolate'? :) For context see or @GaryMarcus critique of deep learning; I think most people would classify ImageNet-A as 'extrapolation', but also unclear what is the unlabeled dataset overlap with ImageNetA
0 replies, 15 likes

Nov 11 2019 Kyle McDonald

a new metric for measuring general intelligence. very curious to see the first baseline results on this, and also the deep criticisms of this framing of "intelligence".
2 replies, 15 likes

Nov 07 2019 SchoolOfAIOfficial

If one can measure the intelligence of students perhaps one can help a student to learn more effectively based on their precise needs and areas of difficulty. Read this new paper to learn about another approach to intelligence:
0 replies, 13 likes

Nov 06 2019 Aakash Kumar Nain

Next paper to read
0 replies, 8 likes

Nov 11 2019 MichelleRobbins

Excellent summary thread 👇 Grabbed the full @fchollet paper here -> for further exploration #AI #AGI
0 replies, 5 likes

Nov 13 2019 Karel Břinda

A very thoughtful paper about how to define and measure intelligence, with a great overview of the history and challenges as well as of the limitations of common benchmarks.
0 replies, 5 likes

Nov 11 2019 YorkU Philosophy

The Philosophy Department at York University is hiring an Assistant Professor in Ethics and AI.
0 replies, 5 likes

Nov 11 2019 Kristin Andrews

We're hiring! Ethics of AI (and other nonhuman systems) because types of intelligences abound. and
1 replies, 5 likes

Nov 12 2019 Gema Parreño

Measuring Intelligence by skill-acquisition efficiency ; scope, generalization difficulty, priors, and experience as pieces to be accounted for in characterizing intelligent systems #AGI #Benchmarking #ML #MeasureAndProgress
0 replies, 3 likes

Nov 06 2019 Dagmar Monett

THIS THREAD 👇 #defining #intelligence #measuring #intelligence #AI
0 replies, 3 likes

Nov 07 2019 Pablo Casas

0 replies, 2 likes

Nov 07 2019

• Intelligence is the efficiency with which a learning system turns experience and priors into skill at previously unknown tasks • measure of intelligence must account for priors, experience, & generalization difficulty
2 replies, 2 likes

Nov 06 2019 Ronald Richman

It’s a good day when there is something new to read from @fchollet. Now just need to wait for v2 of the DL with Python book!
0 replies, 2 likes

Nov 06 2019 Valentino Zocca 🇮🇹 🇪🇺

The measure of intelligence by @fchollet #AI #deeplearning #MachineLearning
0 replies, 1 likes

Nov 11 2019 Markus Wulfmeier

Exciting to see additional perspectives on intelligence as measured through adaptation and efficiency of transfer.
0 replies, 1 likes

Nov 14 2019 Edward Craig

The Measure of Intelligence (in AI) - Cornell / long paper #AI
0 replies, 1 likes

Nov 07 2019 eagerWorks

If you want to go serious about what intelligence is, we highly recommend reading this amazing work by @fchollet. The dataset he provides it’s a really nice bonus to start testing some ideas 💡
0 replies, 1 likes

Nov 13 2019 Philipp Bayer

This is more a book than a paper, dense with thoughts and information on 64 pages: @fchollet on how people focused too much on measuring *specific skills* at the cost of generalization, culminating in a set of tasks similar to IQ-tests for machines
1 replies, 1 likes