When it comes to assessing intelligence, people have all kinds of tests — the SAT, IQ tests, and so on. There’s controversy over how fairly these tests really measure human intelligence, but at the very least, we know that they correlate with some general reasoning skills when people take them. That assumption breaks down when we try to assess intelligence in non-humans. What does it mean when a large language model passes an intelligence test meant for humans? Does it actually have the same reasoning skills that a human does, or is it doing something else? In today’s episode, with guests Erica Cartmill and Ellie Pavlick, we investigate the best ways to assess intelligence in non-humans, whether animals or machines.

COMPLEXITY

[email protected] (Santa Fe Institute)

Nature of Intelligence, Ep. 5: How do we assess intelligence?

NOV 20, 202448 MIN
COMPLEXITY

Nature of Intelligence, Ep. 5: How do we assess intelligence?

NOV 20, 202448 MIN

Description

Guests: Erica Cartmill, Professor, Anthropology and Cognitive Science, Indiana University BloomingtonEllie Pavlick, Assistant Professor, Computer Science and Linguistics, Brown UniversityHosts: Abha Eli Phoboo & Melanie MitchellProducer: Katherine MoncurePodcast theme music by: Mitch MignanoFollow us on:Twitter • YouTube • Facebook • Instagram • LinkedIn  • BlueskyMore info:Tutorial: Fundamentals of Machine LearningLecture: Artificial IntelligenceSFI programs: EducationDiverse Intelligences Summer InstituteBooks: Artificial Intelligence: A Guide for Thinking Humans by Melanie MitchellTalks: How do we know what an animal understands by Erica CartmillThe Future of Artificial Intelligence by Melanie MitchellPapers & Articles:“Just kidding: the evolutionary roots of playful teasing,” in Biology Letters (September 23, 2020), doi.org/10.1098/rsbl.2020.0370“Overcoming bias in the comparison of human language and animal communication,” in PNAS (November 13, 2023), doi.org/10.1073/pnas.22187991“Using the senses in animal communication,” by Erica Cartmill, in A New Companion to Linguistic Anthropology, Chapter 20, Wiley Online Library (March 21, 2023)“Symbols and grounding in large language models,” in Philosophical Transactions of the Royal Society A (June 5, 2023), doi.org/10.1098/rsta.2022.0041“Emergence of abstract state representations in embodied sequence modeling,” in arXiv (November 7, 2023), doi.org/10.48550/arXiv.2311.02171“How do we know how smart AI systems are,” in Science (July 13, 2023), doi: 10.1126/science.adj59