Recall Isn't Intelligence

This one is opinion, not benchmarks. After months of running language models of every size, small ones on my own Mac and frontier ones in the cloud, I've landed somewhere I didn't expect.

I don't think the ability to quickly recall things, or to put ideas together, is intelligence.

The models are already better at it

Watch a language model work and it looks a lot like us talking. It writes one word at a time, and each next word makes sense with what came before. Underneath, it isn't purely word by word. When Anthropic traced what happens inside Claude as it writes a rhyming couplet, they found it picks the rhyme before it writes the line. That's us too. We talk word by word while holding something bigger behind it.

Newer designs go further. Next-concept modeling sketches the rough shape of the whole text first, then fills in the detail level by level. That's closer to a prepared speech, or a scientist who holds a whole theory in their head and keeps refining it. It's early research on small models, but the idea is clear.

Either way, word by word or big picture first, the model is doing the same job. It's recalling what it learned and putting it together. It has read more than any person ever will, and it can recombine all of it in seconds.

If that's intelligence, we lost

If intelligence means recalling large amounts of what you've learned and putting it together, then these models already beat most of us. We didn't just build something intelligent. We quietly wrote ourselves out of the conversation.

I don't believe that. Recall is a skill, like doing arithmetic in your head. Calculators beat us at arithmetic decades ago, and nobody decided calculators were smarter than people.

The AI researcher François Chollet makes the same argument, and he's thought about it a lot harder than I have. In On the Measure of Intelligence, the 2019 paper where he introduced the ARC test for AI, he argues that being skilled at a task is not the same as being intelligent. Enough prior knowledge or training data can buy high performance on almost anything. What he calls intelligence is how efficiently you pick up new skills, in situations you haven't seen before.

He isn't the only one looking past facts. Two other researchers once collected about 70 definitions of intelligence and boiled them down to one idea: the ability to achieve goals across a wide range of environments. Environments, not facts.

What I think it is

I think intelligence is the ability to survive and adapt as your world changes, and to make or find the best tools for it.

That world might be today, with language models and everything else we've built. Or it might be our ancestors, living through harsh climates with almost nothing, and still getting through. The proof is that we're here. Nobody got through an ice age on recall.

The psychologist Robert Sternberg describes intelligence in a similar way. It isn't only adapting to your environment. It's also shaping it, or choosing a better one. Use what's around you, change what you can, move when you have to.

The tool we always had

I studied human resources, so I'll admit my bias. Whatever tools we had, stone or steam or software, one thing never changed. We depended on each other.

There's research that backs this up. In 2007, a team gave the same tests to two-and-a-half-year-olds, chimpanzees and orangutans. On physical puzzles, space, quantity and cause and effect, the toddlers and the apes came out about even. On social tasks, learning from others, communicating, reading intent, the children got about 74% right and the apes about 33%. What set us apart early wasn't solving problems alone. It was learning from each other.

There's a saying I've always heard. If you want to go fast, go alone. If you want to go far, go together. It's usually called an African proverb, though nobody has been able to trace it.

That's the difference, for me. A model can recall faster than any of us. But recall is going fast. Intelligence is going far, and we've only ever done that together.

What do you think intelligence is?