The biggest story in Artificial Intelligence (AI) over the last year has been the failure of Large Language Models (LLMs) to scale.
The idea was that there was a sort of “Moore’s Law” to them. The second model would be twice as good as the first, the third twice as good again, and before you know it super intelligence would be born.
It didn’t work. As Gary Marcus and others have noted, the fourth iteration of most LLM models is little better than the third. As LLMs deal with more data, their ability to run through the sludge gets worse.
But the Techbros aren’t easily deterred. Their latest shiny object is Recursive Self Improvement (RSI). This technique mimics what people and teams do. The first answer is wrong, so we adjust it slightly, run it again, and get a second answer that, while also wrong, is less wrong. The result is a “seed improver,” an architecture mimicking the process of learning.
Once we have the shiny object, the next step is to sell it as the ultimate answer. The arguments on both sides are offered by Joshua Rothman at The New Yorker, between an optimist paper called A.I. 2027 and a skeptical take called A.I. As Normal Technology.
The result, according to Rothman, is that a lot of people will wind up working on “cognitive factory floors,” where self-improving AIs must be herded like cats. The problem is that you can’t herd cats. (The company that produced the famous “herding cats” commercial a quarter century ago was gone within a few years and the company that bought it quickly became irrelevant.)
Does Any of This Work?
The problem with both visions, but especially that of the optimists, is their “once and for all” quality. We’re thought to be evolving toward some final state, either “the machine rules us” or “we fight the machines.”
It’s this I have a problem with, because it’s not how the world works.
In the real world, evolution is continuous. Time marches on. It’s something programmers have been fighting about for years, the difference between “waterfall” programming where there’s some final version, and “agile” approaches where you try to create continuous improvement. It’s the latter approach that best mimics real life, but you need the formality of the former to have a clue where you’re going.
To create true artificial intelligence, in other words, we need to know a lot more about how real intelligence works. We’ve just modeled a fruit fly brain. We’re a long way from modeling yours. And even after we model it, what do we have? We don’t have you.
What’s more likely is that RSI becomes another arrow in a large, growing quiver of AI inputs and insights. Its limits are the result of computers being unable to get outside themselves. Programs are restricted to their inputs, their inputs come from people, and people remain fallible.
This is going to happen with RSI, too. The answer is wrong because the inputs are limited, and you’ll only get better inputs by interfacing with the real souls behind each new machine.
Which is us.
