Einstein, Churchill, and AI
What LLMs Can't Do
Catch-up service:
The Great and Unfortunate Story of Jason Arday (A note on the awful news about Arday further down)
The Last Lunch
Young Churchill
Dear Me, England
Anatomy of a ‘F*ck You’
Happy John and Paul Day
I am not very fond of arguments over what AI can or cannot do, since they tend to devolve into arguments over definitions. For instance, it used to be said that AI can’t reason. Now that it’s solving maths problems which have stumped our brightest mathematicians for decades, the sceptics are saying ah, but that’s not what we meant by reasoning. On the other side, AI boosters address every capability gap with a “Not yet. The technology will obviously continue to improve until it solves whatever shortcoming you’re pointing to.” Well, maybe, but this is a frustratingly unfalsifiable claim which can be wheeled out to demolish any sceptical argument, a literal deus ex machina.
So I appreciated this new paper from a Google DeepMind researcher called Tom Zahavy, which is - unusually for a paper from an AI lab - all about what AI, at least in its current form, cannot do. Its title is “LLMs can’t jump” (Ron Shelton’s movie surely has one of the best and most generative titles of all time). In short, it’s about why LLMs are unlikely to make scientific breakthroughs on their own.
The paper argues that there is a fundamental limit to their capacity for invention. LLMs can ingest tons of data and identify the patterns and rules which govern a particular domain. They can also work their way through a series of logical steps to solve specific, highly complex problems. What they can’t do is make a creative “jump” - the sudden insight that enables a human scientist to go beyond the available data.
Zahavy gives what is possibly the most famous documented example of such a leap. In 1907 (the same year Pablo Picasso re-invented art), Albert Einstein was working at the patent office in Bern and also on a new theory of the universe which yoked together space and time. While pondering how “relativity” would work for someone speeding up or slowing down, he had what he later described as “the happiest thought of my life”.
Einstein imagined a man falling from the roof of a house, or standing in a free-falling elevator. What would that feel like? It would feel weightless. If he was holding an apple and let go of it, the apple would “float” alongside him. From the falling man’s point of view, then, it could be said there was no gravity in his pocket of space. That weightlessness, Einstein realised, would be exactly the same if the man were floating in deep space, far from any planet’s gravitational pull.
In a sense, then, falling, not standing, is the man’s natural state. When he stands, the earth is preventing him from falling. His weight is the earth pushing back. From there, Einstein eventually arrived at the idea that objects don’t fall because gravity is pulling them, as Newton believed, but because mass changes the shape of space and time, and objects move along the paths which that shape creates.
If you don’t fully understand this, that’s OK, neither do I. What matters, for our purposes, is that this epochal breakthrough was based on very little data. It’s not like early twentieth century astronomy was generating lots of conflicting information about gravity and acceleration and space. An 1907 LLM trained on Newtonian rules and fed data on planetary movements would have received barely a squeak of an error signal. There was a tiny unexplained wobble in Mercury’s orbit, which most scientists explained by reference to an unseen planet. Otherwise, Newton had it covered.
But Einstein was unsatisfied. Newtonian gravity and Maxwellian electromagnetism worked in completely different ways, and Einstein couldn't accept that the universe ran on two separate rulebooks. The universe was one, he just knew it - and this led him to make his conceptual leap.
Back to LLMs. Zahavy notes that Einstein’s insight was rooted in physical experience - in knowing what it was like to be a body subject to gravity. It had to be felt because it could not yet be expressed in abstractions - in words or mathematical symbols. Newtonian science offered no logical explanation for his imaginary scenario. Einstein himself described his thought process like this: ”The words or the language, as they are written or spoken, do not seem to play any role in my mechanism of thought.”
LLMs are built out of words and symbols. They don’t have physical experience of the world. They are cathedrals of pure abstraction. If Einstein had an LLM, it could have helped him work through the consequences of his discovery and to make predictions, which involved a huge amount of mathematical work. He would have appreciated that; he wasn’t a very enthusiastic mathematician. But the LLM couldn’t have made the foundational leap for him. No amount of pattern-matching and deduction would have got it there.
Zahavy points out that many scientific breakthroughs did not make sense in the light of available data. They had to be intuited or imagined into being. That in turn has depended, as unscientific as it sounds, on what the scientist wants to believe - on some kind of aesthetic or moral demand they are making of the world. Great scientists often have deep, supra-scientific convictions which operate like search heuristics, pushing them to look for explanations that can’t be spotted in the data.
He mentions Kepler’s neoplatonic belief in the centrality of the sun. We might add Faraday’s discovery of the electromagnetic field (then given mathematical form by Maxwell). That sprang, in part, from Faraday’s Christian belief that since nature was the orderly creation of God, apparently separate forces should exhibit underlying unity. Similarly, Einstein had a profound philosophical conviction that the universe is a unified whole. He wanted that to be true, which pushed him to discover that it is true.
To design an AI capable of invention, Zahavy says, technologists will need systems that “do not just simulate the world but hold strong beliefs or priors about how that world should be structured…”. Note that “should”. Can a machine have a passionate conviction or belief? Would that count as a form of “intelligence” or is that word inadequate to describe the thought process of an Einstein? Zahavy doesn’t explore those questions, but for now, this kind of cognition remains uniquely human.
C.P. Snow’s collection of biographical essays, Variety of Men, includes vivid portraits of Einstein and Churchill, both of whom he had met. He detected certain likenesses. Neither did very well at school, and both developed a deep and abiding conviction that they were different, with something unique to contribute to the world. Once settled on a belief or idea, both were “unbudgeable”; Einstein quietly, Churchill loudly.
In a brilliant passage on Churchill, Snow draws a sharp distinction between judgement and insight. In leadership terms, good judgement means being able to grasp the complexity of a situation and assess the trade-offs. Churchill, says Snow, was not good at this. In fact, most contemporaries agreed that his judgement was “seriously defective”. Here’s Snow:
Churchill had a very powerful mind, but a romantic and unquantitative one. If he thought about a course of action long enough, if he conceived it alone in his own inner consciousness and desired it passionately, he convinced himself that it must be possible. Then, with incomparable invention, eloquence, and high spirits, he set out to convince everyone else that it was not only possible but the only course of action open to man. Unfortunately, the brute facts of life were not always so malleable as his listeners.
Churchill’s obsessive pursuit of certain ideas led to blunders like Gallipoli and his vain attempts to prevent Indian independence. But it was this same obsessiveness which led to his greatest achievement. When all the smart people with good judgment in the British state were proposing that Hitler be accommodated, Churchill alone thought otherwise. Snow again:
Judgement is a fine thing, but it is not all that uncommon. Deep insight is much rarer. Churchill had flashes of that kind of insight, dug up from his own nature, independent of influences, owing nothing to anyone outside himself. Sometimes it was a better guide than judgement. In the ultimate crisis, when he came to power, there were times when judgment itself could, though it did not need to, become a source of weakness. When Hitler came to power, Churchill did not use judgment, but one of his deep insights: this was absolute danger. There was no easy way round.
You often hear it said that “judgement” is where humans will retain an edge over machines. But “insight” might be an even wider moat, and if the AI doomsters are right then it will become very necessary at some point.
Yann LeCun, former chief AI scientist at Meta, now an AI startup founder, was recently asked what he wanted his legacy to be. He said “Increasing the amount of intelligence in the world.” With more intelligence, he continued, there will be less human suffering, more rational decisions, more understanding of the world and the universe. But more intelligence, in the sense he means, wouldn’t have produced Einstein’s leap or Churchill’s insight.
Were Einstein and Churchill obviously more intelligent than their peers? No. Einstein was the greatest scientist of the twentieth century but not because he had a higher IQ than Planck or Bohr (he acknowledged that many of his peers were better at maths). Churchill’s visionary leadership wasn’t the product of rational analysis. Physics and politics are very different realms and Churchill and Einstein had very different kinds of mind, but they both had this ability to “dig up from their own nature” insights that nobody else had grasped, and which weren’t in the data.
After the jump, for paid subscribers only: a note on the awful news about Jason Arday, and why I don’t think the media is to blame. Plus The Ruffian’s famous Rattle Bag of links and brain food, including: a website that reminds you how lucky you are; why polls are more important to politics than ever; Burnham’s phase II; a discussion of Jesus as a historical figure, and a piece of classical music I’m currently obsessed by.
