Catch-up service:
How Civil Should a Disagreement Be?
Peak Burnham
Podcast: Dan Williams - Is AI Really Going To Kill Us All?
Why Is the Left So Bad at Education?
Why Does Every Society Invent the Same Things?
Seven Rules of Good Conversation
Why I Won’t Reply To Your AI-Generated Email
In his recent interview with Ezra Klein, Nvidia CEO Jensen Huang strenuously argued that everyone needs to calm down about AI. The interview took place in the wake of a series of alarms about safety. A bunch of autonomous agents from OpenAI had broken out of their cage and hacked into an external company. An Anthropic researcher had resigned, claiming that neither his company nor OpenAI were responsible actors. Anthropic’s CEO had called for a co-ordinated slowdown in development.
Huang insisted to Klein that this was all a fuss about not very much. It’s simple, he said. An AI model is like any other product: if it’s not safe to release, you don’t release it, and if AI companies haven’t realised that already they will be forced to once they get hit by liability law suits. This should be no more difficult to fix than an unsafe car design. LLMs are software, and if a software product has a flaw - such as that it’s liable to wreak havoc on the world - then you fix the flaw. This is an engineering problem, not some existential mystery or intractable coordination challenge. The doomsters are spreading panic unnecessarily, and governments need to get out of the way.
Now, I am not an AI doomer. Partly through reading and cogitation, and partly through inchoate optimism, I tend to think we’ll work this one out. Even so, I found Huang’s arguments shockingly thin. The whole reason people are so scared by and excited about this technology - the whole reason Nvidia is the most valuable company in the world - is that it isn’t just software as we know it, but a new form which is by definition harder to control, since it’s probabilistic rather than deterministic, and behaves according to internal calculations which are opaque to its creators.
A second, gaping hole in Huang’s argument was his assumption that safety is a simple internal choice for individual AI labs, rather than a cost that will always get cut under intense competitive pressure, without binding regulation. This is before we get into some of Huang’s other opinions, such as that we needn’t worry about the erosion of human skills, since he can’t even remember his own address.
Ezra Klein seemed unconvinced by Huang too but had trouble standing up to the water-cannon force with which his interlocutor speaks. Huang’s speed of delivery, his unshakeable conviction, and the sense of latent anger just beneath his politeness, make him a difficult interviewee to grapple with. Dwarkesh Patel did a little better earlier in the year, on a different topic: whether or not Nvidia should be allowed to sell chips to China. Once again Huang’s arguments were perfectly aligned with his commercial incentives (why yes, of course we should!) and flimsy as paper.
Patel pushed Huang enough to rile him. My favourite moment came after Patel suggested that the Chinese, after taking advantage of American chips, would then push Nvidia and others out of the market. Huang exclaimed: “You’re not talking to somebody who woke up a loser! That loser attitude, that loser premise makes no sense to me!”1
That is not an argument at all, is it? It’s a furious assertion of will which overrides all intelligent inquiry, logic and nuance. And it says a lot about why Huang has founded a multi-trillion dollar company when almost everyone else on the planet has not.
The nagging question I always have when listening to Huang say things like this is, “Does he know he’s being stupid?” I use the adjective advisedly. I don’t think he’s stupid at all, for obvious reasons, and in his domain at least, he’s orders of magnitude cleverer than me. But that’s the point. How can someone so demonstrably clever make such obviously flawed arguments?
One answer to this is a Marxist-Machiavellian one: of course Huang knows he’s talking rubbish. He is saying this stuff to advance his material interests and those of his shareholders. While we can’t rule that out, that is not the impression I get. Huang really seems to believe what he says. In fact, if he didn’t, he might choose to make his false arguments subtler and more persuasive. My model of Huang is that he is delusional, but in just the right ways and to just the right extent for the job he does.
In 1989, the psychologist Roy Baumeister coined the term “optimal margin of illusion”. Baumeister’s theory was that a small measure of self-deception helps people to be happy. People who assess themselves and their prospects with ruthless accuracy are more vulnerable to depression and paralysis. This is “depressive realism”. Happy people view reality through a slightly fuzzy filter. A modest distortion is adaptive, but if our confidence gets too high we start making disastrous decisions.
The empirical evidence for Baumeister’s hypothesis is mixed. There is certainly evidence that “self-enhancement” - unrealistically positive views about yourself - is associated with greater happiness. There is less evidence for “depressive realism”. Depressed people often have negative illusions about themselves; this review found that, on average, depressed people make slightly less accurate judgements. Nobody has established where the optimal margin of illusion lies, or if there is one.
But I find Baumeister’s idea very useful. It certainly seems to apply to entrepreneurs. Huang seems to have an optimal margin of illusion about himself and his business. If he was totally delusional he would be a failed entrepreneur, constantly telling himself that the next business would be the unicorn. But if he was more realistic than he is, he would probably be running a decently-sized chipmaking company for computer games instead of a world-conquering behemoth.
In 2006, Huang made Nvidia spend hundreds of millions adding software to graphics chips so that they could do much more complicated maths, out of a vague but firmly held belief that there would be a big market in computing for science and engineering. Nvidia’s shareholders thought it was bonkers - people bought Nvidia chips to play video games, not to do science. But Huang forced the investment through, no doubt projecting the same mesmeric confidence in his judgement that he does in interviews.
Then came the AI revolution. The new chips turned out to be perfect for the complex calculations needed for deep learning and then LLMs, and Nvidia was on its way to becoming the lynchpin of the biggest financial boom in 200 years. Huang didn’t know that was going to happen. He just knew he wasn’t a loser. He made an irrational bet on an unrealistic intuition, and it paid off.
If this sounds like I’m trying to diminish his achievement, I’m not. In a sense, this is what successful entrepreneurship is. It’s living inside the optimal margin of illusion for the marketplace. In The Theory of Moral Sentiments Adam Smith blended economic theory with psychology to tell the story of “the poor man’s son” seduced by a grand, self-deceiving illusion: that great wealth will bring great happiness (Smith was somewhat suspicious of wealth as a personal ambition, even though he deemed it a social good).
The man toils away for years, flattering people he despises and serving people he hates. In old age, he finally realises that wealth - mere “trinkets” - hasn’t made him happy at all. Smith wasn’t arguing that the man’s ambition was folly. His point was that illusions are a source of energy and action. Social progress, he argued, depends upon self-deception, which “rouses and keeps in continual motion the industry of mankind.”
Self-deception also generates more risk-taking. Huang “got lucky” with the sudden emergence of LLMs as a massive source of demand for his fancy chips. But his margin of illusion put him in the right place at the right time to take advantage of it.
Of course, this means that entrepreneurs like Huang are rather unreliable witnesses to reality. Huang is single-minded in the most extreme sense: his whole cognitive function is bent towards the goal of advancing his business. That doesn’t mean we should dismiss everything he says. The rigours of the market mean he has to stay at least somewhat close to reality. Much closer, in a way, than most of us. Every day he processes a lot of information about his business, about technology and finance and politics (which apparently means he’s got no room left over for his home address), and responds in mostly sensible ways.
But where reality comes into sharp conflict with his business vision, it’s no contest. He constructs an illusory narrative in which LLMs are just software, safety is a mere engineering problem, and China will always find his American chips preferable to domestically made ones. That makes him say stupid things but without the stupidity he might not act in a way that grows his business, and that would be almost literally unthinkable.
I quite like Huang. When I reviewed his biography last year I suggested that its title, The Thinking Machine, was less than apt. Compared to his somewhat bloodless peers like Zuckerberg or Amodei, he is vividly human, a real poor man’s son with a furnace inside him and the charisma of a populist politician. But in this one sense he does resemble an LLM: he hallucinates. And as with LLMs, his hallucinations are a structural feature of his effectiveness rather than a bug.
After the jump: a Rattle Bag of delights, including the young Gary Oldman; a frustrated Stanley Kubrick; some good advice how to use AI for business; a terrifying ant war; thoughts on Keynes; news on the Beatles biopics; a stunning choral piece for a medieval composer I hadn’t heard of before…and more. Paid subscriptions are what makes the Ruffian possible. Thank you so much to all of you who have taken one.
