The Ruffian

The Ruffian

Reasonable and Unreasonable Men

On Dario Amodei’s Good Intentions

Ian Leslie's avatar
Ian Leslie
Nov 21, 2026
∙ Paid
8th Viva Technology : Day One In Paris
Dario Amodei, co-founder and CEO of Anthropic. Pic: Getty.

Catch-up service:
In Praise of Weak Kings
America’s Still Functioning Democracy
Why I Don’t Believe In Centaurs
On the Sally Rooney Case
The Optimal Margin of Delusion
How Civil Should a Disagreement Be?
Why Is the Left So Bad at Education?


In the same week as we read new predictions of an imminent intelligence explosion, Elon Musk reminds us that intelligence is an elusive concept. In a live interview he confidently asserted that America was behind the overthrow of Ukraine’s pro-Russian government in 2014. The most cursory investigation - a mere Grok query - would have shown him that this isn’t true; that it is a myth propagated by Russia and its useful idiots.

If Musk often finds that rumours are too good to check, that’s because his brain is optimised for profitable action, not accuracy. He is highly intelligent, of course, but he didn’t build trillion-dollar businesses by carefully weighing evidence and discounting for biases. That would only have got in the way of what he wanted to do. Rather than coolly assessing reality, he sets about it with hammer and chisel. He and Jensen Huang are the archetypal unreasonable men on whom progress depends. (I say this unironically, by the way. Nobody would be landing rockets without Musk being mad enough to believe he could make them.)

Dario Amodei, the CEO of Anthropic, has a different temperament to Musk or Huang. Where they are impulsive and volatile, he is systematic and logical. He writes long, lucid essays which make sober arguments. He weighs different scenarios, calculates trade-offs. He sweats over the ethics of AI. He has given Claude its own “constitution”. He employs philosophers, something it is merely funny to imagine Jensen Huang doing. He doesn’t come across as blatantly self-serving or quick to dissemble - but maybe that’s why we should be sceptical of him. Everyone expects Musk and to a lesser extent Huang to believe preposterous things, and so we discount for it. Amodei reaches his positions honestly and methodically. That doesn’t mean they’re not crazy.

In the week of Anthropic’s IPO, the New York Times has published a piece about the company’s internal culture, by the excellent Jasmine Sun. Sun spoke to current and former employees to find out how Amodei and his staff really think about the company’s mission. Many of them were about to become enormously rich. She found them unexcited by this prospect. They worked six days a week, twelve hours a day, and lived relatively spartan lives. They found it hard to think about what they might spend their money on. They were also weighed down by the responsibility of building a product they believe may well take over the world, with radically uncertain consequences for humanity.

Amodei has created a company in the image of himself. He is, as far as I can tell, idealistic, thoughtful, and concerned to do the right thing. His staff think and talk like him and share his beliefs about the future. Notwithstanding some emerging differences of opinion over corporate strategy, the company is very unified. This is in contrast to OpenAI, which, in the image of its CEO, has a hyper-political culture, riven by warring factions. That sounds bad, but I’m starting to wonder if an AI lab in which everyone shares the same mission and mindset might actually be more dangerous than one in which everyone is trying to undermine each other.

Sun’s piece identifies three assumptions on which Anthropic’s strategy rests. One is that the AI race is inevitable: a superintelligent AI is going to get built, whether Anthropic participates or not. A second is that whichever company builds this AI will possess enormous power. As a former employee puts it, it will control “the most dangerous cyberweapons in the world, the cures and recipes for every disease, and an army of tireless digital geniuses to do its bidding.” Amodei is said to believe that Anthropic may become more powerful than the US government and ought to start building relationships with nation states.

The third assumption can be framed as an answer to the question posed by the first two. Given that this thing is going to get built, and given it will be unprecedentedly powerful, then who should control it? Sam Altman? Mark Zuckerberg? Xi Jinping? Duh - it has to be us: Anthropic. We’re the good guys. Ergo, in the absence of collective action imposed by government, it is our duty to build AI as fast as possible.

This logic explains why Amodei is making cataclysmic predictions of AI’s impact on jobs while doing his utmost to advance it. It’s why, at the same time as he calls for government help in “pacing the frontier”, he is also focused on “racing to the top” - on building AI systems so superior that they force the rest of the industry to adopt its standards of safety. In Sun’s paraphrase: “Anthropic must keep winning to keep the world safe, and even if it stops, its competitors will not.”

There is a striking and somewhat unlikely coincidence of priorities here. Normally, we would expect commercial and social goals to be in conflict with each other, to some extent. That’s why we make businesses subject to laws and regulations. Amodei believes - I think genuinely - that he is pursuing wealth, power and public good all at the same time. Long and earnest reflection has convinced him that he has a moral imperative to build a multi-trillion-dollar company which is more powerful than governments.

I mean, that smells funny, doesn’t it?

Each of the assumptions on which Amodei’s strategy rests is questionable, but it is the third which seems most dubious. Why should we trust Anthropic to use its power for good? Come to that, why does Amodei trust himself to be a wise, all-powerful steward of civilisation? In the West, after a few hundred years of trials and errors, we concluded that kings should be tightly constrained or abolished. The logic of “this time is different” may apply to this revolutionary technology, but unless human nature is about to undergo a comparable leap forward, it seems unlikely that one man or one company can acquire so much power without using it for authoritarian and destructive ends.

To be fair to Amodei, he is calling on Congress to permit AI companies to coordinate a slowdown without falling foul of competition law. That could be construed as self-interested, too, of course, although I don’t believe he is Machiavellian. I think he looks into his heart and sees only good intentions. Anthropic employees look at him, and at each other, and see the same. But over time, his good intentions always converge with the maximisation of wealth and power. It was out of good intentions that Amodei held the release of Claude back in 2022 on safety grounds, letting ChatGPT go first. It was also out of good intentions that he then decided Anthropic should try and race ahead of OpenAI.

As Sun points out, there is a parallel with the technology here. We cannot parse the inner workings of LLMs, which means we can’t tell if they’re on our side or merely faking compliance. We can only observe their behaviour and adjust their constraints accordingly. Similarly, we cannot be sure whether or not a particular CEO or company is or will always be altruistic. But given everything we’ve learned from history, we ought to proceed on the basis that, under intense competitive pressure, they will choose what’s right for them over what’s right for society.

This returns us to the slipperiness of “intelligence”. You can be highly intelligent and believe false or impossible things. Highly driven people subordinate their rational faculty to their desires. Clever people are particularly good at rationalising their self-deceptions. Amodei is highly driven and very clever, but if he were wise he would not trust himself. He would take what Daniel Kahneman called the outside view, look at the base rate of how powerful leaders behave under pressure, and ask how likely it is that he will be at the far end of the distribution or off the edge of it. If that seems implausible, then his own predictions about AI imply that even a coordinated slowdown wouldn’t go nearly far enough. A more radical constraint on Anthropic’s power will be necessary.

Simply put, if any AI company were to become even half as powerful as Amodei predicts, we should want it under public control. Not because governments are wise, but because in a democracy, great power must be answerable to the people. This seems quite likely to happen anyway, one way or another. The more powerful that Anthropic or another lab becomes, the more it will come into conflict with the state. You can’t have two Leviathans. Anthropic’s skirmishes with the Pentagon were interpreted in the context of MAGA, but after Hegseth and Trump are gone, the government will still need to show that it is in charge, whoever occupies the White House. I wonder if Anthropic’s shareholders have this factored into their valuation of the company.

Either way, what Amodei and his co-founders have built in six years is astonishing. I don’t want to be too hard on him (especially if he’s going to be World-King). He is a business leader who thinks deeply about ethics and engages in argument and debate. That is good. I’m just wary of relentless, systematic reasoners who are convinced of their own correctness, especially when they are creating powerful entities which think like them. “It is not the slumber of reason that engenders monsters,” wrote Deleuze and Guattari in 1972, “but vigilant and insomniac rationality.” Insomniac rationality - what a way to describe Anthropic, and the machine it’s building.


Share

This piece is free to read so do share it if you enjoyed it. Thanks to all of you who have renewed your paid subscription recently, it is hugely appreciated. If you are a free subscriber who hasn’t taken the leap yet, jump in! The Ruffian depends on its subscribers.

After the jump: a Rattle Bag of goodies, including a couple of marvellous essays on the future of work; a explanation of Musk’s reluctance to give Ukraine full access to Starlink; the question of where outer space begins; how American movie theatres are being repurposed; a new (to me) Beatles conspiracy theory, and the insane legal backstory to the one of the iconic songs of the 1990s.

User's avatar

Continue reading this post for free, courtesy of Ian Leslie.

Or purchase a paid subscription.
© 2026 Ian Leslie · Privacy ∙ Terms ∙ Collection notice
Start your SubstackGet the app
Substack is the home for writing, podcasting, and media founders.