Stuart Russell says trillions of dollars have already gone into the current approach to artificial intelligence, trillions more are planned, and the systems it produces cannot be shown to be safe.
The Treasury Secretary told Congress the same week that making the creators liable for what they build is the best way to guarantee safety. Russell said liability does not work on this risk, and used the Boeing 737 MAX as his reason.
"These are gigantic black boxes. They have literally trillions or now even tens of trillions of parameters, and we have essentially no idea how they work."
Russell holds the Smith-Zadeh Chair in Engineering at the University of California, Berkeley, directs its Center for Human-Compatible AI, and took part this week in the bipartisan pro-human rally in Washington.
The full segment is covered here so you can skip it.
Here are the 5 insights that matter.
👤 Guest: Stuart Russell, Professor of Electrical Engineering and Computer Sciences at UC Berkeley, holder of the Smith-Zadeh Chair in Engineering and Director of the university's Center for Human-Compatible AI
🎙️ Hosts: Jonathan Ferro and Lisa Abramowicz, who anchor Bloomberg Surveillance on Bloomberg Television
🧩 Other segments: Gina Martin Adams, Chief Market Strategist at HB Wealth, and Jim Zelter, President of Apollo Global Management
📰 Published: 16 September 2026 on the Bloomberg Surveillance feed
🟣 Apple Podcasts | 🔗 Episode page | ⏱️ length not available
Key Takeaways
The money already spent is not an argument for spending more, and he calls the reasoning a sunk cost fallacy
The mistake he names is the technology itself: the large language model path chosen around 2018 and 2019
Systems with trillions to tens of trillions of parameters that nobody can explain
The safety standard used for nuclear reactors and aircraft cannot be met by a system nobody understands
Liability cannot deter the risk the companies say worries them most, because nobody would be left to pay or collect
Boeing's bypass of certification killed about 340 people and handed the American lead in civil aviation to Airbus
What he wants is behavioral red lines plus market-access requirements before release, not liability alone
Breaking into other computer systems is already a crime when a person does it
1. The Sunk Cost Isn't a Reason
Ferro introduced the segment against the week's news — the Treasury Secretary testifying on the future of artificial-intelligence regulation while senior administration officials met Anthropic's co-founder and chief compute officer, Tom Brown, about safety risks.
Russell's written position, which Ferro read out, is that the scale of the spending is not a reason to continue: "we've already sunk trillions of dollars into the current technology path and plan to sink trillions more. But the sunk cost fallacy is just that."
Asked what mistake he thinks is being made, he answered on a long horizon rather than a market one. "So I think this is a long-term mistake," he said.
Russell was in Washington this week for the bipartisan pro-human rally, which is the setting Ferro used to introduce him.
2. The Path Was the Mistake
His objection is not to a company or a product but to the class of system the industry standardized on.
"The technology path that the companies chose back around 2018, 2019 was the large language model," he said, naming ChatGPT and Claude as the examples a viewer would know.
"These are gigantic black boxes. They have literally trillions or now even tens of trillions of parameters, and we have essentially no idea how they work."
The consequence is a standard that cannot be met. He put it this way: "So as they're becoming more capable, the problem is that we need the kinds of safety guarantees that we have for nuclear reactors and airplanes and so on, but we just can't get those guarantees."
"And we have no way of stopping these systems from doing things that we don't want them to do. So the only guarantee we can get is that they are not safe," he said.
His conclusion is that more capital does not fix it: "So my argument is that the technology path itself was a mistake," he said, "and it's not going to get better by pouring more money into it."
3. Liability Won't Deter It
Ferro put the counter-argument the administration is making, reading the Treasury Secretary's line verbatim — "The best way to guarantee safety is that the creators are liable for what they build" — and asked whether that is sufficient. Russell said no, and gave two reasons.
The first is that the risk the companies themselves name cannot be insured against. He said the risk uppermost in the mind of Anthropic's chief executive is loss of control, which means "the systems have become intelligent enough that they end up taking control of the world, and there's nothing we can do to get that control back."
"And after that, we really have no say in what happens at all," he said.
The financial point follows from that: "And obviously, liability is not going to deter a company from incurring that risk, because if it does come to pass, there will be nobody around to pay the money and nobody around to receive the money."
The second reason is practical rather than existential: "But the second more important point, from a practical point of view right now, is that we need more than just liability."
4. What Boeing's Bypass Cost
The airplane comparison is where he made the case that liability and pre-approval are two different things that both have to exist.
He accepted the liability half of it: "Of course, if there's a defect in an airplane and it results in a crash, then the company is liable. And we saw that with the Boeing 737 MAX 8. But that doesn't mean we don't also have safety requirements for the airplanes."
What an airworthiness certification actually requires, he said, is that "The company that develops the aircraft has to do enormous amounts of flight testing and structural integrity testing and mathematical analysis of the design of the aircraft and software that runs the aircraft and so on."
The price of skipping it is the part he put numbers on: "And exactly what happened when Boeing asked for basically a bypass on that certification process was that they ended up killing 340-something people." He said the company lost about $80 billion and ceded the American lead in civil aviation to Airbus, which he described as having a much stronger focus on safety.
His prescription is both halves: "Requirements before a developer has market access, as well as liability after the fact."
5. Red Lines, Not a Ban
A co-host pressed on the difference between the two cases: an aircraft has one obvious failure to prevent, while artificial intelligence does not. "When it comes to AI, what's the ultimate goal? To prevent artificial general intelligence or to just make sure that some of these companies are keeping the models in a box?"
Russell agreed the range of things an artificial-intelligence system can do wrong is much wider than an aircraft crashing, and called it a good question.
His answer is a concept he called behavioral red lines — a list of prohibited behaviors rather than a cap on capability: "So red lines just demarcate things that we absolutely do not accept that AI systems will do. So that includes AI systems breaking into other computer systems."
That first example is already settled law for people, he said: "So we have clear definitions from the legal process that applies to humans."
His other examples are systems that improve their own capabilities without human authorization and oversight, and systems that give advice on how to build biological weapons. He said the list does not have to be exhaustive.
The reason a short list is enough, on his argument, is that meeting it forces the underlying work: "The point is that as the companies develop the understanding of their own systems and the safety engineering technology to make sure that they can provide evidence of safety with respect to these red lines, then that process will lead them to prevent all the other kinds of harms that we haven't yet enumerated."
Bonus Insights
The segment ran in a programme otherwise given over to the Federal Reserve decision, and Ferro moved into it directly from the market open.
Russell did not ask for the technology to be withdrawn from the market. Everything he proposed — certification before release, red lines, liability afterwards — is a condition on how it is sold, not a prohibition on building it.
He treated the Boeing case as a commercial argument as much as a safety one: the certification bypass cost the company money and market position, not only lives.
Russell's bottom line is that the industry picked a technology in 2018 and 2019 whose workings nobody can explain, that no amount of further spending makes it explicable, and that the fix is the aviation model — evidence of safety against defined red lines before a developer gets market access, with liability on top rather than instead.
Products, Companies & Tools Mentioned
ChatGPT and Claude (His examples of the large language model path he says was the wrong choice)
Anthropic (Whose chief executive's stated fear of loss of control is the risk he says liability cannot deter, and whose co-founder Tom Brown was meeting administration officials that week)
Boeing and Airbus (The 737 MAX 8 certification bypass is his central case: about 340 deaths, roughly $80 billion, and the American lead in civil aviation lost to a competitor he says was more focused on safety)
Listen to the full episode:
If this was worth your time, send it to someone closer to the industry than you are.
Get the latest market chatter as it happens: