The Twenty Minute VC (20VC) Sep 19, 2026 1h 11m 44m saved
With Thomas Sohmers, Co-Founder and Chairman of Positron AI
Thomas Sohmers sells hardware to the companies most often described as burning cash, and he thinks that description is backwards. His view is that OpenAI and Anthropic are already high-margin businesses, and that the money going out of the door is training spend they could stop at any point.
The common worry is that the AI labs are one funding round away from trouble. Sohmers said the opposite: the cached tokens they sell are close to free to serve, the margins on them are what he called obscene, and the only reason the profit does not show up is that it is being spent on the next model.
Like if they stopped training, they'd be massively profitable overnight.
Sohmers co-founded Positron, a Reno-based inference-chip company that raised $875 million at a $5 billion valuation this year, and he has been building semiconductors since he was a teenager. He was in the room at NeurIPS in 2022 when OpenAI mentioned, in passing, that it had launched something called ChatGPT.
The full interview is covered here so you can skip it. 71 minutes of audio, 27 minutes of reading.
Here are the 17 arguments that matter.
Key Takeaways
The AI labs are profitable businesses running a training bill, not loss-makers running out of road
Cached tokens are sold at a markup and cost roughly a thousandth of a regenerated one
Anti-data-center politics is the risk he takes most seriously, and he says it is almost entirely Chinese in origin
His counter-example is that a single In-N-Out uses more water than the largest American data centers
A data center cannot legally take power already allocated to homes, because utilities hold a reserve buffer by rule
Memory, not compute, is the binding constraint on inference — GPUs got roughly 120x more flops in a decade and only 17x more memory bandwidth
About 96% of tokens in a real agentic coding session are cached, on SemiAnalysis's AgentX traces
50 users can consume more memory than the model itself on a long-context frontier model
He trusts Oracle's balance sheet more than the US government's — the difference is that Washington can print and tax
Scaling laws are an observation, not a proof, and he called continued scaling "sort of on vibes"
A local model on your phone will increase cloud token demand, not cut it
The bottleneck he names in AI products today is a human being asked to make a decision
Nvidia is cutting memory per device while Positron says its next generation carries eight times the memory of Nvidia's largest part
1. Inference's Memory Wall
Positron builds chips, the software that runs directly on them, and the racks the whole thing ships in, all aimed at running generative AI models rather than training them. Sohmers opened by separating the two jobs.
Training is compute-bound: more parameters and more floating-point operations produce a better model, which is why export controls are written around flops. Generating output works differently. Every single token has to be read out through the model's weights, one after another, with no way to know what the fifth word ahead will be.
Training can be parallelized; generating text cannot
But when you're inferring, because that's actually generative, you don't know what the token is, you know, five words down the line.
Thomas Sohmers
The host asked whether that is what people mean by the memory wall. Partly, Sohmers said. The phrase predates the AI boom and describes a divergence: transistor density kept improving, memory technology did not keep pace.
Compute pulled away from memory by roughly 7x over a decade
The improvement in memory bandwidth is only 17x.
Thomas Sohmers
He put the flops gain on a single Nvidia GPU between 2014 and 2024 at about 120-fold against that 17x on bandwidth. Asked why the two diverged, he gave two reasons. The physical one is SRAM, the memory built onto the silicon itself, whose six-transistor cell has barely shrunk in about fifteen years and has not changed architecturally in three or four decades. The commercial one is that nobody needed it. The convolutional networks that drove the 2010s, from AlexNet in 2012 through ResNet, were compute-hungry and memory-light. That only changed with the transformer, and he dates the real recognition of it to GPT-3 in 2020 rather than to the 2017 paper.
2. The Cached-Token Margin
Asked what nobody talks about in token economics, Sohmers went to the cache. Providers now list separate prices for cached and uncached input, and he said almost nobody appreciates how profitable the cached side is. Writing something into the cache is billed above the normal input rate; reading it back is billed below. The read costs the provider about a thousandth of what regenerating the token would.
The profit sits in the cheap half of the price list
You make all of your money on selling cash in input and output tokens.
Thomas Sohmers
The host asked whether the reported gross margin at Anthropic had surprised him.
He was impressed rather than surprised
I'm impressed by 80 points of margin in basically any industry.
Thomas Sohmers
He added that he expects competition to compress those margins, that he is fine with it even though those companies are his prospective customers, and that his own margin depends on theirs. What he finds absurd is the persistent belief that these are unprofitable businesses heading to zero.
The cash burn is a choice, not a condition
Like if they stopped training, they'd be massively profitable overnight.
Thomas Sohmers
3. Against Pacing the Frontier
Dario Amodei's essay arguing that the industry should slow capability gains came up repeatedly. Sohmers's first response to it was about incentives.
He reads the slowdown partly as pre-IPO cost control
I mean, I jokingly think that, you know, a little bit of the pacing the frontier discussion is, oh, this is a great way to reduce costs ahead of IPO.
Thomas Sohmers
He then said he does not think Anthropic needs to do that, and that training is the bulk of its cost base. On the substance of safety he was more divided.
He is not dismissive of the safety case
I have mixed feelings on the safety topic.
Thomas Sohmers
His objection has two parts. The first is that arguing for a pause hands ammunition to people who want the technology stopped outright.
A pause argument strengthens the abolition argument
And I see that as a major risk for humanity.
Thomas Sohmers
The second is concentration. His worst case is not a rogue model but a world where the capability sits with a handful of companies and governments — what he described as a modern road to serfdom, with the rest of the population locked out. He extended that to the people asking for the rules: they assume regulation will leave them in charge, and he does not think it will. On Amodei specifically he was careful, calling him and most of Anthropic true believers in both the promise and the risk, and saying he would take the same responsibility in their position.
The host suggested the whole exercise is convenient for everyone involved — compliance cover before an IPO, a reason for OpenAI not to list, and time for Elon Musk's team to catch up. Sohmers agreed that is probably the internal motivation for everyone except Amodei himself.
4. The Chinese Psyop Claim
The host noted that Mark Zuckerberg had come out against pacing while Jensen Huang said nothing, and asked what to read into that. Sohmers said he weighs any commentator's motives, and that he would take Zuckerberg's or Amodei's contribution over a politician's. Then he named what actually worries him, in the line the show used for its title.
The politics, not the technology, is what alarms him
The scariest thing to me on the political spectrum is that it's now become almost unifying issue on left and right about being anti-data centers.
Thomas Sohmers
He said that turn is "almost entirely" a Chinese psyop. Pressed on why, given real complaints about electricity bills, water and appearance, he split the answer. On appearance he agreed and wants a beautification program, arguing the largest sites should be dressed up as the technological world wonders he thinks they are.
On the resource claims he said the early reporting was written by people who did not understand the systems, and that most large facilities are closed-loop liquid-cooled and use little water at all.
His water comparison is a single fast-food restaurant
A single In-N-Out uses more water than the largest data centers in the United States.
Thomas Sohmers
The strategic half of his argument is that China is doing the opposite of pacing while Western debate slows itself down on bad information.
China is adding capacity and moving people out of the way
They're building massive new data centers, horribly displacing people.
Thomas Sohmers
He said he values the freedom to criticize companies and governments, and that it is still a strategic disadvantage against a state that can bulldoze homes and run rolling blackouts to free up training capacity.
5. Defending Data Centers
The host put it to him that neither the US nor Europe can beat a country with no planning restrictions, and said the UK now requires a permit for almost anything. Sohmers agreed, and said America's remaining advantage is land.
He argued that the large majority of federal land in the West, which he put above 90% while saying he did not know the exact figure, is empty desert outside any national park, and that restrictions on using Bureau of Land Management ground for data centers hundreds of miles from anyone are absurd.
He wants his own state used for this
My great state of Nevada has plentiful geothermal, solar all these green energy technologies and we could, you know, build, you know, nuclear and other things in the middle of the desert where it won't impact anyone.
Thomas Sohmers
He said Republican governors who ran on growth have started backing away from data centers in the past year because their own base has turned against them.
Then he took the electricity-price objection directly. Utilities are required to hold a buffer of available generation, which is priced into everyone's bill, so a new facility cannot draw on power already committed to homes.
The power a data center uses was never allocated to households
There is absolutely zero cases where a data center could be potentially pulling power from anything that's already been allocated.
Thomas Sohmers
New sites arrive with their own generation attached
And all these data centers that are getting built right now are coming with generation capacity that covers their own use and beyond that.
Thomas Sohmers
His complaint is about what happens to the surplus.
Blocking grid connections keeps prices higher than they need to be
And we're just not allowing them to hook up to the grid where they could actually be lowering the prices for everyone.
Thomas Sohmers
He added that power companies themselves lobby against new generation, because more supply would lower prices.
6. Will They All Get Built?
Asked what share of announced data centers actually gets completed, Sohmers said the capacity gets built but the map changes. Some communities have blocked individual sites; those projects move elsewhere.
The industry underestimated the politics
I don't think a year ago the major data center builders and operators were thinking that the political problems were as bad as they were.
Thomas Sohmers
More money is now going into local education campaigns, which he expects to help, and the rest of the capacity relocates to places that will take it. He does not see the backlash as a threat to the buildout.
There is a further option if one company delivers
And then of course there's space if Elon's successful.
Thomas Sohmers
The host asked whether orbital data centers are real or a story told to support a market capitalization. Sohmers said a thing can start as one and become the other.
He is not taking the other side of that trade
I would never ever bet against Elon.
Thomas Sohmers
A year ago, he said, he would have seen no near-term reason for it, because building on land is cheaper and easier. He named an alternative he is closer to: Panthalassa, a company he is partnered with and whose chief executive he counts as a friend, which is building ocean-based data centers around a pumped-hydro design in open water. His reason for backing space anyway is longer-dated.
The case for orbital compute is the case for a space economy
I just think we're going to need to have a space economy for humanity to live up to its long-term potential.
Thomas Sohmers
7. Energy or Economics?
Asked whether energy is the real bottleneck, Sohmers answered with what more efficient hardware actually does to demand. Positron's pitch is more compute per megawatt; if a job that needs 500 megawatts of Nvidia equipment can be done in 100, he does not expect anyone to build the smaller facility.
Efficiency buys more output, not a smaller building
You're still going to build the maximum amount of compute that you can.
Thomas Sohmers
On the long view he said human progress tracks the ability to produce and use energy, from fire to nuclear power.
Everything eventually reduces to an energy question
The simple tongue-in-cheek answer to your question is everything all progress is gated by energy.
Thomas Sohmers
But he does not think generation is the binding constraint now, because the technology to build it exists.
The constraint he names is money, not electricity
I would say we have got way more economic limitations.
Thomas Sohmers
The specific question, he said, is how much debt the world is willing to take on over the next few years.
The host asked whether the concern about the AI debt cycle is justified. Sohmers said his worry is sovereign, not corporate. He sees rising perceived risk in Treasuries and the broader bond market feeding through everything beneath it, and a currency devaluation as the acute crisis he expects.
He rates Oracle's credit above the government's, with one caveat
I believe in Oracle's business model and ability to execute and do everything a whole lot more than United States government.
Thomas Sohmers
The difference is coercion, not management
It's just the United States government can issue its own currency and also has guns and nukes to take tax revenue.
Thomas Sohmers
He is not worried about the AI borrowers themselves
I'm really not worried about any of the companies in the AI debt stream like not hitting their revenue targets.
Thomas Sohmers
8. KV Caching, Explained
The host said he had gone deep on KV caching while preparing and wanted it explained. Sohmers started with vocabulary. A token is a chunk of text the model's training process settled on as an efficient unit — often a prefix or suffix rather than a whole word, and worth somewhere between half a word and three-quarters of one in English. A sequence is those tokens in order, and the sequence is what fills the context.
In the original transformer, generating each new token meant redoing the computation for everything already in the sequence. The fix was to store the intermediate results instead.
The cache trades computation for memory
And so the KV cache was born where within the model there's these two matrices called K and V keys and values and those matrices are fully based on the prompt and whatever is generated during a turn
Thomas Sohmers
The reason it matters is that the two costs grow at different rates. Attention compute per token grows with the square of sequence length; storing the keys and values grows linearly with it. The price is that every user session has its own cache, which raises questions about how long to keep it and where to put it.
9. Quantization's Real Cost
Asked what compression means here, Sohmers named two kinds and took quantization first. Before machine learning, computation was done in 32-bit floating point. That turned out to be far more precision than these values need, so the industry moved to FP16, then Google's BF16, then FP8, and FP4 is in use now.
Shrinking the number of bits loses information. Taking sixteen BF16 values and truncating them to four-bit integers saves three-quarters of the space and costs real accuracy.
Naive quantization takes a visible bite out of benchmark scores
But just doing that naively will mean that on a lot of benchmark scores you'll have them go, you know, get 20 30% worse.
Thomas Sohmers
The current technique stores one shared 16-bit multiplier alongside each group, applied at compute time, which recovers most of the loss. He said the state of the art gets a model down to between four and five bits per value, applied to weights and to caches alike, and that the remaining damage is held to about 1% against an unquantized model. His framing throughout was that there is no free lunch.
10. Cache Hits and Big Models
Asked whether caching is the hardest part of running inference, Sohmers said a service can be run without it — badly.
Caching is optional and running without it is expensive
You can run a service and do something without having KV caching at all.
Thomas Sohmers
What makes it decisive is which workloads turned out to be worth money.
The valuable workloads happen to be the cacheable ones
The dark art and magic with it is the workloads that the industry so far has found the most valuable happen to be very very highly cachable.
Thomas Sohmers
He cited SemiAnalysis's AgentX benchmark, which replays real coding-agent sessions with dozens to hundreds of turns and sub-agents.
Almost all of an agentic coding session is re-read, not regenerated
And what they found is over these massive number of interactions of these like real traced code generation you know agentic coding sessions about 96% of all the tokens that go through these entire sessions are cached.
Thomas Sohmers
That changes where the memory goes. A model's weights are fixed; user sessions are not. He worked the arithmetic on a frontier-scale model, using his own expectation of what the next Anthropic model will weigh rather than a published figure.
He expects the next generation of frontier model at 10 trillion parameters
If we take like the high expectations of like Claude Fable you know, that's a 10 trillion parameter model.
Thomas Sohmers
At long context lengths on a model that size, he put an individual user session at around 100 gigabytes.
A few dozen users outweigh the model itself
So with just 50 users on your service, the user context that just those individual sessions end up being greater than the model weights that you're trying to store.
Thomas Sohmers
The operator's trade-off is therefore how much accelerator memory to spend on weights, which sets the speed a user perceives, against how much to spend keeping caches alive, which he said is mostly an economics question rather than a latency one.
11. Bigger Models, Not Smaller
Sohmers described how caches are stored today: weights and active sessions in accelerator memory, recently idle sessions in host memory, which he said runs four to ten times larger, and older ones pushed out to NVMe flash or network-attached storage. He expects his own months-old ChatGPT sessions to be sitting on a slow disk somewhere, which he said is the correct place for them. The cost of that hierarchy is deciding what goes where, for a very large number of users.
The host stopped him on the claim that models will get bigger, given the enterprise pitch that everyone will own a small private model. Sohmers said both happen, in two tiers. Frontier development at OpenAI, Anthropic and possibly Google keeps pushing size and capability, and for his own company's internal use he would pay ten times more per token for ten times the output.
He put the top four model companies at around 80 to 85% of all tokens consumed and produced, with the next three or four accounting for another five to ten, and said he can accept that roughly 5% ends up on-premises. His commercial interest is in the high-volume side.
Where he disagrees hardest is on what local models do to cloud demand. The assumption is that they substitute. He thinks they generate.
A phone model raises cloud consumption rather than replacing it
I think it's the opposite.
Thomas Sohmers
His reasoning is that a person prompts a chat assistant only as often as they think to, while a local model watching email, calendar and messages runs continuously and escalates whatever needs more intelligence to a hosted model.
On size itself he was explicit that the evidence is empirical rather than proven. The field calls them scaling laws, he said, because the pattern has been observed, not because anyone has proved it will hold. Continued scaling, on his account, rests "sort of on vibes".
He expects the curve to run well past today's largest models
So is that going to continue to 50 trillion 100 trillion and beyond and I don't see any indication that's going to stop.
Thomas Sohmers
He also named context, not parameters, as the live constraint.
Context is what limits an agent, not raw model capability
I think the main limiter today isn't like the model capabilities itself and scaling the model size it's on how much context can that model have of all of the data it needs to make smart decisions.
Thomas Sohmers
12. GPT-6 Astra as AGI
Asked what three times current capability looks like, Sohmers said his first day with GPT-6 Astra matched his first day with ChatGPT. He was at NeurIPS in New Orleans in December 2022 when OpenAI launched it, and said the announcement came almost in passing at the end of a party, with Sam Altman and Ilya Sutskever there and what he called "zero fanfare". He went back to his hotel and opened it.
The first ChatGPT session cost him a night's sleep
And I was up for four or five hours straight just giving random prompts and just being that this was the most magical experience that I've ever had with a computer.
Thomas Sohmers
He said Sora 2's launch weekend came close to the same feeling, and that he does think the current OpenAI model qualifies as artificial general intelligence.
Pressed on why, since the host had found it much like its predecessor, he separated three things. On general coding it is a step change but not a shock, though it solved problems in one shot that other models had circled without fixing, and found performance issues and bugs in parts of Positron's codebase he did not know existed. On computer use it was a jump: he had it produce Blender animations and do the interior design of his house from a few photographs.
The third case is the one that matters to his own business. He has tested every model release on taking a simple logic design, an encryption block, through the full flow from specification to Verilog to a finished chip layout. Earlier models managed pieces and failed in the middle, in part because the electronic design automation tools date from the late 1990s, are unintuitive, and are barely documented in public.
The tools these models had least to learn from
None of the documentation exists out like in the public web.
Thomas Sohmers
This time the model took the design through to a finished layout on a TSMC 3-nanometer process, meeting timing above a gigahertz, in a little over fifty hours. He estimated a person new to the flow would need about a week to get it working and one to two weeks more to optimize it.
A two-to-three-week job came back in two days
So compressing that 2 to 3 weeks down to 2 days and change
Thomas Sohmers
He said he is glad OpenAI is shipping those capabilities publicly rather than keeping them for its own chip program.
13. Everyone Builds a Chip
The host listed OpenAI's Jalapeño, Anthropic's own silicon effort and DeepSeek's, and asked what to make of the chip layer commoditizing. Sohmers, asked to answer with his Positron hat off, said it is good for the industry, lowers costs and widens access.
He noted how far the field has moved since he entered it: thirteen years ago, he said, "silicon was a dirty word in Silicon Valley", and now the largest companies in the world are all connected to the semiconductor industry and the most interesting ones are integrating down to it.
The goals converge and the engineering does not
The interesting thing with all the ones that you mentioned and you know the broader set is the companies have the same macro goals.
Thomas Sohmers
He said the implementation details differ in every case, which is what makes it interesting to an engineer, and praised the industry for still discussing designs in public — less than five or six years ago, but still a real amount. OpenAI unveiled Jalapeño at Hot Chips, the main conference for the field, and he said he was happy about that.
His closing view on the competition is that the market settles it
And the great thing about the market is that the market gets to decide what is valuable and those that create value will receive reward for that.
Thomas Sohmers
14. The Limits of Context
Asked whether context windows can keep expanding, Sohmers said not with conventional attention, because memory cost grows with the square of the length. Going from a million tokens to ten million is, in his words, really hard. That is the constraint Positron is building against.
The next Positron generation is a memory bet
So, you know, with our upcoming generation, we're going to have, you know, eight times more memory capacity than the highest memory skew from Nvidia.
Thomas Sohmers
Nvidia is moving the other way
And Nvidia is actually decreasing the amount of memory per device you know, based on the market memory conditions.
Thomas Sohmers
The algorithmic route around the problem is linear and sparse attention, and he credited one group with most of the progress.
Export controls produced the workaround
And those have really really been innovated by the Chinese model labs.
Thomas Sohmers
He gave DeepSeek V3, at the start of 2025, as the example: multi-head latent attention spends more compute to hold a much smaller cache. His current favorite is gated DeltaNet and its derivatives, which he said can cut the time spent on the attention step by about three-quarters.
Asked how that squares with needing new hardware at all, Sohmers said the compression is not free, because it costs model capability, and that he has good reason to believe no major US lab is using multi-head latent attention or its close relatives, though he expects that to change.
He also drew a distinction between advertised context and usable context. Models from two years ago claimed a million tokens and lost the thread above about 64,000.
A stated context length is not a usable one
So just saying that something has this maximum context length is one thing. It's can it actually use that context length effectively is an entirely different thing.
Thomas Sohmers
He used the RULER needle-in-a-haystack test as the measure: a value hidden in a flood of junk text, with the model asked to find it. He said the six-or-seven-week-old previous OpenAI model managed it about 70% of the time and the current one is above 95%, which he took as evidence of how much room is left.
15. What a Token Is Worth
The host cited the Silicon Data token price index, which he said fell below $1 per million tokens this month against $60 five years ago, and asked what a million tokens costs in 2028. Sohmers rejected the frame before answering it.
The cheap token and the expensive token are not the same good
Like that was a complete garbage token, relatively speaking 5 years ago.
Thomas Sohmers
He walked through the comparison. The best model in the world in 2021 was GPT-3, released in August 2020, and nothing better arrived until ChatGPT in November 2022 — two and a half years, with GPT-3.5 being reinforcement learning from human feedback on the same base. Against what a current model does for a person's work or a company's operations, he put the improvement at a hundredfold to a thousandfold.
The price fell 60-fold and the value rose by more
So I think there's actually two points to your axis of going from $60 to $1.
Thomas Sohmers
His own multiplier is deliberately conservative
Yes, that's a decrease in cost, but that token today is let's just say I think conservatively 100 times more valuable.
Thomas Sohmers
16. Pricing Beyond Tokens
Asked whether cost per token is even the right metric, Sohmers brought up Greg Brockman saying at the GPT-6 launch that OpenAI does not expect to price in tokens much longer, and wants to charge for useful results instead. He does not think that is where it lands, because a useful result is hard to price.
Tokens survive because the cost of one is calculable
But I think that the price per token is really great because you can easily calculate the cost like the cost to generate a token.
Thomas Sohmers
The change he does expect is at the top end. If a future model can do a job the way an employee does, and the provider knows what it costs to run continuously, the unit being sold stops being the token.
The frontier product may be priced as a salary
They may decide that it's actually easier and they'll be able to get more adoption if they just charged a million dollars a year
Thomas Sohmers
He was explicit that the million was a made-up number used to illustrate the shape.
17. Where a Crack Would Show
The host quoted Gavin Baker on being unable to name a company whose numbers are not going up, and asked what the first crack would look like — a shift from frontier models to open weights, or the two leaders missing growth targets. Sohmers accepted it as possible and said he does not think it is likely, for the same reason he gave on local models.
Open weights feed the incumbents rather than starving them
The reason I don't think it's likely is I think that the development of open source models and things happening locally etc will actually drive greater use greater you know token volumes for the big guys.
Thomas Sohmers
What limits AI usage today is a person, not a model
I really do think that in a lot of AI applications right now, the bottleneck is actually a human making some sort of decision
Thomas Sohmers
He said he already trusts the current frontier model more than himself on a range of tasks, and still initiates every job it does, even when it then runs for twelve hours or three days.
The next order of magnitude comes from delegation, not capability
I think the next big leap is when I trust a model running on my laptop or on my phone to prompt the smarter models to do even more wider set of tasks.
Thomas Sohmers
The host then raised the data-labeling layer, putting Surge's revenue at $3 billion and asking whether Mercor and Surge become $200 billion businesses alongside $4 trillion to $5 trillion labs. Sohmers's only reservation was that the labs might take the work in-house.
Labeling is outsourced because the labs have better uses for their attention
I think the reason that hasn't happened is because Anthropic and OpenAI have better uses of their capital and mental power
Thomas Sohmers
That reasoning has an expiry date in his account. If a model can manage a large business the way a chief executive does, the argument for paying someone else to do the work disappears.
The in-house case arrives when agents can run the company
Why they wouldn't just have agents be taking over those tasks?
Thomas Sohmers
Bonus Insights
The host set the terms of the geopolitical section himself
Introducing his own question about China's open ecosystem, the host described himself this way: "I'm a podcaster and a venture capitalist for a living. So one of the lowest IQ on the spectrum." He later said he does not understand how the US or Europe can beat a country with no regulatory restrictions, and that in the UK you cannot put up "a paper airplane without getting a permit".
Sohmers dismissed the Terminator scenario without dismissing the risk
I think of course everyone can agree Terminator or similar is very bad, but I think is extremely low probability
Thomas Sohmers
His China argument is that openness is a phase
I think as soon as they get into you know pole position the ladder gets pulled up with them in some way.
Thomas Sohmers
He added that he does not know whether Beijing would let the rest of the world keep access, but that "I don't know if they will decide that it's okay if the rest of the world has some access to the technology but they definitely will not let the billion people that are not CCP party members you know benefit equally from the technology."
He supports free trade with an exception for closed states
But totalitarian regimes should not be able to participate with that especially in the case where they get all of the benefits of that and get to export themselves you know things that make them better able to have that totalitarian you know system keep up.
Thomas Sohmers
On export controls, his objection is that they do not hold
Sohmers said the assumption that the West is too far ahead to be caught is the same naivety he sees in company leaders. His analogy was military: the US has the strongest armed forces on paper, and would struggle with a drone incursion across the Mexican border of the kind now routine in Ukraine and Iran.
I we've we built our military to fight the last war.
Thomas Sohmers
The host's framing of pacing was that it suits everybody
The host argued the pause debate serves four interests at once — regulatory cover, a reason for OpenAI not to list while its numbers trail Anthropic's, time for Musk to catch up, and a shield for the labs. Sohmers agreed it is the biggest internal reason for everyone bar Amodei.
A token is not a word
Sohmers's working figure is that a token equals between half and about three-quarters of a word in large English corpuses, and that the flickers of partial words on screen when a model streams output are the individual tokens arriving.
Sohmers's bottom line is that the constraint on AI is no longer compute or even electricity but memory, money and politics — and that the political half of that will be settled by people who currently believe the technology takes water and power away from them.
Products, Companies & Tools Mentioned
Positron (His company: chips, the low-level software and full rack-scale systems for generative AI inference; raised $875M at a $5B valuation this year)
Nvidia (The reference point throughout — about 120x more flops per GPU over a decade against 17x more memory bandwidth, and now cutting memory per device on market conditions)
OpenAI and ChatGPT (He was at the 2022 launch; says the current model is AGI and took a chip design to a finished layout in about fifty hours)
Anthropic and Claude Fable (Reported at 80 points of gross margin on its API business; he expects the next frontier model at 10 trillion parameters)
Jalapeño (OpenAI's own inference accelerator, unveiled at Hot Chips; his example of the chip layer commoditizing)
DeepSeek (V3's multi-head latent attention cut cache size at the cost of some capability; he says no major US lab is using it)
Panthalassa (A partner company building ocean-based data centers on a pumped-hydro design; his non-space alternative)
SpaceX (Orbital data centers as the fallback if land keeps getting blocked — "I would never ever bet against Elon")
Oracle (He rates its business and execution above the US government's, while conceding Washington can print and tax)
TSMC (The 3-nanometer process the model used to take his test design through to layout)
Google (Named as a possible third frontier lab, and as the originator of the BF16 number format)
Bureau of Land Management (The federal landowner whose restrictions he says block data centers in empty desert hundreds of miles from anyone)
In-N-Out (His water comparison: one restaurant against the largest American data centers)
Blender (The animation software he used to test the current model's computer-use ability)
Mercor and Surge (The data layer the host valued at $3B of revenue for Surge; Sohmers's caveat is eventual vertical integration by the labs)
Hot Chips (Where OpenAI showed Jalapeño, and his evidence that the field still discusses designs in the open)
NeurIPS (New Orleans, 2022 — where ChatGPT was announced to him with "zero fanfare")
Books & Resources Mentioned
We Must Pace the Frontier (Dario Amodei's essay, the spine of the first half; Sohmers reads part of the motivation as pre-IPO cost control)
Attention Is All You Need (The 2017 transformer paper he says was not taken seriously until GPT-3 in 2020)
AgentX (SemiAnalysis's agentic-coding benchmark built on real coding-agent traces, where about 96% of session tokens are cached)
LLM Token Expenditure Index (Silicon Data's blended price per million tokens, which the host cited below $1 this month against $60 five years ago)
RULER (The needle-in-a-haystack long-context test he uses to separate advertised context from usable context)
Gated DeltaNet (His current favorite attention mechanism, which he says cuts the time spent on the attention step by about three-quarters)
Listen to the full episode
🔴 YouTube | 🔗 Episode page
Watch the full episode:
More Thomas Sohmers interviews and podcast appearances
If this was worth your time, send it to someone closer to the industry than you are.
Get the latest market chatter as it happens: