TBPN Sep 21, 2026 29m 18m saved
With Ben Hylak, co-founder of Raindrop
Ben Hylak's observation about who worries about AI risk is geographic rather than philosophical: the people forecasting catastrophe work at model labs, and almost nobody building applications on top of those models does.
The usual reading of that split is that application builders are not paying attention. Hylak's is the reverse — that the risk they deal with every day is a different risk, arriving where people actually meet a model, and that the labs mostly do not think about it.
"Because something I've noticed is that there are no doomers at the application layer."
Hylak spent about four years at Apple before co-founding Raindrop, which sells agent reliability to Fortune 100 companies whose products reach millions of users, and which has just raised $35 million from CRV.
The full interview is covered here so you can skip it. 29 minutes of audio, 11 minutes of reading.
Here are the 8 insights that matter.
Key Takeaways
The AI risk conversation is "five or six different things all muddled up into one" — data-center power, bioweapons, sycophancy and mental health are not one subject
Raindrop now simulates changes before they ship, replaying real traces and generating adversarial ones
Binary classification of agent behavior does not work, because defining good and bad is the oldest unsolved problem there is
He calls it a human alignment company — the hard part is getting a company to state what it wants, not getting the model to comply
A regulated company's morality is its regulator's, not its model provider's, and the tobacco industry is his standing example
Fortune 100s are adopting very few Chinese models, and the competition he sees is on efficient token spend rather than raw capability
More enterprises are building their own agents than he expected six months ago, because internal process is the differentiator
Brand matters more, not less, precisely because taste has been commoditized
Raindrop raised $35 million from CRV, which he corrected on air after a host lowballed it
1. No Doomers Down Here
Asked at the top whether he is a doomer, an optimist or an accelerationist, Hylak answered with a pattern rather than a position.
The worry is concentrated where the models are made
Because something I've noticed is that there are no doomers at the application layer.
Ben Hylak
There is a reason for that, he allowed: the labs are where artificial general intelligence and superintelligence would emerge. But his objection is that the whole conversation has been collapsed into one word.
Several different problems are being discussed as though they were one
It's like five or six different things all muddled up into one.
Ben Hylak
The list he gave runs from data-center power and climate through bioweapons to model sycophancy and its effect on mental health. His own work sits at the application layer, which is the layer where people actually encounter a model.
That is where the risk he cares about lives, and where labs are not looking
And there is a risk there that, A, people at Frontier Labs, like, don't really care about or think about.
Ben Hylak
The hosts supplied the stake: Raindrop's customers are Fortune 100 companies whose products are used by millions of people and affect more.
2. What Raindrop Sells
Asked to reintroduce the product, Hylak kept it short.
The job is catching production failures and stopping them recurring
So we detect issues happening in production and we prevent those issues from happening.
Ben Hylak
The new capability, and the one he was most visibly pleased about, is simulation.
Changes get tested against the world before they meet it
So when people make changes to their agents, we simulate what those changes are going to do in the real world and can show people things they wouldn't have even ever expected to happen.
Ben Hylak
A host asked whether the obvious approach — have a model read every trace — is simply too expensive. Hylak said the cost is falling and that Raindrop has built very cheap classifiers for it. Then he said the cost was never the hard part.
3. Good and Bad Is Hard
Sorting agent behavior into two buckets is the unsolved problem
It turns out that like binary classification, like is this bad or not is really, really hard.
Ben Hylak
And it is not new
It's like humanity has tried, has been trying to define what is good and what is bad for like as long as we've existed.
Ben Hylak
What follows from that is the framing he uses for the company.
The alignment problem he sells into is a human one
So we consider ourselves in some ways like a human alignment company. Like how do you get a company to define what is good or bad?
Ben Hylak
The practical version is surfacing behavior nobody anticipated. Agents now do more than the people who built them imagined, he said, so a team looking at traces finds capabilities it did not know it had shipped, and has to decide whether it wants them.
4. Aligning the Company
His standing example of why one model's morality cannot be everyone's is the tobacco industry, where models routinely refuse to work.
A regulated company will deploy AI regardless
But, like, Altria is going to use AI somewhere.
Ben Hylak
What governs it, he said, is how that business is regulated in each market it sells into, and that will not look like the ethics of whoever trained the model. A host added that the way society processes the morality of cigarettes is through the courts.
Labs and deployers do not mean the same thing by alignment
And especially like, I think the way that Frontier Labs consider alignment is like often very different.
Ben Hylak
His own illustration was a marketplace negotiation where a model refused to write a mildly evasive message, and a host's was a model that could not be told it should prefer Nike inside Nike's headquarters — an off-the-shelf model treats Nike and Adidas as comparable brands, which is not what a Nike employee building a deck needs.
Saying what you want is the underlying difficulty
But yeah, I think that like, I think that telling something, whether that's a human or an agent, what you want is incredibly hard. I think management is incredibly hard is just in general.
Ben Hylak
Which is why companies will want both sides aligned
Yes, I think that companies will want both like humans and agents that are aligned to how their company works.
Ben Hylak
Every large company has a way of working that fits its industry, he said, including parts of it that are suboptimal, and an agent has to conform to that rather than to a general standard. He spent four years at Apple and used it as the case where the internal way of working is unmistakable.
5. Where Simulation Stops
A host pushed on the boundary: you can build a digital twin of a company's AWS instances, its Gmail and its Slack, and then what?
At some point the simulation has to include the internet
But at a certain point, then you also need to simulate the entire internet because that's part of the world. Like, how big can you go?
Ben Hylak
Raindrop's answer is to start from what actually happened.
Replay first, adversarial scenarios second
For now, where we're starting is all about, it's all about replaying historical things that have happened, getting really, really smart about how you choose those historical things.
Ben Hylak
The mechanism he described is reconstruction from tool calls. Each call an agent made is a glimpse of the world on the other side of the interface.
A tool call is a hole through the veil
Every time it called a tool, it's almost like a hole through this veil that lets you sort of see what the world behind it looks like, right?
Ben Hylak
From there you fill in around the holes, monitor the simulation while it runs, detect where the data was never filled in, and supply it at runtime. That, he said, is how you do it well.
6. Apple's Method Breaks
Asked why the most trusted consumer companies are not winning personal agents, Hylak defended Apple first and then explained the problem.
He will not count them out
I think that Apple is, first of all, I would literally never count them out because I think they're ridiculously well positioned.
Ben Hylak
What made Apple exceptional, in his account, is a loop: designers specify how something should work, engineers say it cannot work that way, designers insist, and the thing gets built. That loop produced the attention to detail the company is known for.
And it is the wrong loop for this technology
And the thing is that kind of way of working doesn't work as well for AI.
Ben Hylak
Frontier labs run it backwards
It is entirely flipped at the frontier labs. It's like the researchers went and did something and this is what it can do.
Ben Hylak
The hosts filled in the rest: the model turns out to be able to do mathematics, so go and tell people it can do mathematics, even though it still cannot do comedy and still draws six fingers. A host's contrast was Meta, where the willingness to ship something that misspells a few names out of a hundred is a structural advantage, against Apple, where a photo on your phone must sync to iCloud and nothing stochastic is tolerated.
7. Enterprises Build Their Own
On what it takes to sell into a Fortune 100 now, a host relayed an analyst's evolving position — that there is a duopoly and real price competition at the frontier between Anthropic and OpenAI, with less switching to open-source and Chinese models than expected.
Hylak's field report matches it
So, yes, definitely way less adoption of like, I would just say like Chinese models in general, especially at Fortune 100s.
Ben Hylak
There is a narrative risk attached to Chinese models, he said, even where it is not well specified. What buyers are actually optimizing is not raw capability.
The next leg is efficiency, not maximum tokens
It's more like let's go actually instead of token max, like efficient token spend will take us the next leg up.
Ben Hylak
Cost matters, and it compounds: a workload that is manageable at five thousand becomes something else at fifty or a hundred thousand.
The more interesting shift is what companies are choosing to build rather than buy.
Building internally is more defensible than he expected
I think that like there's more of a justification for companies to build their own kind of internal ways of working than I might have expected six months ago actually, which is really interesting.
Ben Hylak
The barrier to entry is low, he said, pointing at Muse as something heavily inspired by OpenClaw; what is scarce is distribution. But a company with legacy internal documentation and a defined path a ticket has to take now has a reason to build its own harness.
Even a law firm doing it no longer strikes him as odd
So, yeah, when I see like a law firm saying, like, okay, we're going to build our own models. We're going to build our own harnesses. I'm like, yeah, maybe that's not that crazy. Maybe that'll stick around.
Ben Hylak
A host's analogy was the dot-com era, when Amazon became the everything store and a great many brands still ran their own sites successfully for decades.
8. Brand Matters More Now
Asked whether brand design matters more or less than two years ago, Hylak said more, and a host immediately pushed back.
His evidence for the pushback is real: he named a company reaching a $10 billion valuation on a plain text website with no brand to speak of, only a presence on X.
But commoditized taste raises the value of standing out
But also, the second that things are commoditized, you need to stand out.
Ben Hylak
The old shortcut — a nicely rendered 3D asset on a landing page, which used to signal a competent team — is now a prompt. What is left is someone who understands the current moment well enough to make a choice, and the job of convincing a visitor that real people are behind the product.
Writing is where he sees it most clearly. AI writing is still bad, he said, then complicated his own claim: it might not have read as bad ten years ago, and what makes it grating now is saturation.
The annoyance is oversupply, not quality
But yeah, it wasn't annoying then and now it's annoying now because it's oversaturated.
Ben Hylak
A host's observation was that the tells are disappearing from model responses because the labs have trained them out, while the reaction to finding them in someone's blog post — that sending it was disrespectful — persists even when the prose is no worse than what the same reader consumes all day.
Bonus Insights
He labeled a blog post "vibe written" rather than pretend otherwise
Raindrop published a piece about the Hugging Face incident, built around a simulated game, that Hylak said he got about 80% finished before running out of time. Rather than risk being called out for a machine-written phrase, he stamped it. There was no pushback.
He reads about 20 minutes of model output a day
Hylak volunteered the figure as the baseline against which he judges writing quality, and said the responses from coding tools are the ones he finds genuinely obnoxious.
The round is $35 million, led by CRV
A host guessed $25 million. Hylak corrected him on air before the gong.
His Tesla Roadster prediction was the most conservative on the show
Given bounds of zero to 10,000 feet, he said it would be more than zero. The hosts' own guesses ran from half a foot to 10,000, and one raised the possibility that the airspace was cleared for a ramp jump rather than a hovering car.
Hylak's bottom line is that the hard part of deploying agents is not capability but specification — getting a company to say what good looks like, and then proving in simulation that a change will not break it.
Products, Companies & Tools Mentioned
Raindrop (Hylak's company. Agent reliability: detects production issues, and now simulates the effect of changes by replaying historical traces and generating adversarial ones)
CRV (Led the $35 million round)
Apple (Where he worked for about four years; his example of a design-led method he says does not transfer to AI)
Anthropic and OpenAI (The frontier duopoly a host put to him, with price competition rather than switching to open weights)
Meta and Muse (His example of a product heavily inspired by OpenClaw, and of a company willing to tolerate errors that Apple would not)
Altria (The regulated business he uses to show that a company's rules come from its regulators, not from a model provider)
Nike and Adidas (A host's example of the subtle alignment problem: an off-the-shelf model treats them as comparable brands)
Hugging Face (The security incident Raindrop wrote up as a simulated game)
Listen to the full episode
🔴 YouTube | 🔗 Episode page
Watch the full episode:
More Ben Hylak interviews and podcast appearances
If this was worth your time, send it to someone closer to the industry than you are.
Get the latest market chatter as it happens: