TikTok rolled out a safeguard against harmful content loops to 90% of its US users in early 2022 and deliberately withheld it from the other 10%, which was 15 million people.
Companies test features on subsets of users constantly. What a sealed internal document showed, and what Bloomberg reported this month, is that the users held back were the control group for a measurement of engagement, and that the document says so in the company's own words.
"One, the company said TikTok's filter bubble prevention strategies did not take effect on this user by design."
Olivia Carville is a Bloomberg investigative reporter who has covered children and the digital world for years, wrote the 2023 Businessweek story that prompted TikTok's own internal review, and obtained the document that review produced.
I listened to the full episode so you can skip it. 23 minutes of audio, 12 minutes of reading.
Here are the 13 takeaways that matter.
👤 Guest: Olivia Carville, Bloomberg investigative reporter, who has covered social media and children's mental health for years and broke both the 2023 Businessweek story and the reporting on TikTok's sealed internal review
🎙️ Host: Sarah Holder, who hosts The Big Take, Bloomberg News's daily podcast
📰 Published: 31 August 2026
🟣 Apple Podcasts | 🔗 Episode page | ⏱️ 23 min | ✅ Time saved: 11 min
Key Takeaways
TikTok gave a harmful-content safeguard to 90% of US users and withheld it from 15 million as a control group
The company's own document says the balance being struck was between safety and the ability to measure daily active users
The internal review that found this was triggered by a reporter's request for comment
Staff from the algorithm and trust-and-safety teams went through more than 7,500 videos one user had watched
Meta's settlement is $12 billion unconditional and $5 billion contingent on rivals adopting the same rules
Meta's condition is designed to stop teenagers simply moving to a platform with no curfew
The financial penalty is small against Meta's revenue, and the product changes are the real remedy
A midnight-to-6am lockout for children, a two-hour daily scrolling cap and expanded parental controls
The safeguards platforms announce are often used by fewer than 1% of the people they are announced for
This is now mass-tort litigation on the scale of tobacco and opioids
4,000 personal injury suits in four years, more than 1,300 school districts and dozens of state attorneys general
Two senators have given TikTok 13 questions and a 1 September deadline
After the review, TikTok cut the control group from 10% of US users to 1% and the exposure window to seven days
Federal legislation is deadlocked between a Senate bill and a House bill, which is why the accountability has come through the courts
1. A Feed That Turned
The episode opens with Chase Nazca, a 16-year-old from Long Island who died by suicide in February 2022. He was an honors student and a football player who used TikTok for weightlifting videos, basketball, motivational clips and music.
What he searched for and what he was served came apart. Through the winter of 2022 his feed filled with videos about depression, hopelessness, loneliness, self-harm and suicide
His parents filed a wrongful death suit against TikTok the following year, and Carville wrote about him for Businessweek in 2023
Asked what the families have been saying, she gave their own framing. "They've been telling me they're screaming into the wind. No one's listening."
They had been asking Congress to legislate. "And we finally saw change through the justice system, which, you know, is remarkable."
"This litigation has been going on for half a decade." "And August 26th was a huge moment for so many people."
2. The Filter Bubble Problem
TikTok's recommendation engine learns what a user watches and serves more of it. In the summer of 2021, Carville said, the Wall Street Journal and other outlets began investigating it because it was so mysterious.
What the reporting surfaced was a pattern the industry now calls a filter bubble. "Some users were getting caught in something called a filter bubble, essentially an echo chamber of content that could be as benign as cat videos or much more harmful."
TikTok responded publicly. By the end of 2021 it announced it would start testing ways to break up repetitive streams of videos about harmful content such as loneliness or extreme dieting
That announcement is the safeguard the rest of the episode is about
3. 15M Never Got the Fix
The document Carville obtained was produced by TikTok in 2023, turned over to lawyers in litigation, and placed under a court-ordered seal. It shows the safeguard did not go to everyone.
The split was deliberate and the smaller half was large. In early 2022 the company rolled the safeguard out to 90% of US users. "But they held back that control group. 15 million people did not get the safeguard."
Chase Nazca was one of them. His account entered a filter bubble of harmful content on 25 January 2022 and was fed thousands of videos in that vein until his death three weeks later
Carville reported that in the days before he died he had viewed content depicting the method he used. She said so plainly and without detail
4. "By Design" Is the Quote
Asked what TikTok said in the document about why it ran the experiment, Carville pointed at two lines.
The first says the omission was intentional. "One, the company said TikTok's filter bubble prevention strategies did not take effect on this user by design."
The second says what was being weighed. "And they explained that the user impact decision was a delicate balance across safety and the ability to measure impact on DAU and core metrics."
DAU is daily active users
The purpose, in her account, was to measure the cost of safety. The company wanted to know how engagement differed between the 90% who had the safer algorithm and the 10% who did not
"10% of TikTok's user base was 15 million people."
5. The A-B Test on One Boy
Carville's request for comment in 2023 did not only produce a public statement. It set off an internal review, which the company titled "Bloomberg Businessweek Youth Safety and Filter Bubbles Escalation Summary".
The review was granular. People from the algorithm team and the trust and safety team went through Chase's watch history in the weeks, days and hours before he died. "They review more than 7,500 videos that Chase had watched."
What it found was a routine industry procedure with an outcome that was not routine. The account had been placed in what the company calls an A-B experiment — a back test, commonplace in the industry — and he was randomly selected in January 2022
"Within a month of Chase Nazca being placed in that AB experiment, he was dead."
6. TikTok's Answer, Twice
The 2023 public statement and the 2026 one are close to identical, and Carville said the latest response echoed the earlier one almost word for word: sympathy for the family, a commitment to child safety, robust detection systems, work to make the platform safer
Her point is about the gap between the two. "The story is what they didn't say." Nothing about the filter bubble test was disclosed publicly at the time
She was careful to give the company the other side of it. "And don't forget that, you know, TikTok had more than 40,000 people working on its trust and safety team"
"They've invested more than a billion dollars in trust and safety over the years, and they continue to do so."
Her explanation is competitive rather than malicious: the platforms are built around engagement, profit and winning the teenage market, because holding the teen market is what makes a product cool enough to keep
7. Safeguards Under 1% Use
Asked whether anything has actually changed across the years she has covered this, Carville described a shift in the framing and very little in the outcomes.
The conversation moved from a question to a diagnosis — from whether social media could harm young people's mental health to a full-blown public health crisis requiring the companies to be blocked, changed or regulated
The guardrails arrived and did not work. Litigation documents, she said, show a lot of the safeguards being adopted are not very effective
The tell is the usage number. A platform announces a feature that will protect teenagers from something. "And then you ask the question of how many people are using it and it's less than 1%."
"What we've seen from the industry is repeated claims of making the product safer when the reality is it hasn't actually been protecting young people."
8. Meta Pays $12B, Not $17B
On 26 August, Meta settled claims from US states alleging it used deceptive advertising and designed its platforms to get children hooked. The host laid out the terms: up to $17.1 billion, plus product changes in the US.
The changes are specific. Blocking children from Instagram and Facebook between midnight and 6 a.m., a two-hour daily cap on teenage scrolling on both platforms, and expanded parental controls
The structure of the money is the part that has been missed. "Meta is paying $12 billion. It is paying an additional $5 billion only if those companies follow suit and introduce the same framework."
The settlement includes a call-out to the rest of the industry, naming YouTube and TikTok directly, to adopt the same product changes
The reason is competitive rather than altruistic. Meta is protecting its own market share: if it locks teenagers out overnight and nobody else does, they move to Snapchat
"So let's make every social media platform play in the same rules, play the same game."
9. The Product Changes Matter
The host pressed on whether a penalty of this size actually changes behavior. Carville's answer separates the money from the remedy.
"$17 billion sounds like a massive number, and it is — don't get me wrong — but Meta is one of the wealthiest companies in the world."
"The last quarter, its revenue was $60 billion."
"So I think a lot of people have pointed out that really the financial penalty probably isn't enough to make the company do anything different."
The accountability, in her reading, sits in the product. "But really, the change lies in those product updates. That's the significant thing here. That's where the accountability is."
She expects the rest of the industry to move. Other platforms will roll out additional teen safeguards, and Meta is now shipping changes families have been asking for for years
10. 4,000 Suits and Counting
The host put the scale of the underlying litigation on the record: 4,000 personal injury lawsuits filed against social media companies over the past four years.
Three separate categories of plaintiff are running at once. Individuals and bereaved families; "There are more than 1,300 school districts across America that have filed lawsuits against these companies alleging mental health harm and that have drained school resources"; and dozens of state attorneys general
The comparison she used is the one that should interest anyone holding these stocks. "So this is a massive, consolidated litigation, nationwide litigation, that has been compared to big tobacco." She put the opioid litigation alongside it as the other reference point
"We still have thousands of other lawsuits to go. The story is not over. Really, it's just begun."
11. Senators Want 13 Answers
After the reporting on the sealed document, Senators Marsha Blackburn and Richard Blumenthal wrote to TikTok's chief executive Shou Chew and to the head of its US spinoff.
The two are co-authors of the Kids Online Safety Act, which passed the Senate in 2024 and died in the House the same year
The letter is a list of 13 questions, and the substance of them is who knew: how many other children were experimented on, how many users were affected, what other algorithmic experiments have been run, the names and titles of the employees involved, whether leadership knew and when
They also asked for the unredacted document, and gave the company a deadline of 1 September to respond
12. The Control Group Shrank
The same confidential document contained TikTok's own recommendations to prevent future incidents, and Carville said two of them are measurable.
The control group was cut by a factor of ten. From 10% of American users to 1%
"1% is still 1.5 million people, but it's a hell of a lot better than 15 million."
The exposure window was cut far harder. Users in the control group had been left in filter bubbles for between half a year and a year; that came down to seven days
She called that change significant
13. Two Bills, One Deadlock
Asked about federal enforcement, Carville said the legislative route is blocked and has been for years.
"In order to get federal laws, we need legislators to work together. I mean, we're just in a deadlock."
There are two child safety bills and no agreement on which reaches the president's desk. The Senate is championing the Kids Online Safety Act; the House has its own Kids Internet Digital Safety Act
That deadlock is why the accountability arrived through the courts instead, which is what advocates and families turned to
One enforcement mechanism did come out of the settlement. An independent auditor will have to verify that platforms have actually rolled out the safeguards they say they have
Whether TikTok, YouTube, Snapchat and the others join Meta is still open
Bonus Insights
The lawsuit that started the reporting has already been dismissed once. A judge threw out the Nazca family's wrongful death claim against TikTok last year after the company argued it was protected from liability under Section 230. An appeal is pending
Carville's closing argument is the one an investor should weigh against the settlement number. "We have run a giant experiment, a psychological experiment, on an entire generation of children for a for-profit industry."
"And I don't use that word experiment lightly. That's how TikTok described what it did."
"You know, you give your kids this device and you give them access to everything, the entire world, but you give the world access to them as well."
The episode carries a content warning at the top and points listeners to a global list of suicide and self-harm helplines at the end
Carville's bottom line is that the money in the Meta settlement will not change how these companies behave, but the product changes and the independent auditor might — and that with thousands of suits still to run and a sealed-document trail now being pulled into the open, the legal risk sitting under the industry is closer to the start of its story than the end.
Products, Companies & Tools Mentioned
TikTok (Withheld its filter-bubble safeguard from 15 million US users as an engagement control group, per its own sealed internal review)
Meta (Settled state claims for up to $17.1B — $12B unconditional — plus overnight lockouts, a two-hour teen scrolling cap and parental controls on Facebook and Instagram)
Bloomberg Businessweek (Published Carville's 2023 story on Chase Nazca, the request for comment that set off TikTok's internal review)
Snapchat (The platform Meta says teenagers would move to if it were the only one imposing an overnight lockout)
YouTube (Named in the settlement, alongside TikTok, as a platform Meta is asking to adopt the same framework)
Books & Resources Mentioned
Suicide and self-harm helplines (The global list the episode directs listeners to)
Kids Online Safety Act (The Senate bill co-authored by Blackburn and Blumenthal, passed by the Senate in 2024 and dead in the House that year)
Kids Internet Digital Safety Act (The House's competing child safety bill, and the reason neither has reached the president)
If this was worth your time, send it to someone closer to the industry than you are.
Get the latest market chatter as it happens: