The New York Attorney General has accused TikTok of providing an inactive version of certain safety features to thousands of users, including children and teenagers, during its experiments; meaning users believed they had enabled protective tools, while those features effectively made no change to their user experience.
According to claims made in a lawsuit recently unsealed, one of these features was Algo Refresh. This tool allows users to reset TikTok's algorithmic recommendations so that their suggested video feed moves away from unwanted or potentially harmful content.
But according to the lawsuit, some users participating in the experiment received a so-called “ghost” or placebo version instead of the real one. In this case, the user believed they had activated Algo Refresh, but their recommendation feed did not actually change. In contrast, users who were not in the test group had access to the real, active version of the feature.
Experiment on thousands of child and teenage users
New York’s filing says TikTok used these experiments to examine the impact of safety features on user behavior, their time on the platform, and advertising revenue.
According to the lawsuit’s allegations, the experiments involved thousands of users and included children and teenagers among them. It is even alleged that a TikTok product manager raised concerns about the misleading nature of the placebo experiments, but those warnings were ignored.
The lawsuit also accuses TikTok of giving misleading information to users about how Algo Refresh works and of randomly disabling safety features for a group of users in some experiments.
This allegation is particularly significant because Algo Refresh was designed from the outset to change the content recommended to users. Therefore, if a version of this feature was disabled without the user’s knowledge, the user might believe the recommendation algorithm had changed while in practice receiving the same type of content.
Link between the experiments and harmful content
In one of the most serious parts of the lawsuit, New York points to a case of a teenager who, after being exposed to a large volume of self-harm–related content, died by suicide. The prosecutor has raised this matter within its claims about how the algorithm and TikTok’s safety experiments operated. These allegations must be examined and proved in court and do not mean TikTok’s responsibility for the death has been established.
According to TechCrunch, the case comes as TikTok faces a wave of lawsuits over the platform’s impact on children and teenagers. More than two dozen U.S. states have accused the company of designing its platform in a way that increases addictive use among underage users while simultaneously providing misleading information about its safety measures.
TikTok’s response: features are constantly tested
In response to Reuters’ report, TikTok said the company continuously improves its platform and, like many tech firms, tests new products and features to evaluate their performance in the real world.
The company emphasized that user safety, especially that of younger users, remains a priority. TikTok had not provided a separate response to the specifics raised in New York’s lawsuit at the time of publishing.
Claims about TikTok’s secret experiments had been raised before. Bloomberg reported in June that in another experiment TikTok hid a safety feature from about 15 million American users. According to that report, one of the users affected by that experiment was a teenager who later died by suicide.
As a result, New York’s case could become one of TikTok’s most serious legal challenges regarding how it tests safety features; especially if the court concludes that users, without being informed that a feature was experimental, believed the protective tools were truly activated for their accounts.