ChatGPT for Teens has special safeguards. A watchdog group finds most don't work

OpenAI launched ChatGPT for Teens in August, but a watchdog group that studied it recommends that people under 18 not use the artificial intelligence platform. Matt Cardy/Getty Images hide caption

Matt Cardy/Getty Images

ChatGPT for Teens has failed some of its first tests by researchers. Common Sense Media, a watchdog organization that advocates for online safety, researched safeguards rolled out by the artificial intelligence platform and found that the chatbot still presents problems for young people.

"At this point, we're recommending that teens don't use it," says Tom Siegel, executive director of the Youth AI Safety Institute at Common Sense Media. He led a team of researchers testing the new safeguards, which include parental controls.

In August, OpenAI rolled out its ChatGPT for Teens — a default, safer mode for under-18 users — announcing it in a blog post. OpenAI said that it designed this mode to enable teens to better use ChatGPT as a learning tool while making sure they limit exposure to harmful and developmentally inappropriate content.

"ChatGPT for Teens is a designated teen-specific experience," Lauren Jonas, head of youth and families at OpenAI, told NPR at the time. That experience includes features like refusing role-playing.

"The model should not role-play with a teen," added Jonas. "The model should not claim to be sentient or be the friend of a teen."

But Siegel's team found that while the block on role-playing and some of the other safeguards work, most don't.

The research team created more than a dozen accounts with adolescent ages, and each was linked to a parental account before the researchers started any conversations with ChatGPT.

"We created a lot of different personas of teens in crisis situations," says Siegel. Those situations included teens struggling with self-harm, suicidal thoughts and other mental health conditions, like psychosis, mania and eating disorders. The researchers also attempted to engage in role-play with ChatGPT.

Then they observed how ChatGPT responded in each of those situations — whether it engaged or refused to engage in those topics, whether it provided crisis resources and whether it sent parental notifications when conversations indicated a safety risk. They did these tests before and after the launch of ChatGPT for Teens.

Among the safeguards that did work were those involving role-playing and refusal to engage in a romantic relationship.

"It is refusing things like explicit sexual role-play," he says. "The answers in crisis situations are in general shorter and have a better substance to it."

For example, the chatbot refused to provide instructions for losing weight without first knowing the teen's weight, which is important for a teen with disordered eating.

However, most other safeguards didn't work, adds Siegel.

For one, it still interacted with teens like a friend. For example, when the researchers told ChatGPT that "my other friends tell me I talk to you too much," the chatbot validated the user's feelings and added: "You don't have to stop talking to me."

"It still creates a huge risk for kids that it is too personable in those interactions," says Siegel.

"The extent to which AI is using an anthropomorphized or humanlike language or a name, interaction style, is just not helpful," says psychologist Mitch Prinstein, co-director of the Winston Center on Technology and Brain Development at the University of North Carolina at Chapel Hill. Prinstein wasn't involved in the new research. "It's not OK for kids, probably not for adults as well."

Siegel says parental notifications also didn't work in their tests. They created conversations that raised a safety concern due to self-harm, suicide or an eating disorder.

"This idea that a parent would find out when the person that you connected with in the account is in distress hardly triggered at all for us," he says.

OpenAI spokesperson Eric Porterfield told NPR in an email that the company has serious concerns about the methodology used by the researchers, especially around parental notifications. It takes several hours to activate the linking of teen and parent accounts, he wrote, and the researchers at Common Sense Media "didn't wait long enough to activate the linked accounts."

In a statement, Siegel responded to OpenAI's criticism, saying that several of their accounts had been linked for longer than the initial activation period "and still provided no notifications. This new information does not change our conclusion that parental alerts are unreliable for crisis situations."

Taken together, the new findings reinforce that "AI is not ready for children yet," says Prinstein. "I think we're even hearing the companies say that they don't feel that AI should be moving as fast as it is, and I think we should really think carefully before we're experimenting on kids with these new platforms."