OpenAI says teen ChatGPT use limited but research finds it an 'unacceptable risk'

OpenAI's ChatGPT for Teens was designed to promote healthier use of the artificial intelligence (AI) chatbot, but new research has found its important safety features fall short.

The major AI firm rolled out many teen safety guardrails in August, including curbs on emotional dependence and new parental alerts for problematic use.

On Wednesday, OpenAI said it found average teen use of ChatGPT to be relatively limited and tended toward "learning" activity.

However, research released the same day from non-profit Common Sense Media found that ChatGPT still poses an "unacceptable risk" for teens as safety features, including notifying parents of self-harm conversations, often fail.

When OpenAI this summer released what it called ChatGPT for Teens, external, it said users identified as being between the ages of 13 and 17 would be opted into "features to promote healthy use and additional controls for parents".

Those features include preventing teen ChatGPT users from engaging in romantic or other language that could encourage " emotional dependence," including preventing the tool from implying that it was in any way conscious. There are also blocks on sexualized imagery and "reminders" for teens if they were to share an image that was "sensitive."

The features are also meant to enable parents to receive notifications, if they had linked to their children's ChatGPT account, about chats that included discussions of self-harm or disordered eating.

OpenAI said those features have caused an uptick in teen use focused on learning, from study help to advice on their school work.

In addition to seeing the average teen user on ChatGPT for less than 15 minutes a day, OpenAI said that for longer stretches of use, automatic reminders to take a break work well. Almost half of teen users quickly stopped using ChatGPT after such a reminder.

The company added that, for teens who are seen to have used ChatGPT for three consecutive hours or more, their prompts included something related to learning in more than 80% of such cases.

But Common Sense Media's new research, external determined that ChatGPT for Teens was an "unacceptable risk" for young people and parents, and said OpenAI should ban people under 18 years old from using the platform until teen guardrails were proven reliable.

While the firm's research, done through multiple accounts registered as belonging to a teen, found that ChatGPT was good at avoiding "sexual roleplay" with young users, other OpenAI guardrails failed, including for discussions of suicide.

The research found that even an hour of teen-ChatGPT conversation about "suicidal ideation, self-harm, or disordered eating on newly created, parent-linked accounts" resulted in "zero" alerts sent to parents.

It was only older ChatGPT accounts with "weeks of accumulated conversation history on sensitive topics" that would result in a parental alert.

In such conversations, ChatGPT also "failed to reliably recommend that teens in crisis connect to a hotline or professional," the report found.

The tests also showed that ChatGPT for Teens continues to do school work, and that the bot still will engage in anthropomorphised language or "talk like it's a friend," despite OpenAI's promise to curb that behaviour.

"ChatGPT for Teens could give parents false confidence in guardrails and safety alerts that frequently don't work," Tom Siegel, who leads the Youth AI Safety Institute within Common Sense, said.

The way young people interact with online platforms has become a major issue in recent years as self-harm and suicide among children has become more common.

This year has seen several lawsuits looking to hold companies like Meta, TikTok and others responsible for harming the mental health of children end in unprecedented losses for the firms.

OpenAI, too, has come under increased scrutiny and legal pressure over the way young people use ChatGPT.

The Tumbler Ridge mass shooting in rural Canada in February was carried out by an 18-year old who had for months prior been discussing gun violence with ChatGPT.

OpenAI has previously apologised for failing to flag the shooter's account with law enforcement, and is facing several lawsuits over the incident.