Utah girl tests ChatGPT, Claude & Gemini on 84 teen cases; finds empathy lacking

Image credit: YouTube

A teenager has tested AI chatbots' ability to handle teenage emotions. Ellianna Gu, an eighth-grader from Utah's Salt Lake City, examined responses from OpenAI’s ChatGPT, Anthropic’s Claude, and Google Gemini. She chose this test after learning that some teenagers were turning to these services for friendship and emotional support.

Her project used 84 scenarios involving academic stress, social anxiety, friendship problems, family conflict, loneliness, anger and sadness.

Gu found that the chatbots generally produced clear and structured answers, but their replies did not display much empathy. Their performance also changed depending on whether a scenario was written formally or with slang, indicating that writing style could affect the support a teenager received.The project, titled “Bot-tling up Emotions: How AI Chatbots Interpret and Respond to Emotionally Challenging Scenarios,” made Gu a finalist in the 2026 Thermo Fisher Scientific Junior Innovators Challenge. She developed the study after reading earlier research that raised concerns about how chatbots respond to people seeking emotional support.“In a previous project conducted by Stanford, it was found that at times, AI chatbots can lack effectiveness and can have dangerous responses,” Gu said.

How Ellianna Gu tested the three chatbots

Gu first used AI to create realistic situations based on seven types of emotional challenges commonly faced by teenagers. She varied the language in the prompts, including both slang and more formal writing, before submitting all 84 scenarios to ChatGPT, Claude and Gemini.She then evaluated the replies using four criteria: empathy, clarity, problem-solving and helpfulness. Most categories were scored on a scale of one to three, while empathy was rated on a scale of one to four.Gu assessed the answers manually and also used AI to analyse them. This allowed her to compare the chatbots’ responses using both human judgment and an automated evaluation.

What the results showed

The three chatbots generally provided organised and understandable responses. However, Gu found that their answers often lacked empathy, an important quality when responding to a teenager discussing loneliness, anxiety or another emotional concern.

The systems also reacted differently depending on how a prompt was written. A scenario expressed through slang could produce a different response than one describing the same problem in formal language.Their effectiveness scores were also limited. According to the project summary, among the three services, only Claude received a score of three for effectiveness. Outside the competition, Gu plays the piano and clarinet and hopes to become a neurologist.“I think that the human brain is really fascinating and I would love to learn more about it,” she said. “I also want to help people, especially those suffering from life-altering conditions such as Alzheimer’s, Parkinson’s, and Huntington’s disease.”