Experts warn about risks of AI-generated content in child sexual abuse material

WARNING: This story contains disturbing descriptions of child sex abuse. Resources and supports for anyone who has experienced sexual violence can be found at the bottom of this story.

Prince Edward Island RCMP say there has been a "notable increase" in child sex abuse material made using artificial intelligence across parts of the province. Online exploitation specialists say this surge in illegal AI-generated content could be growing the demand for real-world child abuse. 

Internet watchdog reports indicate a trending global rise in AI-generated child sex abuse content, previously known as child pornography.  

In an email, P.E.I. RCMP senior communications adviser Scott Ferris said it’s a "safe estimate" that one third of Island RCMP online child abuse investigations now involve AI-generated material.

That estimate considers only areas under RCMP jurisdiction, which excludes Charlottetown, Summerside and Kensington. 

LISTEN | How criminals are using AI to abuse children:

This is PEI23:45How criminals are using AI to abuse children

AI child exploitation cases are surfacing across the Atlantic region. 

In St. John’s, N.L., this summer, a 17-year-old boy was charged with nearly 60 counts related to the creation and distribution of child sexual abuse material. Police allege he used AI tools to alter images of 19 minors and make them explicit. 

Const. Stephanie Myers with the Royal Newfoundland Constabulary told CBC News On the Go host Krissy Holmes the case is "probably one of our first if not the first" specifically involving AI being used to create child sexual exploitation material.

A 17-year-old in St John's, N.L., faces 58 charges related to his alleged use of AI to alter images of teenagers. Const. Stephanie Myers with the Royal Newfoundland Constabulary (RNC) says this is likely one of the RNC's first AI-specific child abuse cases. (Danny Arsenault/CBC)

What is AI-generated child sexual abuse material?

Technological developments have dramatically changed how child sex abuse crimes are committed. With AI, criminals no longer need to make direct contact with an underage victim to create new abusive content.

But that doesn’t mean these are victimless crimes.

• AI-generated material found in one-third of P.E.I. RCMP online child abuse cases

• Police say 19 victims identified in AI child sexual abuse image investigation

"In many of the cases … AI-generated content is either produced to create material of a child that’s in the family or the neighbourhood of that offender," said Mauranne Ste-Marie, senior research analyst with the national RCMP’s sensitive and specialized investigative services unit.

"It increases the demand for the material, it normalizes the abuse."

Certain generative AI models — which are ultimately trained on images of real people — can create abusive images or videos either in the likeness of a specific person or by generating a non-existent, synthetic child from scratch.  

Some offenders also use AI-powered "nudify" tools to create explicit "deepfakes" — realistic fake images or videos of a victim. 

WATCH | 'Nudify' apps and AI-generated child sex abuse material are on the rise. Here’s what you can do about it:

'Nudify' apps and AI-generated child sex abuse material are on the rise. Here’s what you can do about it

August 17|

Duration

2:48

"Anyone can take pictures that are posted online ... take the face of your children and create very graphic sexual abuse material with it," Ste-Marie told CBC News.

“There are virtually no limits to what can be produced.... If their sexual preferences are for young girls with blond hair, blue eyes, they don’t need to spend time searching for that preference online anymore. They can simply create it."

Mainstream generative AI models like ChatGPT, Gemini and Claude have strict safeguards in place that are designed to block the generation of any child abuse material. Confirmed violations are supposedly then reported to law enforcement. 

To circumvent these controls, offenders often rely on freely available open-source software to generate abusive media offline, where is it far harder to detect.

Since these models are typically used without an internet connection, any exploitative content they produce can be incredibly difficult for police to track. 

Ste-Marie warned using generative AI models this way could lead to more real-world abuse. 

"It is a stepping stone, it is one step closer to sexually abusing a child," she said.

"After viewing material for a certain number of weeks or months, it may not meet their needs anymore."

Sextortion

In an email, Summerside police reported at least one instance where a suspect used AI-generated child sex abuse material in a "sextortion" attempt this year. 

Sextortion is a form of online blackmail where offenders coerce victims into sending money or explicit media by threatening to release sensitive images or videos. 

In April 2023, 17-year-old Harry Burke from Fortune Bridge, P.E.I., tragically took his own life hours after telling his parents he had been victimized by a sextortion scam. 

Carl Burke and Barbie Lavers hold up a photo of their son, Harry Burke. He took his own life after being victimized by a sextortion attempt. (Laura Meader/CBC)

Jacques Marcoux, director of research for the Canadian Centre for Child Protection, said AI-generated or manipulated material is increasingly used in sextortion attempts. His organization operates Cybertip.ca, Canada’s national tip line for reporting online child abuse.

"In lieu of being able to trick the child into giving them an intimate image, they’ll simply create one using AI," Marcoux said, noting that offenders then threaten to release images online, claiming that they're real.

P.E.I. RCMP media relations officer Gavin Moore said there were 26 reported instances of sextortion last year in Island areas under RCMP jurisdiction.

Moore didn’t specifically reference AI-generated material in any of these cases, but he did note reporting rates are typically "very low" for this type of crime. 

"This is a type of crime that victims are extremely embarrassed [about]," Moore told CBC News.

"[It] can have very devastating and significant effects on its victims."

As AI tools become more sophisticated, it’s getting harder to tell what’s real and what’s not, said Marcoux.

Jacques Marcoux, director of research and analytics at the Canadian Centre for Child Protection, says its getting more and more difficult to tell what's AI and what's not. (Jeff Stapleton/CBC)

"Unlike three years ago … you can’t discern whether or not it’s real in many cases. It’s almost a coin flip," Marcoux said. 

But, from a legal standpoint, it doesn’t matter.

Under the criminal code, AI-generated child abuse is treated the same as authentic abuse material. Altering or generating explicit media with AI through written prompts can constitute making child sexual abuse material.

Police in St. John’s, N.L., applied that standard this summer.

The 17-year-old boy, who was hit with 58 criminal charges, faces 19 counts of making child sexual exploitation material after allegedly using AI to digitally alter photos of teenage girls.

What can you do?

Since offenders can now use AI tools to create abusive material based on any image, Ste-Marie warns there is always a risk involved when posting children online. 

"Every time you post a picture or a video of yourself or your children, there is a risk that it could be used by someone with bad intentions," she said, noting that one recommendation is to take precautions like covering a child's face in photos.

• Edmonton teens charged with using AI to create sexually exploitative images of classmates

• Video

  2 charged over violent, sexual AI deepfakes of dozens of Canadian women

"The more images and information that are publicly available about a person, the easier it is for that model to create very convincing fake or manipulated content," Ste-Marie said.

"So, as soon as you reduce that availability, it makes it more difficult to create that type of content."

Although she said people can reduce the risk of being victimized, she stressed that online child abuse is never the victim’s fault. 

"Your child is never responsible for the creation of that type of content, even if they posted images online and they feel like it was their fault, it never is," Ste-Marie said. 

"They have the right to post and to share without thinking that someone will use it against them."

If you or someone you know has experienced sexual violence here’s where to get help:

• Victim Services P.E.I.: Queens and Kings County: 902-368-4582; Prince County: 902-888-8218.
• Rise P.E.I.: Phone or text: 902-218-6143.
• P.E.I. Rape & Sexual Assault Centre: 902-368-8055 or 1-888-368-8055.
• Report Child Abuse: daytime 1-877-341-3101; after hours 1-800-341-6868.
• You can check out a list of support services, programs, and other information on the 211 website or call 211 to be directed to the help you need.
• Kids Help Phone (national): 1-800-668-6868.
• Here is a list of sexual assault centres, crisis lines and support services elsewhere in Canada.