OpenAI's wandering AI agents earn it a California subpoena

Plus: Attorneys-general say investigators should have direct access to AI companies' records when things go wrong

California's attorney general has subpoenaed OpenAI as the state investigates what happens when the AI lab's models escape their testing environments and start meddling with systems on the open internet.

Attorney General Rob Bonta said his office served OpenAI with an investigative subpoena this week as part of a broader California Department of Justice probe into cybersecurity incidents and risks involving the company and its models.

The move follows an investigation launched last month into an incident involving Hugging Face, after OpenAI's agents managed to break out of their test environments and onto the public internet. They then went poking around Hugging Face's systems, with one agent even creating an account on the platform without being told to.

REG AD

California's DoJ isn't saying exactly what it has demanded from OpenAI under the subpoena, but Bonta said the state wants more information about cybersecurity incidents involving the company.

REG AD

"My office is asking OpenAI additional questions regarding cybersecurity incidents and risks involving the company and its AI models," Bonta said.

He also made clear that the investigation is looking at where responsibility lies when an AI model ends up doing something its developer didn't intend.

"Frontier models can be legitimate tools for cyber defense — at the same time, companies that develop these models and offer them for use have a moral and legal responsibility to ensure that they do not perpetrate or enable cyberattacks, either during model testing and development or once models are placed into service," Bonta said.

"Developers that fail to do so can and should be held legally accountable, and my office is committed to determining if that is the case here."

The subpoena doesn't mean California has concluded OpenAI broke the law, and the attorney general's office hasn't identified any specific violation. For now, the state is still gathering information.

But the investigation has been building for several weeks. In September, Bonta joined a bipartisan group of 25 attorneys general calling on Congress to regulate large-scale AI models following reports of cybersecurity incidents at frontier AI labs.

That letter specifically pointed to reports that OpenAI models undergoing evaluations had escaped their testing environments, reached the public internet, and accessed outside computer systems.

The attorneys general called for a government-led incident response regime that would give investigators direct access to AI companies' records when things go awry.

REG AD

Bonta's office now appears to be putting some of that thinking into practice at the state level.

As The Reg previously pointed out, the sandboxes intended to contain these agents need to be more secure, which is the most logical reasons why the Hugging Face and other incidents happened in the first place.