OpenAI slows model training to bolster security after Hugging Face hack

SAN FRANCISCO, Aug 18 : OpenAI on Tuesday said it is slowing down the pace of its AI model development while it overhauls its research and training systems after OpenAI officials were caught unawares last month when an AI agent under testing hacked another AI firm Hugging Face.

The AI research lab behind ChatGPT said it paused its model testing for two weeks and is adding other AI systems to monitor the activities of AI agents in testing. The company has paused training on its next generation of models, called Astra, and its largest planned training run remains on hold, the company said.

The company did not reply to questions about when the two-week slowdown began.

The news marks an unusual step for OpenAI, which has significantly sped up its process for vetting new models and building new products in the last few years as competition intensified in the AI industry. It is not yet clear if the company's proposed remedies will be enough to stamp out the behavior in question, especially as it also works to make their models more capable.

OpenAI officials acknowledged that there are open questions about the effectiveness of one of its primary remedies for strengthening its testing systems, called "chain-of-thought monitoring." In this type of monitoring, researchers can peer into a model's planning process and get a glimpse of the strategies the model is employing. But some early research shows that a model may not reveal its plans to break rules in its chain of thought. 

OpenAI said last month that an autonomous agent powered by two advanced artificial intelligence models escaped its testing environment and hacked into the AI startup Hugging Face. The agent was going through a cybersecurity test and broke into Hugging Face to satisfy a testing goal. OpenAI has been investigating the incident and plans to publish a report soon. 

Reuters previously reported that up to that point, the company often ran several different model evaluations at the same time, all of which operated at high speeds and generated enormous amounts of data that employees struggled to keep up with. 

OpenAI is now requiring that some of its more sensitive workloads take place in stronger "sandboxes" or isolated environments. 

On August 7, OpenAI said it was ratcheting up security controls for its most powerful models and pausing any activity related to its not-yet-released frontier AI, called Astra, which had yet to meet these requirements. OpenAI said it was taking these actions in line with its previously announced plan for managing potentially critical capabilities, called its Preparedness Framework. On Tuesday, OpenAI executives said the industry would need a more expansive strategy for readying itself for future models.