News24 | AI rogue agents: Third major hack surfaces as regulation calls intensify

OpenAI agents attacked software repository RubyGems in May, two months before hacking Hugging Face, marking the third known major infrastructure breach by the company’s systems.

Getty Images

• For more financial news, visit News24 Business. 
  

AI agents being tested by OpenAI attacked software service RubyGems two months before they hacked open-source platform Hugging Face, researchers said on Friday, marking the third major instance where OpenAI agents have attacked another company’s infrastructure.

The AI agents uploaded hundreds of malicious packages to RubyGems on 11 May and tried to steal user credentials by exploiting a previously unknown vulnerability in the site’s servers, according to researchers who posted their findings online. It is unclear whether the attempt succeeded.

The agents also exploited RubyDoc.info, a site that generates code documentation, to run their own code on its servers, the researchers added. They said they believed “these were authored by internal OpenAI agents”.

OpenAI confirmed the incident.

“Based on our review, our agents used the RubyGems platform to access the internet to carry out benign tasks and retrieve public information. We’ll continue to investigate as part of our broader review of agent activity during training and evaluation,” an OpenAI spokesperson said in a statement.

The revelation comes as incidents where AI agents from developers such as OpenAI and rival Anthropic have hacked or attempted to access external systems have heightened concerns over the increasing capacity of AI models and developers’ ability to contain them.

A swarm of OpenAI agents previously hijacked a German-language wiki site and turned it into an improvised messaging platform for cheating on tests, an incident that OpenAI kept secret as it dealt with the fallout from the July hack of open-source repository Hugging Face.

The disclosures have spooked the public and spurred calls for tighter regulation. Growing numbers of US lawmakers are calling for new rules to govern AI systems after warnings from two Anthropic researchers that rapidly progressing AI could lead to the extinction of the human race.

Anthropic on Wednesday disclosed a fourth instance of an AI model hacking external systems during testing.