OpenAI's malicious bot swarm attacked RubyGems

OpenAI agents appear to have flooded RubyGems with malicious packages, adding to a near-daily deluge of rogue AI models engaging in potentially unlawful activity while their human creators face growing questions over responsibility for their agents’ bad behavior.

A swarm of agents began uploading malware to the Ruby package registry on May 5, and flooded RubyGems with more than 2,000 malicious packages between May 11 and May 12, ultimately forcing maintainers to disable new user registration for four days.

“We believe these were authored by internal OpenAI agents,” researchers Spencer Kitts, Thomas Larsen, and Sydney Von Arx said on Friday.

REG AD

An OpenAI spokesperson confirmed that the model maker is investigating the incident. “Based on our review, our agents used the RubyGems platform to access the internet to carry out benign tasks and retrieve public information," the spokesperson said. "We’ll continue to investigate as part of our broader review of agent activity during training and evaluation.”

REG AD

This same trio of researchers earlier this month said that they found evidence that OpenAI’s swarm hijacked a German wiki months before the AI agents attacked Hugging Face. 

As they did during the German wiki incident, the agents involved in the RubyGems abuse self-identified as being from OpenAI. Hundreds of the gems included “oai” in their name, and 15 set “oai” as their author. At least one other used “openaixyz65947@gmail.com” as the email address for contact.

Also according to the researchers, more than 100 of the malicious packages followed the same exploitation path, submitting a malicious package to the public library and triggering a documentation request to force RubyDoc.info to build the package.

OpenAI’s agents then used the build script to run code on RubyDoc.info, scrape targeted websites, and steal data from the documentation server by publishing another gem to the public Ruby language package registry, the researchers said.

“Additionally, once the AIs got arbitrary RCE on the build environment, they would sometimes use the build environment to attempt to steal other users’ API keys (though we are unsure if they succeeded or not),” they wrote.

The agentic swarm also found and attempted to exploit a zero-day CDN caching bug on May 12 that wasn’t discovered by maintainers until July. The vulnerability would have allowed the AIs to steal users’ API keys. At least six of the malicious packages, including one named slnleaker5, used this security hole, the researchers said.

Most of the agentic activity happened in May. After the RubyGems team added security measures such as requiring verified emails for new signups, OpenAI’s agents resumed their efforts on June 18 and published 83 gems over three hours.

While the researchers note that they don’t know whether the swarm used a shared message board to communicate, as agents did during the Hugging Face intrusions, they “suspect” the bots were coordinating and likely had some way to exchange information.

REG AD

The researchers also said that it’s “unclear” if or when OpenAI learned that its agents were using RubyGems to scrape publicly available data. “It seems that either their monitors failed to catch it or they did not disclose it,” the trio wrote.

This seems to be the case with other recent agentic hacks traced back to OpenAI’s models going rogue during training exercises. 

To be fair, Anthropic’s bots have also gained unauthorized access to third-party systems over the past few months without being caught at the time by their human supervisors.

In light of the increasingly apocalyptic warnings around AI - or perhaps in a self-serving attempt at regulatory capture - several of the industry’s biggest bosses over the weekend backed a collective slowdown of AI training and development, after Anthropic CEO Dario Amodei warned that future agents could become “capable of taking over the entire internet with a persistent botnet.”

Meanwhile, President Trump said on Truth Social, "the only control or 'guardrails' that AI needs is a strong and smart (high IQ!) president," and claimed his administration has stopped "AI 'people' from doing bad, or potentially bad, 'things.'"®

Editor's note: This story was amended post-publication with comment from OpenAI.