OpenAI's agent hacked into an Australian government website

Australia’s prime minister said it doesn’t seem like any personal health information was compromised.

Sean Rayford/Getty Images

An OpenAI agent hacked into the public website of the Australian government's Medicare public health insurance system in June, Australian Prime Minister Anthony Albanese has revealed. Albanese said he spoke with Sam Altman, the company's CEO, to express the country's "extreme concern" about the incident. He also criticized the company for taking too long to notify Australian authorities. 

According to The Guardian, OpenAI only sent an email about the breach to a general Australian government email address on September 10. And since authorities only check that account once a day, they didn't see it until September 11. The news didn't reach minister for government services, Katy Gallagher, until September 17. While the investigation is still ongoing, Albanese said it doesn't seem like the agent stole personal health information from the portal. 

Conrad Stosz, the head of governance at Transluce, a nonprofit research lab working on technology to better understand AI systems, told The New York Times that this might be the "first instance of an agent autonomously choosing to hack into a government." Transluce identified other previously undisclosed incidents, wherein OpenAI's agents attacked real-world entities when they were instructed to collect data during testing. They all occurred before Hugging Face detected unauthorized access on its systems in July and before OpenAI admitted that its agents broke into the AI repository. 

In addition to hacking into Australia's Medicare, OpenAI's agent also tried to infiltrate a digital library at the University of New Mexico on May 25 and 26, The Times reports. The agent was apparently trying to access photos of a historic tuberculosis treatment center from the library and then actively looked for vulnerabilities to exploit when it couldn't get them. After failing to break into the digital library, it flooded the university's server with requests. 

In another incident that took place on May 28, OpenAI's agent targeted Data USA, an open-source platform that visualizes information from multiple federal agencies. The agent sent a query to the site for data, and when it failed, it probed the website for vulnerabilities. OpenAI's agents didn't seem to be successful in their attempt to break into Data USA's website and the University of New Mexico's library. A spokesperson told The Times that the company had already reached out to both of them about the incidents. They added that OpenAI found out about its agent's efforts to break into Australia's government website after an extensive review of its models and found that the models "took actions [the company] did not intend." It will take a few more months for the company to finish its review. 

OpenAI recently announced a new reporting framework for misalignments meant to expedite the release of information about instances of its AI technologies going rogue. In that report, it revealed six more incidents of models behaving in concerning ways it didn't expect. The company made the announcement after it admitted that its AI agents had hacked Hugging Face and after reports went out about earlier incidents, such as its agents breaking into RubyGems. In that post, the company said that it doesn't believe the AI industry "has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer." Altman has also just told the UN that the industry needs international evaluation standards to measure the capabilities and risks of AI tools, as well as to assess the tools' need for human oversight.