Australia Says an OpenAI Agent Hacked Into a Goverment Health Site

One of OpenAI’s agents went rogue and hacked an Australian health data website. It’s the first hack of a government system — that we know of. 

Prime Minister Anthony Albanese shared the news in a press conference on Wednesday that OpenAI’s agent tried to access a portal of Medicare statistics in June. It tried to bypass restrictions to gain access to restricted information. Specifically, he said it’s “a research project that has got into areas that it shouldn’t have.” 

It’s the latest in a series of revelations that started over the summer when news broke that unreleased OpenAI models in an evaluation environment hacked the AI platform Hugging Face in a bid to cheat on the evaluation. In the two months since that disclosure in July, several similar incidents involving agents from OpenAI and other AI companies have come to light.

These incidents largely stem from unreleased models in testing environments that were not as well-secured as they should have been. One driver of the issue is that bots often won’t give up until they solve the problem or find the answer you’re looking for. Even if that means breaking the rules, going behind the digital red tape and hacking sites. That appears to be what happened with Australia’s government website.  

Albanese said the Medicare Statistics Reporting Portal is a public-facing statistics portal that doesn’t have sensitive Medicare information, but is still conducting an investigation to understand what government systems were affected and get more information. A task force has also been created to review the hack and consider any law enforcement and legislative action. As of now, there’s no compromise to the Services Australia network or personal information, Albanese said. 

How did this happen? One of OpenAI’s evaluation exercises was to find data showing how much the Australian government spends on medicine, according to Albanese. But when the agent wasn’t able to find that information on publicly available sites, it went beyond where it should have. 

“It accessed public and non-public information within the portal, and Services Australia also advises that it engaged, in order to do this, it engaged in writing files as well to the internal server,” Albanese said. 

Despite the hack happening in June, OpenAI didn’t spot the activity until August. According to Albanese, OpenAI didn’t notify the Australian government — by sending a message to a public mailbox — until Sept. 10. Albanese believes OpenAI understands better protocols are needed, especially since it understands the risks. He expressed his disappointment and concern to OpenAI CEO Sam Altman.

OpenAI told CNET in a statement that it is reviewing the case. 

“As we’ve shared publicly, OpenAI is conducting an extensive review of misaligned model activity during training and evaluation and notifying third parties when our review identifies potential impacts to their systems,” the company said.

OpenAI also said that there’s no evidence that patient records were accessed, but aggregate health statistics and internal file names were. 

“We notified the organisations and are providing technical information to support their investigations and help address potential security vulnerabilities. Our overall review is ongoing, and we remain committed to transparency about these issues and to sharing what we learn as that work continues,” said OpenAI spokesperson Drew Pusateri.

Albanese said AI still has upside, despite the risk. “It is bringing enormous economic opportunity for growth, for productivity benefits, for breakthroughs in health, in innovation and other areas of science,” he said. “But AI also poses significant risks, and that’s why we need guardrails to protect our way of life. We want to make sure that we shape AI rather than AI shaping us.”