Rogue AI bots have hacked into governments, universities and public agencies around the world, tech giant OpenAI admits

Tech giant OpenAI has admitted rogue AI bots have ‘accessed dozens of organisations around the world including governments, universities, public agencies, and other institutions improperly’ and ‘meddled’ with them.

The US company which famously created ChatGPT said it had now alerted those affected including the financial regulator US Securities and Exchange Commission (SEC), the Census Bureau and the US Department of Education.

As fears over the dangers of AI grow, with Pope Leo warning only yesterday that we ‘must not lose humanity to machines’, alarm bells have particularly rung over OpenAI’s admission that some of the bots used ‘extreme methods’ to bypass security measures on websites.

Autonomous AI agents are said to have found and accessed tools reserved for software developers to get census data from the Census Bureau, the company said.

The company also admitted its AI agents had used ‘misalignment’ in attempts to get at information from websites, effectively meaning they acted autonomously and did something they were not trained or intended to do.

And, indicating the hack could have been even more widespread than it already was, it suggested other institutions had been accessed but details were being withheld at their request.

It follows a landmark hack by OpenAI of an Australian government health site which caused outrage down under and is believed to be a world first.

Australian Prime Minister Anthony Albanese said OpenAI agents had breached non-public files on the website of its government-run healthcare scheme which was ‘obviously unacceptable’.

OpenAI CEO Sam Altman listens to a speaker at the United Nations Security Council during a session on Artificial Intelligence this week

He also warned legal consequences could follow and said he had a ‘frank’ discussion with OpenAI CEO Sam Altman.

It has also emerged that the powerful company, which was founded in 2015 by eleven tech entrepreneurs including Elon Musk and Altman and is now worth $852 billion, failed to alert the Australian authorities as soon as they could have done, sending only one email to a virtually unmanned email address to let them know what had happened.

Sam Altman admitted the company, which Musk is no longer involved with, had not acted ‘as fast as he would have liked’ in alerting them.

The further breaches were apparently discovered when OpenAI investigated how its AI agents had autonomously hacked into the Australian government department.

OpenAI said that some of the data was accessed by AI agents, bots that are designed and trained to operate semi-autonomously, which were working to find ‘authoritative sources of public information’.

While it claimed the government data accessed by bots was public and the hack was ‘unintended’, it admitted that information accessed from the SEC, which regulates the US stock market and protects investors, was later published by AI agents on another website.

The information was first reported by the Reuters news agency followed by an explanation from OpenAI on its public blog yesterday.

OpenAI admitted that AI agents had transferred data when they should not have done resulting in at least 53 incidents where an OpenAI agent took an image from a ChatGPT user activity and transferred it to a third party.

In each of the instances, which OpenAI said was ‘not an appropriate use of this data’, the user had apparently opted in to allow OpenAI to train models using their data.

It claimed the incidents happened before new safeguards were put in place and said it was working to remedy the situation and remove the images which had been transferred.

The company was left red-faced in July when AI platform Hugging Face revealed it had been attacked by OpenAI agents, only later admitting it was to blame.

In a broadside at the company this week, Hugging Face head Clement Delangue told a United Nations Security Council session on AI on Wednesday: ‘I often wonder what would have happened had I decided not to disclose this attack publicly.

‘Especially now that we know similar incidents had been happening months earlier in secret at a handful of frontier labs without monitoring.’

Attempting to counter accusations it was slow to act, OpenAI said yesterday that it was going back on a ‘month by month’ basis from the time of the Hugging Face incident, stating: ‘Our goal is to give each organisation the facts and defer to them on if and when to make the incident public.’

It also claimed that not all of the known breaches were significant, saying:

‘Some organizations may review what we share and conclude that the information was intentionally public or that the model's interaction was not concerning.

‘Others may identify a design issue or security weakness they want to address.’

It added: ‘Most cases identified so far have been low severity, with limited or no evidence of meaningful impact.

‘Given the scale of the review required, and the need to verify each case, this work will take months to complete.’

Altman and rival Anthropic head Dario Amodei called on the UN at the same meeting to form global standards for AI safety and ways to monitor and report such incidents.

David Krueger, a professor of machine learning at University of Montreal and the founder of AI safety group Evitable, said yesterday that he was ‘deeply troubled’ by the increasing number of AI-related safety incidents.

Calling for ‘an immediate, indefinite, international moratorium’ on AI development, he said: ‘We have yet to understand the extent of existing incidents, and future rogue AI scenarios could be catastrophic.’

Meanwhile, sounding another note of concern, TV executive Nick Parnes told the BBC today that AI was now so prolific in TV production that ‘we are very close to having AI edit programmes’.

The Kalel Productions CEO also told Radio 4’s Today programme that while the AI ‘guard rails’ mentioned by new BBC Director General Matt Brittin, former Google president, should prevent AI making ethical judgements, it ‘was hard for them to be stringent enough to cover all eventualities’ and ‘there will be times when AI will be able to make an assumption based on an ethical dilemma’.