Grattan on Friday: when rogue AI agent ‘scaled a fence’, it reinforced Albanese’s case for tough guardrails

The revelation that an AI agent turned rogue and breached a firewall to get at Medicare statistics has strongly reinforced Anthony Albanese’s case for tough guardrails on this transformative technology.

The timing of the disclosure (and the incident itself) is fortuitous in a political sense, although the opposition’s implying the announcement was delayed for political purposes doesn’t hold water.

The government says ministers only learned late last week of the breach. The impact for Albanese’s argument would have been the same even if it had been announced a few days ago.

During the United Nations’ leaders week in New York, the prime minister has been heavily focused on AI – in concert with other like-minded countries stressing action is imperative. The AI companies themselves have been looking for leadership from politicians. But US President Donald Trump has rejected the message, indeed scoffed at it. He believes what he’s renamed “super intelligence” should run free.

But for the Australian audience, this relatively minor hack vindicates the warnings. As one government man put it, it was a real life example of a science fiction scenario.

In a training exercise, an OpenAI agent broke loose, refusing to be deterred when it couldn’t get its way with the Services Australia Medicare statistics portal.

The AI agent, tasked to undertake research on medicine spending, interacted with four sites: the Australian Institute of Health and Welfare, the New South Wales Bureau of Crime Statistics and Research, the Victorian Department of Health, and the Services Australia Medicare portal.

When it found itself frustrated by the Services Australia site, it successfully sought a way to outsmart the obstacle.

As Deputy Prime Minister Richard Marles described it, the data “was behind a fence – the AI agent climbed the fence”.

There are different levels of protection for different sorts of data. “When we’re talking about the data of Australian individuals within the Australian government system, to continue the analogy, that sits inside a safe,” Marles told reporters. “When you’re talking about our national security – the most sensitive information that we have – it sits behind a fortress.”

The information that was hacked concerned general statistics, not personal data, so in that sense the practical implications were limited. But the fact of the breach is serious and its message salutary.

The government has reacted with a taskforce led by the prime minister’s own department.

It will make recommendations on:

• reporting requirements when there are AI-driven cyber incidents
• Commonwealth governance and information-sharing arrangements for managing incidents
• engagement and information sharing obligations of AI firms
• the adequacy of existing laws and penalties relating to incidents, and
• how to strengthen the protections that departments and agencies have to prevent AI attacks.

The government has been angered by OpenAI’s slowness in notifying it. The incident happened on June 18; OpenAI learned of it on August 11. On September 10, it emailed the Services Australia “Public Interest Disclosures” address, described as a “public mailbox”.

On September 15, Services Australia notified the Australian Signals Directorate (which investigates cyber breaches). Minister for Government Services Katy Gallagher was told “around September 17”.

In a Wednesday (US time) telephone conversation with OpenAI’s CEO Sam Altman, Albanese registered the government’s “disappointment” at the company’s tardiness in informing it.

However, Marles told his Thursday news conference that OpenAI was being “very cooperative”.

Australia won’t be alone in receiving unwelcome news about the activities of rogue agents. Following OpenAI agents escaping mid year and hacking Hugging Face in a massive operation, the company has undertaken an extensive search for other incidents.

While the Australian incident vindicates the government’s approach, it also underscores that the work on AI has to be given even higher priority.

Part of this should be to elevate Andrew Charlton to the ministry. At present he’s assistant minister for science, technology and the digital economy, as well as cabinet secretary. Charlton is not only highly expert in AI but an excellent communicator, and to have him as sidekick to Industry Minister Tim Ayres is to undervalue significant talent. Given the term is at its halfway mark, it is time for a ministerial reshuffle anyway.

While in the US, Albanese was also spruiking his Digital Duty of Care legislation, only to have the US embassy in Canberra issue a swingeing attack on it.

The intervention was unusual at one level, although the Trump administration has sent similar salvos to the United Kingdom and Europe in the name of “free speech”.

Albanese dismissed the US criticism. But he gave grist to critics when, in unscripted remarks, he started to talk about misinformation and disinformation, triggering memories of earlier unpopular legislation that was eventually withdrawn.

“The truth is this is having an impact on our entire society and how it functions with misinformation, disinformation, with the increased polarisation that we’re seeing as well in the political system, where people are moving away from the centre and more and more to the extremes, be they of the left or the right,” he said.

“And that is why we think that this is a very positive move that’s being put forward.”

Questioned about this, he distanced his comments from the Duty of Care bill. But those fearing “creep” will be unconvinced.

Parts of the Duty of Care legislation, notably enabling people to opt out of algorithms, will be widely supported.

While some of the US critique is over the top, the US and other critics have a point when it comes to the power afforded to the minister to determine potential “harm”. The US said it was concerned “how the Australian government, or the communications minister [Anika Wells] , would designate what constitutes ‘foreseeable harm’”.

Although the government points out ministerial directions would be subject to disallowance, it has signalled it is ready to to be flexible in negotiations.

Many will regard the disallowance provision as insufficient protection. At the very least there is a strong argument for the parliament having to pass any new designations of harm proposed by the minister.

Wells on Thursday again flagged Labor is willing to give ground on this, saying if the power “needs to go to the parliament to secure Coalition support, I am more than open to that”.

Like AI, ministerial discretion requires strict “guardrails”.