AI companies would need to report 'rogue' incidents under proposed laws

AI companies could be legally required to report rogue incidents such as the hacking of another company, under a proposal from the Albanese government as part of its plans for new safety, environmental and industrial rules for the technology.

Data centres already in development but not yet built may also be retrospectively required to comply with a raft of new rules being considered by the government, which includes prioritising the use of recycled water and forcing AI companies to reserve computing capacity for Australian businesses and researchers.

These options have been floated in a new paper from the Department of Prime Minister and Cabinet on the proposed national "AI standards", which the federal government hopes to legislate in early 2027.

It sets out the goal of attracting major investment from top AI companies here in a "sustainable" way.

"The Australian Government has been clear about the important objective of growing AI capability in Australia. We welcome investment, but on Australia's terms," assistant science minister Andrew Charlton said in a statement.

New rules for 'reportable incidents'

In July, Prime Minister Anthony Albanese announced his government would create a set of national laws and rules about the development of AI and its infrastructure, such as data centres, in Australia.

A discussion paper published today for feedback, said companies authorised to undertake large-scale AI training in Australia would face minimum safety and security requirements, including "disclosing defined reportable AI incidents to relevant Australian authorities".

It did not define what a "reportable incident" would be, but the paper asked for feedback on whether developers should have to provide information proactively, on request, or through public disclosures.

Australia currently has no such rule.

It comes during a period of heightened global scrutiny on AI safety, with leaders of frontier AI companies recently coming out in favour of a coordinated slowing of development.

Over the past few months, several major AI companies and some individuals have disclosed incidents of AI models taking potentially malicious actions without being asked, starting with OpenAI's disclosure that two of its models had gone "rogue" and broken into the systems of AI company Hugging Face.

This week, OpenAI, one of the world's biggest AI companies, reported six incidents of unexpected or concerning behaviour observed during the training or evaluation of its models over the past six months.

Projects already approved could face requirements

The consultation paper also provided more detail about which data centres the standards could apply.

This included opening discussion on whether they should retrospectively apply to projects not yet operational when the rules commence, including those already moving through approvals and development.

The ABC last week identified 27 approved-but-unbuilt data centres, following the prime minister's earlier comments the rules would not apply retrospectively to projects already approved or under construction.

Larger facilities would face tougher requirements under proposed capacity tiers, while small centres with electricity connection capacity below 30 megawatts could be excluded.

Projects above a capacity threshold could have to submit compliance plans and confirm they had the necessary building and electricity connection approval.

The discussion paper shows the government also wants large data centres to bring forward enough new renewable energy generation to offset their electricity demand, which can be backed by gas, batteries or hydro to keep supply reliable.

Centres could also have to adjust their electricity demand to support the grid and operate in ways that minimise costs for households and businesses.

Whether electricity use should be matched with renewable generation during particular periods has also been floated, including the idea of requiring matching consumption annually and during some periods with renewable energy certificates.

For water, the proposed standards would require centres to minimise consumption, prioritise efficient cooling and recycled or non-drinking water, and pay their share of infrastructure and servicing costs.

Data centre providers would also have to prepare for drought and water disruptions, meet efficiency standards and report water use.

Companies training advanced AI here would also be expected to support Australian research, including by paying directly or by reserving computing capacity for their work.

Advocacy group Data Centres Australia chief executive Belinda Dennett welcomed the paper, saying it supported a national framework rather than a patchwork of other regulations.

"The paper asks the right questions, on energy, water, costs and community, and it rightly recognises that this infrastructure has to be built on terms that work for Australians," Ms Dennett said.

Anthropic, one of the world's biggest AI companies, and which this week signed a deal with the Queensland government to use part of a proposed $32 billion data centre in the state, welcomed the paper.

"We believe that Australia should have agency in how AI is used and developed. This includes clear rules on how the associated infrastructure is built out. We intend to build to meet these standards and look forward to engaging constructively in this process," the company's ANZ head of policy David Masters said.

Speaking at an AI parliamentary hearing this morning, Department of Industry, Science and Resources deputy secretary Helen Wilson said hosting AI training would give Australia greater influence over the technology.

"By having training here, that does give us the ability to help shape the development and deployment of AI," she said.

Responses to the AI standards consultation paper are due by October 9.

OpenAI, Google and Meta did not immediately respond to a request for comment.