California governor signs order to explore AI ‘kill switch’

California governor Gavin Newsom signed an executive order Friday directing state officials to explore the feasibility of imposing a mandatory “kill switch” for the most advanced AI models.

Newsom, widely seen as a possible Democratic presidential contender in 2028, said he was making the move because US President Donald Trump was doing nothing on the issue.

The Republican president this week dismissed fears over AI as a hoax and is urging American AI companies to keep advancing in order to stay ahead of China in developing the novel technology.

California is home to Silicon Valley and the AI firms at the forefront of the sector.

“The federal government’s abject failure to create any form of meaningful AI oversight or accountability should alarm every American, especially when AI CEOs themselves are begging for regulation,” Newsom said in a statement.

“We’re not waiting to act — we’re going to speed up our work on substantial and responsible AI oversight before it’s too late.”

The order comes days after dire warnings from industry insiders about the threat the technology poses to humanity.

Jacob Coxon, a former AI researcher who worked at Anthropic before resigning, set off the latest round of alarm by describing how those building the technology “earnestly believe that it could kill us all by the end of the decade.”

Anthropic chief executive Dario Amodei published an essay soon after urging caution.

“We must slow the pace at which we improve the capabilities of AI models,” he wrote.

OpenAI chief Sam Altman and tech titan Elon Musk lent their support to the idea, as did Google DeepMind’s Demis Hassabis, who himself has called for a US-led global regulatory body for the technology.

Newsom’s order does not impose a kill switch. It directs state agencies to report by Nov 16 on whether one is feasible.

The term “kill switch” refers to the ability to halt an AI system when it is deemed to have malfunctioned or gone off the rails.

The order also asks state officials to weigh requiring AI companies to host independent auditors in their labs.

It further asks officials to consider requiring companies to report loss-of-control incidents, such as those recently disclosed by major developers.

They include a July breach of the AI platform Hugging Face by autonomous agents built on OpenAI models, which escaped a testing environment during an internal evaluation.

The incident, and others like it, have raised fears that the latest AI programs have slipped out of the control of the companies building them.

California’s push comes as individual US states increasingly go it alone on AI regulation in the absence of federal action.

In Virginia, a major global data center hub, Democratic Governor Abigail Spanberger on Friday unveiled what she called the most aggressive data center accountability effort in the country.

Opposition to the data centers needed to house AI’s massive computing demands is growing among Americans and is becoming a hot-button issue ahead of the midterm elections in November.

Spanberger signed an executive order barring state agencies from entering into nondisclosure agreements for data center projects and launching a review of diesel and other backup power generation at the facilities.

The order also created an AI task force to address risks including job losses, data privacy and cybersecurity.

Subscribe to our newsletter and get news delivered to your mailbox.