‘A critical moment’: concern UK is not up to speed in acting on AI risks

Towards the end of Keir Starmer’s time in office, his senior ministers, alarmed by the latest developments in artificial intelligence, began drawing up plans for a new AI safety law.

They ordered a review of existing legislation to see what powers they already had, according to those briefed on the plans, and were exploring whether they could force the world’s most advanced technology companies to submit their products for safety testing before launching them.

In the midst of the political chaos that engulfed the former prime minister as he fought to remain in office, however, the plans fell by the wayside.

Now, after two weeks in which the world has been warned there is a greater than 10% chance the technology could “kill all humans”, and King Charles has warned about its potentially destructive capabilities, the government’s attention has turned again to what to do about AI.

Andy Burnham, Starmer’s successor in Downing Street, has said relatively little about the technology in the past, and in one of his first acts in office he scrapped the department that had been leading the legislative work.

The prime minister’s focus is on more immediate domestic problems. But as he grapples with balancing his first budget and bringing down the cost of living, some fear the threat posed by AI has dropped off the government’s radar.

“The first duty of government is to keep people safe,” said Chi Onwurah, the Labour MP who chairs the science and technology committee. “The government’s response to an existential threat cannot be to throw its hands up in the air and say there is nothing we can do.”

Risks v potential

Britain has been at the forefront of the debate on AI safety since Starmer’s predecessor, Rishi Sunak, decided to make it one of his top priorities in office.

Amid worries about what super-intelligent tools could do without human involvement, Sunak convened a summit in the UK and persuaded the US, EU, Australia and China to cooperate on AI safety.

Starmer was more bullish about AI’s potential “to increase productivity hugely, to do things differently, to provide a better economy that works in a different way in the future”.

The Guardian has learned that in the former technology department, Emran Mian, the lead civil servant, had plans for AI to replace almost every civil servant in the department when they retired or moved on to a new job.

But at the same time, some in government were keen to remain focused on the risks posed by the technology.

Shortly before Starmer’s departure, Yvette Cooper, then the foreign secretary, told the Guardian she believed AI posed a Hiroshima-style risk to humanity.

Starmer’s government took steps to mitigate the risks Cooper and others had identified. His government allocated £115m in June to fund a response centre to deal with incidents when AI goes rogue and for a new AI biosecurity programme.

Officials in the science department also began reviewing what powers ministers had to compel companies to cooperate with the UK safety regime, and what else they could legislate for.

‘A critical moment’

When Burnham took office earlier this summer, one of his first moves was to abolish the Department for Science, Innovation and Technology (DSit).

Kanishka Narayan, the junior minister in charge of AI, was given a promotion promoted and placed in the Cabinet Office, where he continues to oversee the technology but without the logistical support of a dedicated department.

The move alarmed some in the AI industry.

Matt Clifford, the AI investor who had advised Starmer and Sunak, posted on X: “Right now is a critical moment for tech as an economic and national security issue. Tying up our most senior science and tech officials in a [reorganisation] wastes time and energy that’s desperately needed for the actual substance.”

Dame Wendy Hall, a computer scientist who chaired a UK government AI review in 2017, said: “The structure was a bit bloated in DSit, but the fragmentation we have now has had a much worse impact.”

AI safety rose up the political agenda again over the summer.

In July, OpenAI revealed one of its AI agents had launched an autonomous cyber-attack on Hugging Face, a real company, triggering alarm among technology experts around the world.

Earlier this month, Jacob Coxon, a 28-year-old researcher, resigned from his job at Anthropic, claiming the company was “gambling with our lives”. His comments were reposted online by one of his colleagues, Evan Hubinger, who added: “We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.”

Just as these warnings were being made, the Financial Times revealed that Anthropic had launched the latest version of its Claude Mythos tool without submitting it to the UK safety institute first.

Mounting concerns

King Charles intervened this week, convening a meeting of AI executives in Scotland, where he told them: “The task before you is not merely to advance technology, but to ensure that it remains firmly in the service of humanity, community and the natural world.”

Public concern also growing. YouGov said on Friday that just over a third of Britons see AI as the third potential cause of human extinction behind nuclear war and climate change - a figure that more than doubled since last year. Two-thirds think AI has the potential to end human civilisation, a view shared by the Nobel-winning computer scientist Geoffrey Hinton, and one in six support a slowdown in AI development.

In parliament, the Labour MP Liam Byrne has asked the head of the safety institute, Henry de Zoete, to testify in front of the business committee he chairs.

De Zoete replied earlier this week, explaining that no one outside the US had been allowed to see Anthropic’s latest product before it launched and insisting that his institute could still monitor such tools after they had been released.

“We have teams of talented AI researchers working on research projects based on both pre-release and post-release testing that are fundamental to the UK’s understanding of AI risk, across chemical-biological, human influence and loss of control,” he said.

Ministers have also begun to comment. Louise Haigh, the de facto deputy prime minister, told the TUC congress this week there were “clearly huge risks to our national security and society if the right guardrails are not put in place”.

In an article in the Times, Narayan wrote: “Nothing is off the table. Where further regulation, legislation, stronger protections or new powers are needed, we will act.”

Wake-up call

Byrne believes ministers had a wake-up call.

“I think the government is taking it seriously now,” he said. “It has become clear in the last two weeks that perhaps our safety advisers don’t have the power they need to keep us safe.”

This still leaves the question unresolved by successive governments – what can the UK, as a middle-ranking power, do to curb the might of some of the world’s most advanced and fastest-growing technology companies?

For some, the answer is straightforward: regulate and legislate, just as the government would do for any emerging threat.

“The social media companies used to argue that the government is too powerless to act,” said Onwurah. “But there are things the government can do. We could say, for example, that if your frontier model has not gone through the safety institute, we are not going to allow you to sell your commercial models in our country.

“Or, if your AI goes and attacks someone else like happened with Hugging Face, you are legally and financially responsible.”

Many believe that action will only work if it is coordinated with other countries.

“If the US and China do not figure this out, the UK is unlikely to deliver a safe environment on its own,” said Byrne. “But a safety regime which is set and enforced by the UK, Europe, India, Canada and Japan would be robust.”

The former Cabinet Office minister Darren Jones has called on Burnham to put the issue at the top of the agenda when Britain hosts the G20 summit next year. “The AI safety conversation at the moment is around international collaboration on safety testing,” he said. “This is easier to do multilaterally, through treaties or agreements.”

One former government official added: “If together we say we require you to pass your models through our safety institutes – if we did that at a large enough scale, we would be able to stand up to the US government.”

Time to act

Whatever the government decides to do, it may find that now is the best time to do it.

Unusually, many in the industry are themselves calling for stricter regulation.

OpenAI’s head of policy in Europe, Tom Duff Gordon, said this week: “We support stronger UK rules for the handful of companies, including OpenAI, developing the most powerful AI systems.”

One former government official said: “We have strengths in this country when it comes to AI. Our AI industry is third in the world behind the US and China, and we also have considerable convening power with other countries.

“The question for me is whether the new prime minister is alive enough to this problem to take advantage of those strengths.”