King Charles warns AI leaders of 'existential dangers'

King Charles III has warned AI leaders of the "existential dangers" posed by the technology falling into the wrong hands.

The monarch highlighted the potentially catastrophic consequences, saying safeguards are needed before it becomes "too late".

On Thursday, local time, the King was hosting the co-founders of Nvidia and Google DeepMind, along with leaders from OpenAI and Anthropic, in Scotland.

The calls from the King come as concern grows that AI safety efforts are lagging behind the development of increasingly powerful systems.

In recent weeks, the debate has centred on whether the industry can and should regulate itself, drawing interventions from global figures including Pope Leo XIV and US President Donald Trump.

King calls for control 'before it's too late'

The monarch said artificial intelligence had the potential to improve and save lives, particularly in life sciences and medicine.

But its creators have increasingly warned it could develop darker capacities, "perhaps even to take life", he said.

"There seems urgency in adequately considering the existential dangers of such technologies falling into the wrong hands, and being used in potentially catastrophic ways," he told leaders gathered at his Dumfries House estate.

"Surely, we need sufficient means of control before it is all too late?"

The King also raised the environmental impact of AI development.

Attendees included Nvidia founder and chief executive Jensen Huang, Google DeepMind founder and chairman Demis Hassabis, OpenAI Chief Financial Officer Sarah Friar and Britain's Minister ​for Artificial Intelligence Kanishka Narayan.

Paolo Benanti, an adviser to the pope who has called for governments to slow AI development, was also expected to attend.

Debate on threat to humanity

The debate over AI has been thrust back ​into the spotlight after two Anthropic researchers warned that the rapidly advancing ⁠technology could lead to human extinction in the not-too-distant future.

Anthropic chief executive Dario Amodei has also called for companies to slow the pace of AI development.

However, that has sparked an international debate over whether some executives have ulterior motives, such as setting rules that preserve the lead of particular companies.

The meeting in Scotland is not expected to produce any binding agreements.

Buckingham Palace said the discussions would consider whether a shared set of principles could guide the future application of AI, "not merely as a driver of capability and efficiency, but as a tool that upholds human dignity and contributes to the flourishing of both people and the planet".