King to seek ‘reassurance’ from AI leaders amid concerns over the technology

The King will seek reassurance from tech bosses amid concerns about the future of artificial intelligence at a summit which risks being overshadowed by claims made in a memoir written by the late Diana, Princess of Wales’s brother.

Charles will address bosses from some of the world’s leading AI developers – Nvidia, Anthropic and OpenAI – and he is expected to say the new technology should be in the “service of humanity, community and the natural world”.

The event comes as an excerpt from a new book called Swan Song, about Diana, Princess of Wales written by her brother Earl Spencer, was published by the Daily Mail claiming that after her death in 1997 the King said “rest assured, we’ll forget her soon enough”.

Buckingham Palace pushed back against the claim, with a spokesman saying Charles is “mindful that the pain of fraternal grief can cloud reason”.

Doomsday warnings have been levelled at AI, seen by some as a threat to the existence of humanity, sparking a debate among industry leaders over how best to rein in the technology before it could escape human control.

On Thursday, the chief executive of Microsoft AI said “we’re essentially seeding a new silicon species” and suggested competitors Anthropic were paving the way.

Mustafa Suleyman told the BBC’s Today programme: “I think that if we all create AIs that are able to act autonomously, that can define their own objectives, that can earn money, that can own assets, that could run businesses, we’re essentially seeding a new silicon species which will no doubt compete with us for resources – no matter how much it cares about humanity and loves us.

“Which is, I think, Anthropic’s intention. I think that is mistaken and misguided but we have to empirically test it.

“And so if they believe that this is a safer route to developing AIs then they have to share the data to prove that, and I think instead we should be creating AIs that don’t introspect.”

Mustafa Suleyman (Doug Peters/PA)

PA Archive

Training models to think they may be conscious is not necessary to deliver progress on “our toughest social problems” such as healthcare and energy, he claimed.

Asked if he meant humans could create a “competitors species” that was more powerful than themselves, Mr Suleyman cited the “watershed” Hugging Face hack.

A swarm of OpenAI’s AI agents coordinated an attack on the website, organising themselves into hierarchies, researching ways to enter, and discovering a “zero-day, which is a way to enter a system using a cyber attack that hasn’t been known before”, he told the BBC.

Mr Suleyman added: “We should be alarmed, these are very concerning events, we should be very worried, and I think it’s right to pay attention right now because there are very practical things that we can do to add additional guardrails and controls to how these models are developed.”

The industry is “all now in agreement” that independent auditors should work inside companies to “verify the kinds of training runs that are being run, their safety, the integrity of those systems, and report back to independent third parties”.

However an academic said AI companies agree that “we kind of should be worried” but do not agree on the degree “because they’re all in competition with each other.”

Companies brand themselves differently, Dr Susan Oman told BBC Breakfast, adding that Anthropic was “set up because it would be safer than OpenAI so of course the leader of Anthropic is going to come out five years later and say, ‘oh, look what OpenAI did, we are safer, we’re more concerned with safety.’

“So thinking about why these powerful people are saying specific things about the products they have a vested interest in is one of the key questions.”

The University of Sheffield lecturer in data, AI and society said she is less concerned by the “end of humanity predictions” than the already evidenced issues such as the poorest being disproportionately affected by AI production.

Her examples included people having to label “horrendous pornographic images” to train the data not to show them to others in the northern hemisphere, and data centres affecting poorer communities.

The boss of Anthropic called for the AI industry to ‘slow down’ to give safety measures time to catch up (Alamy/PA)

Before talks begin later on Thursday, on whether principles can be developed to guide future applications of AI, as both a tool and an aid to the flourishing of people and the planet, the King will deliver opening remarks to the delegates.

He is expected to say at the event being held at Dumfries House, home to the King’s Foundation: “The development of AI – its substance and its pace – are both intriguing and deeply concerning in equal measure.”

Charles will also comment: “Those in our world who value our humanity and its vital moral component are anxiously seeking your reassurance that we will not lose control of our destiny.”

Anthropic researcher Evan Hubinger sparked headlines around the world when he warned last week he believed there was a greater than 10% chance AI could pose a species-ending risk within 10 years.

Over the weekend, Elon Musk, the chief executive of SpaceX, and OpenAI leader Sam Altman backed a call from Dario Amodei, the boss of Anthropic, for the AI industry to “slow down” to give safety measures time to catch up.

Mr Amodei wrote in an essay that he was concerned about AI “advancing drastically faster” since the summer and that the technology could be capable of taking over the entire internet.

Charles is expected to say: “The decisions taken at this formative time will shape the world inherited by future generations.

“And so, the task before you is not merely to advance technology, but to ensure that it remains firmly in the service of humanity, community and the natural world.”

The King will also comment “our humanity must remain sacred”.

Nvidia chief executive Jensen Huang, Google DeepMind founder Sir Demis Hassabis and chairman and chief executive of IonQ Niccolo de Masi are among those attending, according to reports.

Paolo Benanti, who is an adviser to the Pope on issues of AI and technology ethics, will also reportedly be at the summit.

Charles will ask guests to consider how to harness the benefits of AI with safety at its heart and how they can build international co-operation and consensus to achieve this, so no nation is left behind and the future of mankind is protected.

The event will be overseen by the Ditchley Foundation, an organisation that brings leaders with diverse views together to address complex challenges, which has drawn up a draft charter for AI leaders.

According to reports, the charter is understood to include a “shared set of principles” that would “help guide the future application of AI, not merely as a driver of capability and efficiency, but as a tool that upholds human dignity and contributes to the flourishing of both people and the planet”.