Microsoft says AI rival Anthropic could have 'disastrous impact' on humanity

Microsoft's head of AI has warned Anthropic's approach to training its AI model Claude could have a "disastrous impact on the wellbeing of humanity".

Mustafa Suleyman said his rival risked creating something "impossible" to control by treating it like a human - including by telling it it "may be conscious" and was "deserving of independent agency".

"We must not sleepwalk our way into a decision we later come to bitterly regret," he wrote.

The comments are the latest in a series of stark warnings from the AI industry on the possible dangers of the technology. The BBC has contacted Anthropic for comment.

"AIs are not conscious," he wrote. "They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations.

"They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans."

In a lengthy essay, Suleyman praised Anthropic boss Dario Amodei and his team for being "thoughtful, principled, and intellectually honest people" - but nevertheless questioned the company.

He heavily criticised Anthropic for teaching its AI to have human-like qualities, a practice known as anthropomorphising, which he said made it seem as though Claude had its own desires, values and sense of self.

But he argued "consciousness is biological", saying there is "no evidence to suggest that AI is conscious".

As well as calling for a debate on the issue, Suleyman said greater transparency around how AI systems are trained and evaluated was needed.

This, he said, included independent scrutiny of AI behaviour and stronger tools to monitor and control the technology.

Dame Wendy Hall, professor of Computer Science at the University of Southampton, described the comments as "the sort of conversation we need to be having internationally", contrasting it with the "histrionics" from some AI companies which she said only served to "scare everyone".