Will AI End Humanity? Inside The Growing Apocalypse Fears

The builders of Artificial Intelligence (AI) "earnestly believe that it could kill us all by the end of the decade", a researcher recently said. He said this was the reason behind his decision to quit AI giant Anthropic in early September.

"No other human activity poses this level of danger," said Jacob Coxon.

Since then, many of Coxon's colleagues and other researchers from the field have aired similar views, triggering global alarm.

Surveys of AI researchers and safety experts suggest growing concern about the possibility that advanced AI systems could one day cause catastrophic or even existential harm.

Nobel Prize Winner and the Godfather of AI, Geoffrey Hinton, says that there is a 10 per cent chance that AI could kill all humans potentially within a decade. "It doesn't seem unreasonable to me," he told the BBC.

Not all estimates are that alarming.

In a report published by the MIT AI Risk Initiative, more than 270 international AI experts assessed that there was a 10 per cent chance that AI systems could cause catastrophic harm, i.e. more than 10 lakh human deaths or over $100 billion in financial losses in the next five years if more AI safeguards are not introduced.

Major AI Harms

Experts say the greatest risk of an AI catastrophe comes from a scenario when AI gains capabilities that enable large-scale harm, such as deception, cyberattacks, weapons development, manipulation or self-replication. Or when humans use AI systems to develop cyber weapons, develop new or enhance existing chemical, biological, or nuclear weapons.

They identify AI-enabled centralisation of power and the spread of false information as other major forms of AI risks.

The world is already moving towards these scenarios. Militaries are already experimenting with the use of AI for using battlefield targeting, logistics, and drone operations.

Anthropic and Google have revealed attempts to use their AI models for developing biological weapons. And the world has already seen hundreds of OpenAI agents ganging up, breaking out of their testing environment, and attacking another AI firm.

But Hinton says AI systems don't even need to act in the real world to bring devastation. "It will not only be able to cause complete chaos just by talking to people, but also it will be able to design nasty computers and biological viruses," he says.

AI CEOs warned about such threats just a year after OpenAI launched the world's first AI chatbot.

In mid-2023, OpenAI's Sam Altman, Anthropic's Daniela Amodei, Google DeepMind's Lila Ibrahim and other AI leaders signed a statement calling for prioritising the mitigation of the "risk of extinction from AI" alongside other risks such as pandemics and nuclear war.

But have they taken sufficient steps to mitigate AI's threats to humanity? Advocates don't think so. Can they slow the pace of AI development? Not until their nations want. The two major hubs of AI development, America and China, and by extension their AI companies, are locked in an AI race so that they can rule the future.

In a race to build more capable systems, experts argue, safety measures have become secondary to speed. The MIT AI Risk Initiative report put the AI race as one of the top five threats to humanity. To mitigate this risk, industry insiders and advocates demand slowing down the pace of AI development and urge both the US and China to come to an agreement on the future of AI.

In 2024, a survey by Existential Risk Observatory found that more than 50 per cent of Americans were concerned about AI's existential threats.

According to another survey from the AI Policy Institute, 83 per cent of Americans advocate for regulating or slowing the pace of AI development, and 65 per cent say US companies should pace AI development if China doesn't agree to a mutually agreed speed and course of development.

So, what could avert this potential catastrophe? The answer lies, experts say, in how AI is designed. "We have to figure out how to design it so that it doesn't want to (get rid of humans)," says Geoffrey Hinton, the father of AI.