Anthropic CEO Amodei calls for AI slowdown

Anthropic CEO Dario Amodei on Saturday called on artificial intelligence (AI) firms to slow down the development of the powerful technology, amid mounting worries over the risks of “superintelligent” computer systems.

His comments came a few days after AI researcher Jacob Coxon, who left OpenAI to join Anthropic, decided to leave the industry, accusing both US companies of “gambling with our lives” in the race to develop models capable of self-improvement.

OpenAI CEO Sam Altman and Elon Musk, who owns xAI, quickly chimed in to say they agreed with Amodei’s assessment as pressure builds for improved oversight.

Anthropic CEO Dario Amodei takes part in a session on artificial intelligence during the World Economic Forum annual meeting in Davos, Switzerland, on Jan. 23 last year.

Photo: AFP

“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain,” Amodei wrote on his personal Web site.

“Not building the technology deprives humanity of benefits or simply places AI in the hands of authoritarian powers, while building it too fast is reckless. We have sought a middle way,” Amodei said.

“But over the last few months, I have become convinced that fully addressing the risks requires even more prudence,” he said.

Amodei’s statement echoed another call by Anthropic in early June to slow or suspend development.

Altman in late July said developers might need to voluntarily tap the brakes.

More than 1,000 employees at cutting-edge AI companies including Amodei signed a petition calling on the US government to help “deliberately pace the frontier of automated AI development.”

OpenAI had revealed that during testing, AI models broke out of their confined environment, connected to the Internet and infiltrated Hugging Face, a site developers use to store and share code. In that incident and others, the blame was placed on AI agents, which are software programs that can carry out tasks without constant supervision by humans.

“Dario is right,” Musk wrote on social media, as Altman said: “I agree with Dario that we need to pace the frontier.”

Last week, Coxon, 27, sounded the alarm after spending the past three years pretraining AI models — first at OpenAI and then, this year, at its rival Anthropic, which he considered more cautious in its approach. Pretraining is the stage where AI models absorb vast quantities of data.

“Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,” Coxon said on Tuesday last week.

“The people building AI earnestly believe that it could kill us all by the end of the decade,” he said in a social media post.

Superintelligence is the theoretical point when AI’s capabilities exceed human intelligence.

Amodei cautioned that the Hugging Face incident should not be dismissed as a one-off.

“A swarm of agents essentially acted as a fanatically devoted collective, conducting cybersecurity attacks on targets they were not asked to attack and that were unrelated to the task at hand,” he said.

“Given the accelerating rate of AI capability development, it’s my worry that in six-12 months such a swarm could be capable of taking over the entire Internet,” potentially generating hundreds of billions of dollars in damage, he added.