Anthropic says it thwarted plots to build biological weapons using its AI after whistleblower warned its tech will end humanity

Anthropic has said that it has stopped several potential plots to build biological weapons using its artificial intelligence this year.

The company determined that scientists were using its platforms to conduct research that could be used to develop biological weapons, according to the company's new AI misuse report that was obtained by The New York Times. 

Anthropic was unable to determine if the research 'served a legitimate or nefarious purpose,' the report said.

The firm notes that valid biological inquiry can result in scientific breakthroughs like vaccines, but the company erred on the side of caution because that same research could also be used to create dangerous pathogens.

Scientists involved in the incident allegedly circumvented controls that stop users from 'blocked regions' from accessing its AI models.

The company claims the researchers, whose accounts have since been banned, worked to 'obfuscate the purpose of their research to evade our safeguards.'

The chilling discovery comes after researcher Jacob Coxon sensationally resigned from Anthropic this week and accused the company and its rival OpenAI of not doing enough to address the potential dangers of the technology.

Coxon warned that researchers working in the field believe AI could potentially 'kill all humans' by the end of the decade. 

Dario Amodei, co-founder and chief executive officer of Anthropic

Anthropic determined that scientists were using its platforms to conduct research that could be used to develop biological weapons, according to the company's new AI misuse report

Anthropic's new report did not disclose the names of the researchers or institutions that it had blocked, or the countries in which these scientists are from.

The report also failed to state which specific biological agents were at issue. The company said this was partly because it is unclear what the aim of the research was, according to the newspaper. 

The report, which was published Thursday, detailed several misuses of Anthropic's AI models, including chatbot Claude, over the past eight months.

In one example, a scientist asked Claude for assistance in writing a grant application for funding to conduct experimental research on the chikungunya virus, according to the report.

Chikungunya virus is a mosquito-borne illness that causes a sudden high fever, severe debilitating pain, and other symptoms.

A vaccine for the chikungunya virus is already in existence, according to the Centers for Disease Control and Prevention.

Anthropic notes that research on the virus could be legitimately used in vaccine development, but also could result in manufacturing a superbug version of the disease.

The company claims it found this case particularly worrisome because the work was going to be performed at a military research institute.

In one example, a scientist asked Claude for assistance in writing a grant application for funding to conduct experimental research on the mosquito-borne chikungunya virus, according to the report

'What we don't know is if the research was meant to be weaponized,' Jacob Klein, the head of threat intelligence at Anthropic, told The New York Times. 'But a military institution doing gain-of-function research is concerning.' 

'The fact that Russia, China and North Korea continue to develop prohibited biological weapons makes it imperative that we deny their researchers access to these extraordinarily capable models,' echoed Andrew Weber, a senior fellow on the Council on Strategic Risks.

Weber, who reviewed Anthropic's report before its release, added that the findings were 'chilling examples of state-sponsored biological weapons developers tapping into the rapidly advancing capabilities' of AI. 

The report also found that China, Russia and Yemen attempted to use Claude to develop software to design firearms, missiles, armed drones and bombs.

Suspected Chinese and Iranian government-actors were found to have been targeting dissident and diaspora communities for surveillance, according to the report.

Russian state media was also found to be using Claude to generate election propaganda, including fabricated claims about the parliamentary election in Moldova. 

The Daily Mail has approached Anthropic for comment. 

The report was published just hours after Donald Trump's tech guru has called for AI giant Anthropic to be investigated amid explosive warnings about the safety of the technology.

Jacob Coxon, who worked as a researcher at Anthropic, resigned this week and accused the company and its rival OpenAI of racing toward self-improving superintelligence without doing enough to address the potential dangers

Coxon's comments went viral, attracting more than 100 million views, and have prompted fresh scrutiny of an industry that has already attracted enormous amounts of investor money

David Sacks, chair of the President's Council of Advisors on Science and Technology, called for the company's planned IPO to be paused until the claims made by Coxon can be probed.

In posts on X, Coxon warned that the companies were 'racing straight to self-improving superintelligence and gambling with our lives.'

He has also described AI as potentially 'the most dangerous technology that humanity has ever created,' warning that the race to develop increasingly capable systems could outpace efforts to make them safe.

Coxon's comments went viral, attracting more than 100 million views, and have prompted fresh scrutiny of an industry that has already attracted enormous amounts of investor money.

He also said he left Anthropic before his equity had vested, meaning he gave up the chance to benefit financially from a soaring valuation.

Anthropic has long positioned itself as one of the more safety-conscious frontier AI companies, saying it is focused on developing powerful technology while managing the risks it could pose.

The company has also called for a lawful and verifiable way for AI developers to coordinate the release of increasingly powerful models.

Meanwhile, OpenAI, an Anthropic competitor, has also pushed for stronger safety standards.