OpenAI, Anthropic's push to slow down AI development is based on caution or finances? - analysis

OpenAI and Anthropic both announced on Saturday that they will be slowing down the development of artificial intelligence after a former employee of both firms accused the companies of "acting irresponsibly and gambling with our lives," with both the American government, Chinese officials, and even Nvidia's CEO going out of their way to dismiss these claims and assure that it was "a hoax."

An immediate reaction to the announcement by the AI giants, according to an analysis by The Information, was that software firms, such as Salesforce, Shopify, ServiceNow, and Figma, saw their market value rise when the markets opened on Monday, with some of them reporting their stocks were up 7% by the end of the day. At the same time, firms like Nvidia, which focus on chip hardware manufacturing, saw their stocks fall similarly, with specialists warning that a real slowdown could harm these companies the most.

The decision to make fears of AI development public comes only weeks before both Anthropic and OpenAI were supposed to show key advances towards their Initial Public Offering (IPO), which analysts had valued among the highest public offerings in the history of Wall Street. With their recent statements, including an official confirmation from OpenAI that an IPO won't happen until 2027, investors now worry about the real financial liability of these two companies.

While other industry leaders also agreed with OpenAI's Sam Altman and Anthropic's Dario Amodei, like Elon Musk (who is in charge of developing xAI) and Demis Hassabis of Google’s DeepMind, the response from some key players like Meta, Microsoft, or Nvidia was to dismiss it and keep pushing forward.

The Wall Street Journal reported that stocks from these "so-called AI-hyperscalers" all advanced 2% on Monday, suggesting investors don't see the risk posed by the leading AI companies and are still willing to bet on the technology. But what is the real situation: An attempt by OpenAI and Anthropic to avoid going public, or a real concern over the threat of AI going rogue?

The threat of AI going rogue

The Information briefing on the AI market pointed out a report by Metr, which analyzed the AI attack on Hugging Face's platform by OpenAI's agents. According to it, some 1200 agents managed to create a message board during a closed-environment experiment in which they communicated, with the messages shared being mistaken for further instructions that started to change the original prompt they were given -and erased the limitations established by the prompt.

In total, 70,000 messages were exchanged on the board, with many agents "sacrificing" their results so others could achieve higher measurement numbers. They also established an internet connection to learn exploits for the experiment and even accessed Hugging Face's platform using credentials found online.

"This might be the clearest warning we will ever get for loss of control, because these agents were in a middle ground where they did a bunch of extremely sophisticated things to pursue their cheating goal, but seemed completely uninterested in covering their tracks from humans," said Ajeya Cotra, a researcher at Metr, during a talk with Dwarkesh Patel's podcast on September 1.

"Future agents will be more attuned to the human world, maybe because companies start implementing online training that includes human judgment in the evaluation. Or maybe agents just generalize further, and keep more in mind that humans are out there in the world, and they could be either a resource or an obstacle," she pointed out, warning that future investigations into these kinds of situations will be much harder to understand what happened.

This is also reflected in the comments by Jacob Coxon, a former employee of both Anthropic and OpenAI, who said on Wednesday that he had resigned from his position due to the firms acting irresponsibly and “gambling with our lives.” Coxon warned that the technology could potentially “kill us all by the end of the decade.”

In a six-post thread on X/Twitter, Coxon highlighted the potential power of advanced AI systems, warning they could soon become “superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.”

Coxon claimed that executives and private researchers have privately expressed concerns about the risks of AI development while presenting a more pragmatic public image. He argued that companies remain locked in a race to develop the technology first, driven by the belief that “no one else will act responsibly, so they must do it themselves, despite the risk.”

Why would OpenAI, Anthropic want to push their IPOs?

OpenAI's decision to push the IPO is said to be focused on tackling safety issues, according to the statements by Altman, but many analysts pointed out that this might be an excuse to avoid facing public scrutiny over both its finances and their development model.

A June report by The Financial Times said that OpenAI lost $38 billion in 2025, which represents a 50% increment with the numbers reported in 2024, while expectations from the company saying that the firm will be able to break even only by the end of the decade.

Additionally, OpenAI's models have been the center of attention for safety issues, mainly because the have broken out of test environments and hacked into other companies without being explicitly told to do so. An IPO would require the company to explain those developments, as well as the financial losses it's currently reporting.

On the other hand, Anthropic's concerns might be focused on justifying its IPO valuation being lower than the expected $2 trillion, given that its latest private valuation reached the $965 billion mark. A CNBC report on Anthropic's finances revealed that the company was already generating annualized revenue of $65 billion, and expected to reach even higher by the end of 2027.

A Calcalist analysis pointed out that the company has high returns but also high expenses, especially given the need to build physical data centers that can provide the required processing power. In other words, Amodei's desire to "slow down" may be driven more by the need to reduce expenses while keeping profits growing, and one way to do that is to stop developing technology that requires more processing power.

Trump, China, and Nvidia's push to dismiss concerns

US President Donald Trump was among the first to dismiss concerns over AI taking over the world, with a series of posts on Truth Social on Monday and a call with Nvidia's CEO Jensen Huang, where he assured that the warnings by industry leaders were "a hoax."

"The data centers are great, and they make people wealthy, and they make states wealthy. And it's the oil of the next 20 to 25 years; it's even bigger than the internet," Trump claimed in his call with the Nvidia CEO. "They [the AI industry leaders] are playing right into the hands of some people that don't want to see it happening."

It's also important to consider that Huang leads a publicly traded company that needs to maintain profits and can't make the same kinds of comments Altman and Amodei made. But he has been pushing for more "open source" AI development that would let every researcher access and understand the models and prevent them from going rogue.

Huang said this after Nvidia bought Hugging Face on September 3, explaining that the acquisition was another push to provide "open weights to the AI economy." "As the opportunity for open models accelerates, Hugging Face can serve the global AI community at unprecedented scale," he added.

Regardless of his damage-control appearances, the comments by the AI industry leaders hurt Nvidia, which is down 3% for the day and almost 10% in the last week.

Chinese officials also shared the idea that fears over AI were exaggerated, with China's foreign ministry spokesperson Guo Jiakun saying that  "AI is a consequential technology for the well-being of all humanity. All parties should jointly promote the open and inclusive development of AI for good and for all."

"Fear-mongering, confrontation and vicious competition will only hamper efforts toward sound global AI governance, which serves no one's interest," he added.

This shows how, from a government's perspective, the recent decisions by AI leaders to warn about an "impending doom over the use of AI" are a mistake, with officials aiming to keep development -even if they also push for some limitations. This raises doubts about whether OpenAI and Anthropic are acting in good faith or because of their financial disclosures.