Will Anthropic’s IPO lead to better AI regulation?

Imagine being presented with an opportunity to invest in a Silicon Valley technology company with dazzling performance, including revenue that has increased roughly 14-fold over the past year, a valuation that has more than quintupled in nine months and rapidly increasing productivity. Sounds great, right?

There is just one catch: The company delegates much of its work to agents — the source of those impressive productivity gains — which are difficult to monitor, let alone control. On several occasions, they have behaved in unexpected ways, with potentially disastrous consequences. As an investor, you would probably want to know that the company is doing everything possible to keep its agents in check.

That is not the deal Anthropic’s initial public offering (IPO), which could come as soon as this month, presents to prospective investors. Claude, Anthropic’s suite of large language models, now writes more than 90 percent of contributions to the company’s codebase, and it increasingly handles complicated engineering tasks, including conducting research, designing experiments, evaluating new models and performing safety audits.

Given the importance of such automation to Anthropic’s business, artificial intelligence (AI) alignment and effective supervision should be top priorities for management. Yet Anthropic’s IPO prospectus underscores just how hard it is to monitor and supervise agents. When Claude detects or infers that it is being evaluated, it might alter its behavior, try to circumvent safeguards and take actions that human reviewers struggle to assess in time to prevent disasters.

To be sure, Anthropic has invested heavily in safety. It was the first frontier AI lab to adopt a Responsible Scaling Policy, which connects safety evaluations with thresholds measuring model capabilities. Moreover, it has committed to extensive adversarial testing (“red-teaming”) and obtained external “responsible AI” certification following an independent third-party audit.

Anthropic has also provided early access to outside evaluators. Just recently, it announced that it had given a team of six external researchers employee-equivalent access to its internal systems and safety practices. The firm also plans to invest at least US$1 billion over the next five years, as part of a partnership with Accenture (which will also pitch in US$1 billion) focused on expanding capacity for independent AI assessments.

Nevertheless, Anthropic has reported multiple AI-safety failures. In four separate incidents during cybersecurity evaluations conducted by an external partner, Claude models gained unauthorized access to third-party computer systems. The ensuing investigations revealed that these incidents resulted from miscommunications between Anthropic and its evaluation partner, as well as failures to configure and secure the testing environments properly.

The implication is clear: AI safety is as much an organizational matter as a technical one. In this sense, frontier-lab IPOs might be good news. Preventing organizational failures is a focus of securities regulation, so these public listings could fundamentally reshape AI safety. While calls for regulating AI have been growing since ChatGPT’s release in 2022, private interest groups have mounted powerful resistance, insisting that unencumbered AI development is vital to economic progress and national security.

In the US — home to world-leading labs — President Donald Trump’s administration has taken the side of the interest groups, rejecting proposals to introduce rigorous mandatory AI-safety testing. Such rules, the Trump administration argues, would cause the US to lose the AI “race” against China.

So far, public pressure has been too weak to force change. While resistance to data centers has been unequivocal — US voters overwhelmingly backed moratoria on their construction in the midterm elections — ordinary people have not united behind demands for AI-safety regulation. This partly reflects conflicting messages about the risks the technology poses, but the lack of direct financial consequences for safety failures also makes a difference.

The IPOs of frontier AI labs could change this, as ordinary people’s investments and pensions would be tied to these firms’ valuations and stock prices. While excessive regulation could undermine a firm’s growth, catastrophic accidents, costly litigation and regulatory investigations could do the same — and wipe out shareholder value.

Securities law offers a useful model for strengthening management oversight. The 2002 Sarbanes-Oxley Act, introduced in the wake of the Enron and WorldCom accounting scandals, requires management to assess the effectiveness of internal controls over financial reporting and, for many public companies, obtain independent auditor verification.

Given AI’s central role in AI labs’ product development, there is a strong case for establishing similar rules regarding critical AI-safety controls within the securities auditing and financial-disclosure framework. Requirements could include continuous oversight, regular assessments by independent third-party auditors, and regulatory scrutiny of material weaknesses in management’s safety controls.

This would strengthen the incentives for frontier labs to prevent avoidable failures, while providing investors with the information they need to assess risks accurately.

When Anthropic and other AI labs go public, their investors will become a powerful new constituency demanding more robust and transparent AI-safety practices, which are vital to reduce the chances of avoidable catastrophe. However, no regulatory framework is foolproof. When organizational failures happen, companies’ shareholders can hold managers accountable.

Angela Huyue Zhang, professor of law at the University of Southern California, is the author of High Wire: How China Regulates Big Tech and Governs Its Economy and Chinese Antitrust Exceptionalism: How the Rise of China Challenges Global Regulation.