Big AI sets out its terms for regulatory capture and calls it ‘Pace the frontier’

Amodei, Altman, and Elon Musk agreen on how government can tame the monster they created

ANALYSIS The leaders of major AI labs spent the weekend agreeing on a plan to capture regulators, make more money, and avoid responsibility for their dangerous behaviour. They call it “pacing the frontier.”

Anthropic CEO Dario Amodei set the ball rolling with a post in which he professed alarm at how quickly AI is improving and suggested the attack on Hugging Face caused by rogue agents run by his rival OpenAI as a warning (which he referred to as ‘OAI-HF’) represented a moment that proved something needs to change at so-called “frontier” AI companies – essentially the big US model-makers.

“It’s also easy to dismiss OAI-HF as the failure of one company, but I believe that would be a mistake,” he wrote. “I believe it’s incumbent on every frontier AI company to act as if OAI-HF had happened to them,” he added, because he worries that OAI-HF shows that a swarm of agents “could be capable of taking over the entire internet with a persistent botnet.”

REG AD

Amodei therefore suggested “We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain.”

REG AD

The CEO proposed a three-point plan to regulate AI:

• Requiring AI labs to host “embedded evaluators” whose job is to “verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes.”

• Frontier AI companies that operate in democratic countries collaborating “to establish common safety standards as well as limits on the rate of unchecked AI progress,” with undefined “forms of coordination” that would be “legally challenging” and “require government support.”

• A vague call for democratic governments “to coordinate with authoritarian governments, to the extent this is possible, while taking seriously the challenges of verifying compliance.”

OpenAI boss Sam Altman endorsed Amodei’s ideas. So did Elon Musk.

And just like that, three billionaires all signed up to the same set of rules they think the world’s governments should adopt to regulate their activities.

Amodei even gave democracies a threat/villain to unite against – authoritarians willing to use AI unethically.

This new consensus comes after years during which Big AI argued for light oversight and used that time to build tech that generates child sexual abuse material, dispenses terrible health advice, and made the OAI-HF incident possible.

Big AI has also argued it will deliver a productivity revolution. Daryl Plummer, chief of research at analyst firm Gartner, used his keynote speech at the company’s annual Symposium in Australia today to cite research that found software vendors are pitching 50 percent productivity gains from AI, but customers report a 16 percent lift.

Plummer also doubted that Amodei’s promise to slow development is real.

REG AD

“I will believe that when I see it,” he said in the keynote.

Write your own rules

Amodei’s ideas therefore read like an attempt at “regulatory capture” – the situation in which vested interests find a way to define the rules their regulators impose. Those supine regulators then make decisions that benefit the entities they oversee more than they benefit the rest of us.

In this case, Anthropic, SpaceX, and OpenAI get regulations that amount to a cease fire during which they don’t need to compete so fiercely.

OpenAI boss Sam Altman also all-but-admitted that regulation will be good for investors when he used a weekend interview with the billionaires’ bible Fortune to reveal his company won’t seek a public listing this year because it is inopportune to do so while AI safety concerns are unresolved.

“We got a lot of stuff to do, like meeting this moment of what is going to be required for safety and alignment, and how the industry and governments can work together,” he said.

Or in other words: Our investors will get a better return if we pause our IPO until confidence is higher, and regulatory capture is the way to get that.

Amodei, meanwhile, outlined some very specific things he wants from Washington: stop selling Nvidia chips to China so its AI companies can’t build better models, and a crackdown on model distillation.

REG AD

“If we execute these measures well, I believe they would slow China’s progress enough to widen America’s lead significantly over the next 3–5 years — the window when AI becomes geopolitically most important,” he wrote.

This plan would, of course, also mean that Chinese companies’ AI services would be inferior to Anthropic’s for a longer period – at a time when Chinese clouds are pushing into the rapidly growing middle eastern and southeast Asian markets.

Pacing the frontier landed badly in Washington. Speaker Mike Johnson warned Amodei’s plan could “smother innovation” and see China dominate global AI. President Trump dismissed warnings that AI can have deleterious effects and said the USA must lead the world in AI.

Such responses may reflect the fact that vast spending on AI infrastructure is delivering GDP growth in otherwise stagnant economies. Or perhaps they prove, yet again, that lawmakers are nearly always late to understand technology companies’ ambitions and the means they use to achieve them. ®