OpenAI Chief Sam Altman Shares 2 Ways AI Progress Could Go Very Badly

OpenAI CEO Sam Altman has flagged two ways in which the progress of artificial intelligence "could go very badly" amid the larger ongoing debate.

• What happens if humans lose control of increasingly powerful artificial intelligence?
• And what happens if that power ends up concentrated in the hands of one company, one person or even one country?

Altman addressed both concerns in two posts on X as the debate over the pace of AI development intensifies. His comments come after Anthropic CEO Dario Amodei called for advanced AI development to be slowed down, a position that Altman, Elon Musk and other technology leaders have backed. 

Altman said there are "two ways AI progress could go very badly and that we must avoid."

The first is the possibility that AI could become too powerful for humans to control. "First, we could lose control of the future to AI. This is unacceptable; we are unapologetically on Team Humanity, and AI must always serve people," Altman wrote.

The second danger, according to him, is concentration of power. "If an extraordinarily powerful AI is used by one person or company to impress their worldview onto everyone else, the results could be extremely dystopian," he said.

Altman also pointed to the possibility of one country gaining too much power through AI, or a single AI lab becoming disproportionately powerful. His answer is what he describes as a "narrow middle path".

However, that does not mean stopping AI development. Instead, Altman wants the industry to move more carefully as AI systems become increasingly capable. "When we talk about 'pacing', we do not mean 'stopping'. Progress has been rapid and will continue to be," he said.

But there is a catch. Altman acknowledged that making AI development safer will also make it slower and more expensive. He said safety measures such as advanced monitoring and "safety cases" come with significant costs. Still, he argued that these costs are justified.

"Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring," Altman wrote.

Altman Wants Common AI Safety Rules

Altman's comments come as AI companies face growing pressure to show how they plan to control increasingly capable systems. He said OpenAI supports a federal framework with consistent safety requirements for frontier AI.

But he argued that companies should not wait for new legislation before taking action. "We do not believe we need to wait for an anti-trust exemption or legislation to begin the work of providing this confidence," Altman said.

One idea he specifically welcomed was the use of independent auditors. The proposal is significant because independent evaluators are also at the heart of Amodei's plan to "pace the frontier". Anthropic has argued that external safety evaluators should have deep access to AI companies so they can assess whether safety commitments are actually being followed. Altman has backed the idea for OpenAI as well. 

Altman also said OpenAI is changing how it evaluates its most advanced AI systems. In the past, safety frameworks largely focused on models before they were released. Now, he said, the company is putting greater emphasis on what happens during development.

"At OpenAI we now formulate explicit safety cases in advance of frontier reinforcement learning runs we expect to significantly increase capability," Altman wrote. He wants other AI companies to develop their own approaches and eventually establish shared standards around issues such as misalignment, monitoring and safety.

AI Race Is Getting More Uncomfortable

The timing of Altman's comments is important.

The AI industry has been racing to build increasingly capable models, while researchers and executives are becoming more vocal about the risks. Amodei recently argued that the pace of development has become too fast to safely manage, while former Anthropic researcher Jacob Coxon warned after his resignation that AI could pose an extreme threat to humanity. 

Microsoft CEO Satya Nadella has also entered the debate. He said the pursuit of superintelligence should ultimately depend on whether AI remains useful to humanity and under human control. He has also backed independent evaluators and argued that AI power should not be concentrated among a handful of entities. 

That makes Altman's latest comments more than a warning about hypothetical risks. The leaders building some of the world's most powerful AI systems are now openly debating how fast the technology should move, who should monitor it and who should control the power it creates.

Altman summed up his position simply: AI should keep advancing, but safety must advance faster. "And first we should do what we can ourselves," he said, while calling for governments to help with international coordination.