Don’t hold your breath for AI agreements or arms control

Ours is an era of great power competition. The United States acknowledges as much in its official national security documents; China concedes that is the tenor of the times but insists that the U.S. is responsible for this state of affairs. Japan’s National Security Strategy simply has repeated references to “intensifying geopolitical competition.”

That competition should not — must not — block cooperation on crucial issues. Even during the height of the Cold War, the U.S. and the Soviet Union worked together to reduce the threat posed by nuclear weapons, signing arms control agreements and cooperating with other nations to block the proliferation of weapons of mass destruction.

The “crucial issues” that supersede narrow calculations of national interest tend to be somewhat arbitrary. WMD still makes the list. Climate change is getting there. Last week, artificial intelligence rose to prominence following warnings by prominent officials and experts that the risk of catastrophic damage from AI was nontrivial.

There has been a cascade of incidents in which AI have gone rogue, cajoling, flattering or in some cases blackmailing human operators to attain some form of agency. After reports this summer that models from leading AI companies escaped their testing environments to hack real computer systems, Jacob Coxon, a senior researcher at Anthropic, one of the world’s leading AI developers, quit his job and warned on social media “Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight and acquire real power and resources.”

Evan Hubinger, a top safety researcher at Anthropic, agreed, concluding that there is a greater than 10% chance that AI “could kill all humans” within the next decade. Those developments prompted Dario Amodei, head of Anthropic, to publish an essay last week that called for a slowdown in the development of AI. Leaders of other leading AI companies, Sam Altman, chief executive of OpenAI, Elon Musk, head of SpaceX and Demis Hassabis, chair of Google DeepMind, all echoed his position. In the essay I mentioned last week, Bill Gates said that addressing risks from AI should be “the world’s top priority.”

AI is expected to be on the agenda of the Washington summit that will be held later this month between U.S. President Donald Trump and Chinese leader Xi Jinping. The men may take up the topic, but don’t expect restrictions. Trump doesn’t favor a cautious approach. On Sunday, he said that “We’re leading China in AI, we’re the most sophisticated country in the world, and frankly I want to keep it that way,” adding, “Whoever wins AI wins.” He later called the dangers posed by the technology a “HOAX.”

In contrast to Trump’s nonchalance, China has been calling for international AI governance, arguing primarily for ensuring access to AI for all countries, hammering home a key point of its international diplomacy: China represents the interests of developing nations, the Global South in particular.

When it comes to AI safety, the Chinese line is skeptical, but for different reasons. Writing in the ChinaTalk substack earlier this month, Irene Zhang explained that “Beijing perceives the AI safety discourse as overly captured by forces like Anthropic, which in turn serves the interests of Washington. As a result, China will not accept a definition of AI safety wholly borrowed from Western thinkers; in fact, in its eyes, those thinkers are implicated in ‘American disease.’”

She turns to Global Times to make the case. The nationalist tabloid cited Xiang Ligang, a veteran telecom and tech policy observer, who argued that Amodei’s position reflects that of U.S. officials “who believe that China’s AI development should be contained to preserve the U.S.’ technological lead, with groundless claims about China’s so-called AI threat being repeatedly hyped.”

Amodei isn’t being misquoted. He wrote that “Chinese lead in AI would pose grave danger for the U.S. and the world.” Global Times counted 12 mentions of China in the 3,800-word essay, along with a call for steps to “slow China’s progress enough to widen America’s lead significantly over the next 3-5 years — the window when AI becomes geopolitically most important.” The measures include restrictions on sales of advanced AI chips and manufacturing equipment, restrictions on unauthorized model distillation and stronger internal security to prevent model weights from being stolen.

This is familiar ground. China has resisted U.S. entreaties to join nuclear arms talks, insisting that its arsenal is far smaller than that of the U.S. and Russia and will only join negotiations when they reduce their stockpiles to its level. Consistent with the logic, Global Times calls Amodei’s article “a Cold War script.”

That is the age-old dilemma of arms control (which is what this is). Countries leading a competition don’t like constraints unless they lock in their advantage. Countries that are behind reject them for the same reason. They favor measures that level the playing field. The profound belief in China that second-class status exposes the country to blackmail, coercion and manipulation stiffens the nation’s collective spine to resist any initiative that blocks its rise.

Bill Bishop, longtime China watcher and publisher of the inestimable Sinocism newsletter, concludes that the two countries “are in what political leaders on both sides increasingly see as an existential struggle, not so much for humanity but for their own countries. I do not see how they bridge that gap, even if they wanted to.”

For there to be a success, all must recognize that it is the competition that is dangerous, not the standings.

A prod is needed. In conversations with Chinese interlocutors, there is an almost wistful longing for a crisis to jumpstart negotiations. It may have arrived a few weeks ago in an announcement by a California research firm that it had hacked the WeChat app.

The exploit, which it called WeWorm, hijacked a WeChat user’s account and spread to their contacts without even answering a call. It was developed in less than 48 hours with the help of AI. Ryan Fedasiuk, a tech researcher at the American Enterprise Institute, on social media called it “cause for concern, a shadow brokers for everyone.” (Shadow Brokers was a hacker group that put zero-day exploits it had allegedly obtained from the U.S. National Security Agency on the web, available for all.)

WeChat has an estimated 1.4 billion users and is much more than a messaging app. It is used for communications, financial transactions and file sharing. It is considered an integral part of daily life. Zhang observed that “A successful attack on WeChat would paralyze the entire country.” In its reporting on the hack, the New York Times cited Zhao Minghao, deputy director of the Center for American Studies at Fudan University, called WeWorm “a new kind of nuclear weapon.”

Kyle Chan, a Brookings Institution fellow who focuses on Chinese tech, thinks this could be the motivation that spurs China to reconsider cooperation. He told the NYT, “It hints at the massive potential for other actors, nation-states or nonstate actors, to use AI to attack China’s digital infrastructure.”

My guess is that if Xi pushes hard enough, Trump will follow his lead. Getting a meaningful agreement is another matter. The biggest issue will be — as always — verification. Trust between the two countries is nonexistent, to say nothing of that between regulators and the regulated (at least in the U.S.) Nuclear weapons verification entails physical items that can be observed and counted; software is just digital manipulation. Moreover, the work is done on infrastructure that can be used for a variety of purposes, some subject to regulation, but most of which are not.

A first step is likely some agreement on principles to govern AI use and then unilateral measures that governments and companies can take to make them work. Only when there is evidence of compliance with those initiatives and continuing threats posed by noncompliance will there be a basis for collective action. Let’s hope it isn’t too late.

Brad Glosserman, a senior adviser at Pacific Forum, is the author of “Peak Japan.” His latest book, written with David Lee, “Tech-Gemony,” on the geopolitics of high-tech, is out now from Hurst Publishers.