US senators weigh requiring AI giants to commit to preventing catastrophe

WASHINGTON, Sept 14 : U.S. Senate negotiators are debating legislation that would require AI companies to demonstrate they are taking reasonable precautions to keep their tools from causing harm, according to a Senate aide and a lobbyist familiar with the talks. 

Alarm about the potential harm from AI grew last week when Anthropic researcher Jacob Coxon said he had resigned, in part because the "people building AI earnestly believe that it could kill us all by the end of the decade." AI-related stocks fell worldwide on Monday after leaders of some of the industry's largest companies called for slowing the pace of the technology's development.

The senators are debating giving the U.S. commerce secretary the power to request evidence that AI developers are taking reasonable steps to prevent harm, known as a “duty of care," according to the Senate aide and lobbyist. The secretary would also have the power to deploy government auditors to test companies' products themselves, they said. Reuters could not immediately determine what would qualify as “reasonable.”

The legislation is still being debated. Even if Congress approves the measure, it faces long odds of becoming law. U.S. President Donald Trump said on Monday that existing authorities already provide adequate tools to regulate and prosecute tech companies, suggesting he wouldn't sign a bill setting new rules for AI. 

The proposal is being discussed by Senate Majority Leader John Thune, Senate Commerce Committee Chairman Ted Cruz and Senator Amy Klobuchar, the top Democrat involved in the negotiations. Spokespeople for Thune and Cruz did not immediately respond to requests for comment from Reuters.

"I’m continuing to work toward a bipartisan agreement on legislation for government oversight of the greatest risks posed by AI models," Klobuchar said in a statement. "That includes requiring developers to work with government experts to verify and test models to make sure AI is safe."