News24 | Senators debate forcing AI firms to prove safety as Trump balks

US President Donald Trump holds up a signed executive order in the Oval Office of the White House on 23 January 2025 in Washington, DC. Trump signed a range of executive orders pertaining to issues including Artificial Intelligence, cryptocurrency and clemency for anti-abortion activists.

Anna Moneymaker/Getty Images

• US Senate negotiators are debating legislation that would require AI companies to prove they are taking reasonable precautions against harm, including a “duty of care” enforceable by the commerce secretary.
• The push follows growing alarm after an Anthropic researcher resigned saying AI builders “earnestly believe that it could kill us all by the end of the decade,” sending AI stocks tumbling worldwide.
• Even if Congress passes the bill, President Trump indicated he would not sign it, saying existing authorities already provide adequate tools to regulate tech companies.

US Senate ‌negotiators are debating legislation that would require AI companies to demonstrate they are taking reasonable precautions to keep ‌their tools from causing harm, according to a Senate aide and a lobbyist familiar with the talks.

Alarm about the potential harm from AI grew last week when Anthropic researcher Jacob Coxon said he had resigned, in part because the “people building AI earnestly believe that it could ‌kill us all by the end of the decade.” AI-related stocks fell worldwide on Monday after leaders of some of the industry’s largest companies called ‌for slowing the pace of the technology’s development.

The senators are debating giving the US commerce secretary the power to request evidence that AI developers are taking reasonable steps to prevent harm, known as a “duty of care,“ according ‌to the Senate aide and lobbyist. The secretary would also have the power to deploy government auditors to test companies’ products themselves, they said. Reuters could not immediately determine what would qualify as “reasonable.”

The legislation is still being debated. Even if Congress approves the measure, it faces long odds of becoming law. US President Donald Trump said on Monday that existing authorities already provide adequate tools to regulate and prosecute tech companies, suggesting he wouldn’t sign a bill setting new rules for AI.

READ | Trump slams AI regulation as ‘sick conspiracy’, says US must beat China

The proposal is being discussed by Senate Majority Leader John Thune, Senate Commerce Committee Chairman Ted Cruz and Senator Amy Klobuchar, the top Democrat involved in the negotiations. Spokespeople for Thune and Cruz did not immediately respond to requests for comment from Reuters.

“I’m continuing to work toward a bipartisan agreement on legislation for government oversight of the greatest risks posed by AI models,” Klobuchar said in ‌a statement. “That includes requiring ‌developers to work with government experts to verify and test models to make sure AI is safe.”