World on cusp of autonomous weapons revolution

Lethal autonomous weapons powered by artificial intelligence (AI) are set to revolutionize warfare, but there is no proper regulatory or legal framework governing the new technology.

A conference starting tomorrow at the UN in Geneva would discuss how to establish curbs and prohibitions on such weapons, called killer robots by some.

However, some analysts doubt the discussions at the Convention on Certain Conventional Weapons will keep pace with the fast-changing weapons.

Rescuers on Friday work at the site of a hospital in Kyiv, which was damaged in a Russian drone strike.

Photo: Reuters

Lethal autonomous weapons systems (LAWS) “can identify, select, and engage a target without the intervention of a human operator in the execution of these tasks,” Geneva Centre for Security Policy global and emerging risks head Jean-Marc Rickli said.

The critical issue for potential regulators lies in determining the level of human control over the decision to employ such a weapon.

The US refers to an “appropriate levels of human judgement” in the use of autonomous weapons. France maintains that a human must remain in the decisionmaking loop and bans LAWS, but permits “partially autonomous” systems known as PLAWS.

“The human role is always ensured through what is termed ‘appropriate control’ by a military commander — a concept that varies in scope depending on the system’s criticality and the operational context,” said Rear Admiral Vincent Sebastien, deputy director of the French Ministry of Defense’s Agency for Defence Artificial Intelligence.

“We are now dangerously close to crossing a moral red line: the autonomous targeting of humans by machines,” UN Secretary-General Antonio Guterres and International Committee of the Red Cross president Mirjana Spoljaric said in a joint statement in August.

Analysts said the Geneva discussions, while important, will touch only part of the issue.

Today, the diminishing role of humans is being driven by the growing use of AI in analysis and targeting.

The scope of the agenda in Geneva is “very narrow,” Rickli said, adding: “It fails to encompass the full range of issues surrounding AI on the battlefield.”

This was seen in the US-Israeli war against Iran this year “where the Americans engaged as many as 1,000 targets per day” inside Iran, he added.

“There are a host of problems and unacceptable risks associated with targeting and decision-support systems,” Laure de Roucy-Rochegonde of the Centre for Geopolitics of Technology at the French Institute of International Relations said. “What kind of human supervision is required to ensure that the operator is genuinely exercising their own judgement rather than simply providing a rubber stamp for the machine’s recommendations?”

The autonomous weapons to be discussed in Geneva are not yet in widespread use, although analysts said they have been deployed by Russia in its invasion of Ukraine as well as US attacks in Iran.

However, the spread is inevitable.

“Autonomy will very quickly become a reality on the battlefield,” Sebastien said.

“Autonomous weapon systems are not part of a distant future — a race toward greater autonomy in weapons systems is already well underway,” Guterres and Spoljaric said in their statement.

Significant doubts and dangers remain, Anthropic cofounder Dario Amodei said.

“Today, frontier AI systems are simply not reliable enough to power fully autonomous weapons,” he added.

Ultimately, the incentive for major powers is too strong.

Swarms of autonomous drones — which “aim to neutralize an adversary through saturation” — would be highly effective, and some states view them as a crucial strategic advantage, Rickli said.