Autonomous drones and the future of war in an AI-driven world | Letter

Stuart Russell is right that AI’s builders fear losing control of it (Experts are warning: our AI arms race is putting humanity at risk, 11 August). But it is not only a future superintelligence’s problem; it is already loaded into weapons, where a loss of control kills a civilian today, not humanity tomorrow.

An autonomous drone told to clear a “kill box” is the genie your other columnists described (How do we prevent AI agents from going rogue? It starts with a new kind of measurement, 18 July): it does what it is told, killing whatever is inside, combatant or child. The gap between a target and a lawful target is the principle of distinction, which no instruction closes, because the instruction was lawful.

Russell doubts asking a superintelligence how to control itself. In war the check need not be another opaque mind but a deterministic one: its rulebook the Geneva conventions, its judgments auditable, returned to a human. We need not trust it; we can check it. Some of us are working on it.

A further danger is peculiar to war. Autonomy is sold as combat without our own dead, and a bloodless war is easier to start. But that is a fallacy: you can take the human from the targeting loop, not the nation from the war. Ground must be won and held; the dead still come, in attack or defence. Armies no longer go to war; nations do.

If we cannot teach a drone to keep the laws of war, we will not teach a greater intelligence to keep faith with all we value. To abandon those laws for survival is the confusion of interests with values that our strategists warn against; here the two are one. Machines that kill without distinction, answerable to no one, leave no one safe. Our interests do not merely permit us to hold the line; they demand it.
Dr Darren Mann
Military surgeon, London