Self-replicating AI: Is your fridge a cybersecurity risk?

In the living room, a robot vacuum cleaner moves back and forth, the lights are dimmed by voice assistants, the heating is controlled via smartphone and the TV connects to the  internet to access a favorite streaming service. These are all examples of the "Internet of Things," in which connected electronic devices exchange data and communicate with one another.

According to recent market research by industry association Bitkom, about half of all households in Germany use such or similar "smart home" applications — and the figure is rising.

Now, the artificial intelligence company Anthropic has published a report warning that AI-powered malware could soon infiltrate connected household devices, copy itself to other devices and become active there.

Computer viruses that copy themselves from one machine to another have existed for decades. They are also known as worms and follow a predefined sequence of instructions. They exploit known security vulnerabilities or poorly protected access points. However, once the vulnerabilities are patched up or suitable attack vectors are no longer available, a worm can no longer spread so easily.

AI worms, however, would be far more complex and potentially much more harmful. They could solve problems autonomously and adapt to different devices, for example by coping with a different operating system or limited memory. They might also figure out for themselves how to operate on other devices or identify and circumvent new security updates

What is AI's biggest risk: Speed or autonomy?

Spreading via TVs

Anthropic researchers investigated whether AI models could also run on relatively inexpensive hardware. In an experiment, they distributed an AI model across 15 low-cost Android TV boxes, with each device handling part of the computation. This demonstrated that certain AI applications can run without a large data center.

In a controlled test environment, the researchers set up eight additional simulated Android devices. The AI model was able to infiltrate one of them, replicate itself and infect other devices. In four out of five attempts, it reached all eight target devices. In the remaining attempt, it reached seven of the eight.

If equipped with malware, such an AI model could potentially adapt relatively quickly to previously unknown environments and disrupt processes and operations. It could learn to circumvent various security measures rather than simply exploiting a single vulnerability. According to Anthropic's developers, this sort of "AI worm" could take roughly one to two days to establish itself on a new, previously unknown device.

A botnet of household devices?

One particularly worrying possibility is that AI could link the infected devices into a large botnet. A single infected device has limited computing power, but the more devices are compromised, the faster the AI worm could spread. The malware would no longer depend on a central server that could simply be shut down.

It might also continuously learn as it spreads. It is conceivable that it could jump to entirely different types of electronic devices, all running different software.

Infected devices could be disconnected from the network and security vulnerabilities could still be patched. The malware's decentralized structure, however, would make it difficult to contain. Anthropic itself compares this potential development to a digital slime mold slowly spreading.

Researchers at Anthropic deliberately made access to the test devices very easy and left out major digital safeguards. Even so, the company believes smaller AI models could reach the necessary level of capability later this year to spread in the real world.

Overheating cooling systems

The potential consequences of cyberattacks on connected control systems has already been seen in an incident in France. Overnight into Monday, attackers tampered with refrigeration and freezer systems at numerous supermarkets. Temperatures in the affected units rose to as high as 50 degrees Celsius. Food spoiled, and some incidents also involved fires and technical damage.

France's cybersecurity agency, CERT-FR, initially confirmed that remote-control systems made by Danish manufacturer Danfoss had been compromised. The warning was later extended to systems made by other manufacturers. Investigations into the circumstances of the attack are ongoing.

Modern supermarkets use digital controllers to monitor and regulate refrigeration systems. Maintenance companies can access these systems remotely, for example to check temperatures. However, some of the security software was reportedly outdated, which the attackers may have exploited as an entry point for their cyberattack.

A new dimension in cyberattacks?

It remains unclear who was behind the attacks in France and what motivated them. There is also no evidence so far that a self-replicating AI worm was used in the attacks on French refrigeration systems. However, Anthropic's latest research shows how significantly AI could facilitate the development of malware, potentially enabling attacks on connected devices with serious consequences and without requiring much computing power.

Fearing further attacks, France's cybersecurity agency advised all companies to update their connected devices regularly, install essential security patches and, where possible, to disable unnecessary internet access altogether.

This article was originally published in German.

Will AI destroy humanity?