Decoys, safe AI hacking help cyberdefenders counter AI-powered attacks

SINGAPORE – Artificial intelligence may be helping hackers speed up cyberattacks from days and weeks to mere minutes and hours. But, for now, the “gullibility” of AI models makes them uniquely vulnerable to deception – a growing strategy among cyberdefenders.

“If you put well-placed deception and you build the processes to react and contain the attack, you can render AI attackers completely obsolete,” said cybersecurity firm Horizon3’s chief executive, Snehal Antani, at a recent October media briefing.

He explained that the AI systems from many leading technology companies are trained on public cybersecurity data, such as vulnerability reports and research. This data is not as useful as the internal data from organisations, such as network configuration information.

Public data-trained AI tends to open decoy files, such as fake log-in details, because that is what it knows and remembers from its training data. When it does that, an alert notifies organisations so they can take action early before an attack reaches more important systems.

Research by Horizon3 published in June showed that 21 AI models, including from Anthropic and OpenAI, fell for decoys 78.5% of the time compared with 37% for human hackers. Some AI models, including cutting-edge frontier models, fell for the bait over 90% of the time.

“Well-placed deception is the cheapest, fastest, most effective way at catching AI hackers,” Antani said, adding that he expects the gullibility of AI models will become the hottest area of cybersecurity research in the next 12 to 24 months.

But a capable attacker may still recognise the deception, he said, urging organisations to also have other controls such as patching and system access limits.

Experts also noted that AI is not equally effective at every stage of an attack, giving cyberdefenders room to counter the threat.

Cybersecurity firm Ensign InfoSecurity tested 10 frontier AI models, including from OpenAI and Anthropic. The results of the study, released in August, found that all 10 AI models initially broke into the test system, which Ensign said was a “trivial task”.

But the models were less reliable at later stages of an attack. At least half of the models struggled to steal log-in details and move between systems consistently.

The limits of AI-assisted cyberattacks are a silver lining amid accelerated attacks.

Jess Ng, cybersecurity firm Fortinet’s country head for Singapore and Brunei, said that organisations have less time to learn about and fix a system weakness before criminals exploit it.

Unprecedented strain on cyberdefenders

The recently disclosed critical flaws in PHP, a programming language that powers roughly 70% of all websites globally, provides a wake-up call. The flaws were discovered by cybersecurity firms with the help of AI, and after their disclosure on Oct 21, web security firm Cloudflare observed hacking attempts surge into the millions.

“Organisations cannot assume they will have weeks to respond after a critical vulnerability becomes public,” said Gareth Cox, Horizon3’s vice-president for Asia Pacific and Japan.

While it is not clear if the PHP exploit attempts used AI, there are other cases of AI speeding up attacks. An Oct 1 global report from tech giant Microsoft said that by using AI, the time hackers take to steal data, discover log-in details and infiltrate deeper into organisations’ computer systems has been shortened from days to minutes.

Cybersecurity firm Palo Alto Networks said on Sept 2 that it had responded to an incident where a human attacker used frontier AI to breach an enterprise network autonomously in under 10 hours. The attack would have taken several human attackers around two weeks to execute.

Using AI, criminals can translate technical details released on security flaws into simpler terms. “Work that once needed an experienced expert can now be done in a fraction of the time, and by people with far less technical knowledge,” said Fortinet’s Ng.

For example, AI has allowed cybercriminals with limited programming know-how to create a platform, called EvilTokens, to charge other hackers a fee to develop e-mail phishing strategies. At just six months old, that platform had impacted 10,000 organisations in 120 countries, including Singapore, before Microsoft and other partners disrupted it in September.

Mike Yeh, Microsoft’s vice-president and deputy general counsel for customer security and trust, said: “AI has... lowered the barriers to entry and made it easier for cybercriminals to operate.”

Hackers can also use AI agents to autonomously stitch together several less severe bugs to create a major attack, said Siddharth Deshpande, Palo Alto Networks’ director for industry solutions.

This is an issue because organisations tend to prioritise fixing more serious flaws over less severe ones, said Deshpande, adding that less severe vulnerabilities also need to be taken seriously.

A new Frontier AI Resilience and Security Taskforce announced by the Infocomm Media Development Authority on Oct 14 seeks to look into these issues.

The high speed of attacks is putting unprecedented strain on security professionals.

Said Philippa Cogswell, vice-president and managing partner of Unit 42 at Palo Alto Networks in Asia-Pacific and Japan: “Forced to make critical decisions under extreme time pressure and with incomplete forensic data, security professionals face record levels of burnout and decision paralysis as they scramble to establish what occurred, what was accessed, what was stolen, and what the overall impact is to the organisation and the supply chain.”

Fighting back with AI

Association of Information Security Professionals president Emil Tan said cyberdefence must now be more proactive. Organisations should anticipate how they might be attacked and take steps to prevent or disrupt it, he said, urging the use of AI and automation for defence.

Horizon3 uses AI to safely test the cyberdefences of an organisation by autonomously hacking to find vulnerabilities before hackers do and verifying fixes for those vulnerabilities. One of its customers is a Singapore-based hospitality organisation with 1,000 employees.

Accelerating demand for such tools has underscored investment interest. Horizon3 said in August that it had raised US$250 million (S$320 million). Among its investors is EDBI, the investment arm of Singapore’s Economic Development Board.

The funding will help the company turn its Singapore office, which began operations in August, into its ASEAN headquarters. This is to meet growing demand from South-east Asian organisations to defend against cyberattacks increasingly augmented by AI.

Horizon3’s Antani is already looking ahead: Fixing vulnerabilities automatically at machine speed is the next frontier his firm is working on.