When the attacker becomes the user: Rethinking account takeover

In a world full of cyber risk, account takeover has emerged as one of the most insidious of all threats. If an attacker can successfully compromise a legitimate email account, they are then able to pose as a trusted individual or organization without triggering any warnings. Traditional email security will see nothing wrong. From there the threat actor can pick their moment to cause havoc with objectives that range from stealing money, expropriating sensitive data and propagating catastrophic malware. Serious financial loss and a damaged corporate reputation are commonplace outcomes.

The typical account takeover lifecycle starts with the bad guys obtaining credentials, usually through the medium of phishing. Generative AI (GenAI) has made it easy for an attacker to trick an unwary individual into giving away login data, perhaps by posing as a colleague or other trusted party or by encouraging them to enter confidential information on a fake login page.

You might expect, once a legitimate identity has been successfully compromised, that all hell would immediately break loose. These days the typical account takeover artist is more stealthy and sinister than that, in no rush to act at this early stage since no alarm bells have been triggered. Most will take their time to test out and verify their stolen login data, and see where it might take them.

“Conventionally attackers would take over an account and immediately start firing off spam and phishing,” says Dave Baggett, SVP Cybersecurity with software developer Kaseya. “Now they are starting to get clever. They’ll get access to an account and sit tight, watching all the email traffic passing and waiting for an opportune moment to inject themselves into the conversation.”

From the inside out

Once they’ve figured out an organisation’s weak points and come up with a plan of attack, the bad guys will start to weaponize the compromised email account to target other employees and internal systems. Assuming their motivation is financial, they might be on the hunt for a way into payroll or invoicing databases. But their ultimate aim might lie further afield in the form of the organisation’s customers and suppliers, and that’s when the real nightmare can start. So called outbound phishing offers the criminal a way to assert control over a whole new range of sensitive end points.

Victims of outbound phishing are highly vulnerable in the face of what seem to be messages from a trusted partner and ally. It is one thing for an organisation to protect its own email accounts with robust controls like multifactor authentication (MFA), but no one has control over the authentication practices of other parties: “Let’s say we have a customer who we’re protecting,” explains Baggett. “They’re safe, but imagine that they start getting suspicious emails from a third party. Our customer has no idea what is or isn’t protecting that third party. Emails coming from that source might appear to be from a real account, but it could be a hacker typing them.”

Given that you can’t run checks on third party organisations, something more sophisticated than regular email security is required. Ideally this would be a GenAI-powered solution that can bring language understanding to bear, able to examine conversation histories in depth to see if something anomalous can be spotted.

It makes life even harder for defenders when the bad actor has successfully assumed the identity of somebody senior. If they are impersonating a top-tier manager with an extensive and trusted reputation then they have a powerful asset on their hands. All the trust and goodwill of that senior individual is now inherited by the criminal.

Conventional email filtering systems will often be configured to allow senior personnel past certain defensive layers with minimal friction, so as not to restrict important business communications. Recipients of mail from such sources may be disinclined to challenge and report them, should suspicions be raised by the nature of what they are asking for. It takes a bold employee to call out as fraud what seems to be a message from somebody at C-level, especially when the email system has apparently green lighted it as authentic.   

 Bringing the power of AI to bear

The heat is on not just for IT departments but for MSPs who have multiple organisations under their protection, many of which will have no real in-house security knowledge of their own. Kaseya’s own research shows just how exposed many are. New approaches are needed that deal with the menace of account takeover before damage can be caused. And should a takeover transpire, consideration must be given to containing its impact before vital data and infrastructure are affected.

The good news is that modern tools exist that perform just these tasks. INKY is Kaseya's generative AI-driven email analysis tool. It has capabilities that go well beyond any kind of traditional email security, tuned to detect, through the most subtle and marginal signs, when an email conversation is in the process of being infiltrated.

Using techniques such as advanced sender profiling, stylometry and machine learning, INKY is adept at identifying inconsistencies between the known writing patterns of a legitimate employee and the slightly unfamiliar tone of a fake message. It can sniff out that something is wrong, even when message headers seem real, just by spotting the kind of contextual anomalies that expose a phoney message.

INKY is especially useful when it comes to third party account takeover. It achieves this by building and maintaining detailed profiles of trusted senders based on technical and behavioural characteristics. Over time it learns what “normal” looks like and can see when a third-party email doesn’t match a sender’s established profile.

Containing an attack

If a takeover attempt has been successful, there’s more to stemming the damage than just putting a block on a suspect account. By the time of its discovery, a compromised identity may have already led to inroads being made into connected SaaS apps, cloud storage and collaboration tools, widening the possible blast zone well beyond a single email thread.

Modern IT environments are a jungle of identity-driven endpoints and gateways, and therefore increasingly difficult to defend against intruders armed with legitimate credentials. To compound matters, when the bad guys have AI at their disposal they are able to treat these SaaS-driven ecosystems as a single attack surface. Defence that relies on a patchwork of individual tools, each created to solve a particular challenge in one silo of the organisation, will not be up to countering this kind of threat.

That’s why INKY is configured to work with Kaseya SaaS Alerts, a cloud-based detection and response platform designed to help MSPs and others to monitor and protect business applications. Once INKY has picked up that a mailbox has been hijacked, it isolates the user and blocks outbound mail. It then sends an event to SaaS Alerts so that it can lock down that user's access across the rest of the stack. The tools work as a team, INKY blocking the compromised account and SaaS Alerts preventing the damage from spreading.

The power of this partnership comes from INKY’s ability to stream real-time email security telemetry into SaaS Alerts, including inbound analysis results and threat detections, user-reported phishing activity and outbound mail protection events. This type of integration helps to automate real-time threat detection across a complex SaaS ecosystem.

“It means if we see suspicious mail going out, we can shut the account down, and take other remedial actions,” says Baggett. “The more you understand the meaning of the mail, the more you can catch.”

Smart tools to the rescue

The battle against account takeover is never won, and future innovation is always needed. We’ve seen how AI is a dangerous weapon in the hands of the attacker. It’s also a very handy tool for the defender – but there are some qualifications: “You can’t just put every inbound and outbound mail through an LLM since the economics don’t add up,” points out Baggett. “You’ve got to be more clever than that.”

One of the most important and cleverest emerging uses for GenAI lies in not simply protecting against account takeover but predicting where weakness might lie, helping to stop a problem before it happens. Baggett believes that the security industry is at the very beginning of being able to use models to predict in this way. An essential preliminary is making sure the many different components that go into securing an organisation can share information with each other.

That integration work is underway, an early example being the real time dialogue between INKY Mail Protection System and SaaS Alerts. Next step is a unified data layer where any security component can publish an event in real time, and any other component can subscribe.

“We’ll continue to make more integrations of this sort over time,” promises Baggett. “We’re also building a centralised capability for AI. With so many of the different components having their own AI, you need an underpinning platform. We also have a service bureau of Machine Learning experts who can make customised models for different tasks. There’s no sleeping on the job. The volume of attacks is clearly going to increase.”