AI in Cybersecurity

In mid-September 2025, the security team at the AI company Anthropic noticed a pattern of activity on its own platform that did not look like ordinary use. Over the next ten days it pieced together what was happening. A group that Anthropic assessed with high confidence to be Chinese and state-sponsored had broken a hacking campaign into small, innocent-looking tasks and handed them to an AI coding agent, telling it that it worked for a legitimate security firm. The agent probed about thirty organizations: technology companies, financial institutions, chemical manufacturers and government agencies. By Anthropic’s estimate the AI did 80 to 90 percent of the work, with humans stepping in at only four to six decision points per campaign. At its peak the agent made thousands of requests, often several per second. A small number of the intrusions succeeded¹.

The report contained a second detail that is easy to miss. The agent made mistakes. It sometimes claimed to have stolen credentials that did not work, and presented public information as if it were a secret find. Speed and scale had arrived; perfect judgment had not.

Those two facts describe AI in cybersecurity on both sides of the contest. Attackers are not waiting for the technology to mature. They are already using it to do what they have always done, faster and on more targets at once. Defenders can use it to do the same: to read more signals, connect them sooner and act within minutes rather than days. Neither side has a machine that is always right, so for an executive the question is where AI shortens the path from signal to sound decision, and where it must not be allowed to act alone.

Better decisions, not more alarms

Security leaders have a common vocabulary for their work, and one of the most widely used is the US National Institute of Standards and Technology’s Cybersecurity Framework. Its 2024 revision organizes security into six functions. Five of them form an operating loop: identify what you have and what threatens it, protect it, detect when something goes wrong, respond, and recover. The sixth, govern, was added in that revision and sits above the others: it sets the risk appetite, the roles and the rules².

AI can help at every point on that loop. It can keep the inventory of systems current, rank weaknesses by how likely they are to be used, sift telemetry for the few events that matter, draft the incident timeline and recommend containment. These are capabilities; the outcome evidence later in this chapter comes from a breach-cost survey and a competition, not from controlled tests in production. It can also hurt at two points: by suppressing a signal that mattered at detection, or by taking a heavy action on a false alarm at response. The useful executive question is therefore narrower than “where can we use AI in security?” It is: which step in our loop is the bottleneck, and what may AI do there without a person?

Attackers use AI mostly to go faster

The evidence on attackers is now broad enough to describe with some confidence. IBM’s 2026 study of 602 breached organizations found that one in four malicious breaches was AI-enabled, through means such as deepfake impersonation and AI-assisted malware, and that those breaches cost an average of 6 million dollars, about a million more than the global average of 4.99 million across all breaches³.

What attackers do with AI matters as much as how often. Verizon’s 2026 breach report, which drew in part on data from AI providers’ misuse investigations, found that the median threat actor used AI assistance in fifteen different documented techniques, from choosing targets to writing malware. Yet fewer than 2.5 percent of AI-assisted malware observations involved techniques that were not already well known⁴. AI is mostly a force multiplier for familiar attacks, not a source of new ones.

Fraud data points the same way, with a warning about measurement. The FBI’s Internet Crime Complaint Center received more than 22,000 complaints in 2025 that reported AI-related information, with losses of 893 million dollars. Business email compromise, the impersonation of executives and suppliers to redirect payments, cost 3.05 billion dollars in total, but only about 30 million of that was tagged as involving AI. The two figures count different things: the 893 million covers complaints of any kind that mentioned AI, while the 3.05 billion covers all business email compromise, whether or not AI was noticed. The FBI’s own reading is that many victims do not realize how far AI was involved⁵. The safest assumption is that a convincing email, voice or face may now be synthetic.

That conclusion has a reassuring side. If attackers mostly use AI to run known plays faster, the known defenses still work, provided they are actually in place. How attackers turn an organization’s own AI assistants against it, through hidden instructions and poisoned content, is a different problem, and it belongs to Security and AI Attacks in Module 06⁶⁷.

An anomaly is a signal, not a verdict

Much defensive security AI starts with anomaly detection. The system learns what normal looks like for each user, device and service, then flags what deviates: a login from an unusual place, a burst of file access, a server talking to an address it has never contacted. That is useful, and it is also where many programs go wrong, because an anomaly is only a deviation. It is not proof of an attack.

Consider, as an illustration, a single flag: an employee downloads an unusually large volume of files late in the evening. It could be data theft before a resignation. It could equally be a planner pulling a year of records for an audit due the next morning. The anomaly is identical; the meaning is opposite. What separates the two is context, and correlation is where AI earns its keep. One unusual login means little. An unusual login, followed within the hour by a privilege change, a new device and a large download, tells a story no single alert can. Systems that join those dots across identity, email, endpoints, network and cloud turn thousands of weak signals into a short list of incidents worth a person’s attention.

Two kinds of error follow, and an executive should ask about both. A false positive treats normal work as an attack: a locked account, a blocked shipment, an angry customer. A false negative misses a real attack, or quietly drops the alert that would have revealed it. Dashboards tend to show the first because it is visible. The second is the one that ends up in the incident report.

Insider-risk monitoring sharpens the point, because the signals are about people. Watching employees’ behavior closely enough to spot data theft is processing of personal data that usually needs a formal impact assessment in Europe, and often the agreement of a works council, before it starts. The legal anchors appear in the box on monitoring and reporting duties below.

Fix what attackers will actually use

The same logic, ranking by meaning rather than by volume, applies to the weaknesses in an organization’s own systems. In 2025, 48,185 new software vulnerabilities were published, about 132 a day and a fifth more than the year before⁸. No team patches that many. Yet only about 6 percent of published vulnerabilities have ever been seen exploited in the wild, and ranking by the predicted likelihood of exploitation achieves the same risk coverage as fixing every flaw with a high generic severity score, for about one-sixth of the effort⁹.

The order matters more each year. Exploiting a vulnerability is now the most common way attackers get in, ahead of stolen credentials. Yet in 2025 organizations fully fixed only about a quarter of the flaws already on the US government’s list of vulnerabilities known to be exploited, and the median fix took 43 days⁴.

AI helps here because the ranking has to combine sources no person can merge daily: the scanner’s findings, the asset inventory, threat intelligence about what is being exploited this week, and the business’s own view of which systems carry revenue, safety or customer data. The re-sorted list changes as conditions change. The executive decision is to fund the inventory and the business context, because without them the AI is sorting a list it does not understand.

Defenders’ AI can shorten the loop

The business evidence on the defensive side is encouraging, though it is an association from a survey, not a controlled test. In IBM’s 2026 study, breaches at organizations that used security AI and automation extensively cost an average of 1.93 million dollars less³. The gains come from the same places the loop suggests: faster identification, faster containment and less manual assembly of evidence. The technical frontier points the same way: in a public competition run by the US Defense Advanced Research Projects Agency in 2025, automated systems found and patched software flaws in minutes¹⁰. A competition is not a production environment, and a found flaw still has to be fixed, tested and shipped. But it shows that defenders’ tools can, in principle, work at attacker speed on the tasks where speed decides the outcome.

Recommending a response is not taking it

During an incident, AI can collect context, assemble a timeline and recommend containment. Suppose it concludes that a laptop is compromised and recommends isolating it from the network. Isolation is an action with consequences. Applied to one laptop it is prudent. Applied to a thousand on a false alarm, it stops the business as surely as the attacker would have.

Every organization therefore decides three things separately: when AI may recommend, when it may act, and when a person must approve. Tightly bounded, high-confidence and reversible actions can run automatically with monitoring and a human override: blocking a domain already confirmed as malicious, isolating one device the evidence clearly shows is compromised, forcing a reset on credentials known to be stolen. Broad, lasting or customer-visible actions, such as locking out many accounts, shutting a production system or cutting a network segment, stay recommendations until a security professional approves. Where that line sits is a statement of risk appetite, and it belongs to the govern function at the center of the loop. The general case for matching autonomy to consequence is made in Agentic AI and Autonomous Actions in Module 06. The security case adds one twist: an AI agent that can isolate devices and reset credentials is itself among the most privileged systems in the building, and it needs the narrowest rights that let it do its job.

Measure the risk removed, not the alerts cleared

A common failure in security AI is a measurement failure. Workload metrics are easy to collect and pleasant to report: alerts closed, share of triage automated, analyst hours saved. Risk metrics are harder: how long attackers stay undetected, how fast incidents are contained, how many past attacks the system catches when real incidents are replayed against it, and how the exposure of critical systems is changing.

A triage assistant that cuts the alerts needing human review by more than half looks like a win. If it has been quietly discarding one class of alert that mattered, workload goes down and risk goes up, and nothing on the dashboard shows it, because nothing on the dashboard measured detection. The remedy is cheap if it is designed in from the start: test the system against a library of past real incidents before it may suppress anything, report false negatives alongside false positives, and decide in advance where freed analyst time goes, whether threat hunting, security engineering or resilience testing. The goal is a safer organization, not automatically a smaller team.

Speed of detection has a legal edge as well. Several laws now start a clock when a significant incident occurs, and an organization that takes weeks to understand what happened cannot meet a 24-hour or four-day deadline. The lifecycle of an AI incident, and who decides when a clock has started, is the subject of Human Oversight and AI Incidents in Module 06.

Story: the video call that cost 25 million

Early in 2024 a finance employee in the Hong Kong office of Arup, the British design and engineering firm behind the Sydney Opera House, received a message that appeared to come from the company’s chief financial officer in the United Kingdom. It asked for a secret transaction. The employee suspected a phishing email, which was exactly the right instinct¹¹.

Then came a video call. The chief financial officer was on it, along with other colleagues the employee recognized. They looked right and sounded right. According to Hong Kong police, every other person on the call was a deepfake. Reassured, the employee made fifteen transfers, a total of 200 million Hong Kong dollars, about 25.6 million US dollars¹². The fraud came to light only when the employee later checked with head office¹¹. In May, Arup confirmed that “fake voices and images were used”, and said that none of its internal systems had been compromised¹³.

Notice what was attacked. Not a server, not a firewall, not a password. The target was a payment process and the trust a careful person placed in familiar faces on a screen. No security tool raised an alarm, because nothing technical was broken. Arup’s chief information officer said at the time that attacks of every kind, including invoice fraud, voice spoofing on messaging apps and deepfakes, were “rising sharply” in number and sophistication¹³.

What would have stopped it was not a better firewall. It was three plain controls. Any unusual payment request is confirmed out of band, by calling the requester back on a number already on file rather than one supplied in the request. Large or unusual transfers need a second approver who was not part of the conversation. And “keep this confidential and move fast” is treated as the signature of fraud, with staff thanked rather than blamed for slowing it down. Tools that detect synthetic voice and video exist and will improve, and they are worth using as one more signal. But the attacker also had AI, and needed only a process that trusted a face. A callback rule works whatever the attacker’s model can fake, and an executive can introduce it in an afternoon.

What this means for leaders

Four lessons follow. First, assume the attacker has AI now, and expect it to make familiar attacks cheaper, faster and more convincing rather than exotic. Second, fund defensive AI where your loop is slowest, usually triage, investigation and the ranking of vulnerabilities, and fund the asset inventory and business context that make its judgments meaningful. Third, write down which actions AI may take alone and which need a person, before the next incident forces the decision. Fourth, judge every security AI investment by the risk it removes, measured against real past incidents, and reinforce the human processes, such as payments, access changes and supplier bank details, that no detection tool protects.

Check yourself

1. Attackers might start using AI once the technology matures.MythFact
2. Most AI-assisted attacks use techniques defenders already know.MythFact
3. An anomaly flagged by security AI is evidence of an attack.MythFact
4. Fixing every vulnerability with a high severity score is the most efficient way to reduce risk.MythFact
5. The Arup fraud succeeded because attackers broke into the firm’s systems.MythFact
6. A security AI that halves analyst workload has proven its value.MythFact

Reflection: follow the money and the access

What comes next

Security AI works best when it can draw on what the organization already knows: past incidents, system owners, policies and context. Much of that knowledge exists but is scattered, out of date or impossible to find when it is needed. The next chapter, AI in Knowledge Management, looks at how AI can turn that scattered information into knowledge people can trust.