How AI Is Making Phishing More Convincing—and Why Traditional Email Security Can Fall Short
Summary
The article explains how generative AI enables more convincing, personalized phishing and deepfakes, and outlines defenses including behavioral detection, phishing-resistant authentication, separate verification channels, and procedure-focused training.
Key points
- The article describes the 2024 Arup fraud, in which an employee made 15 wire transfers totaling $25.6 million after a video call featuring AI-generated impersonations of executives and colleagues.
- It cites Microsoft research reporting a 54% click-through rate for AI-generated phishing emails, compared with 12% for manually written phishing.
- Generative AI and purpose-built tools can reduce the time and effort needed to create personalized phishing lures.
- Compromised legitimate accounts, plausible requests without malicious links or attachments, and changing message patterns can make phishing harder for traditional filters to detect.
- The article recommends behavioral detection, phishing-resistant MFA such as FIDO2 keys or passkeys, and independently initiated verification for financial requests.
- It advises training employees to follow verification procedures and pause when requests create urgency, rather than relying mainly on spotting typos.
Article Details
- Topic
- AI-powered phishing: how generative AI enables more convincing and scalable social engineering, and recommended defenses.
People
Andrei SavineThanked for ongoing support on Substack.Erich WinklerRecommended by the article as a person covering privacy and related topics.Jeff MorhousAuthor of The AI-Augmented Engineer, which featured ToxSec.Joel SalinasCollaborated on a previous piece about AI-powered scams.MohibAuthor of the article on AI-powered phishing.Nelson LopesThanked for ongoing support on Substack.Saqib TahirNamed author of the SK NEXUS article on AI-powered phishing.Tate JarrowRecommended by the article as a person covering privacy and related topics.
Vendors
HoxhuntSimilarly, Hoxhunt’s 2026 Phishing Trends Report tracked the share of AI-assisted phishing in its detection network rising from under 5% in November 2025 to 56% in December, a 14-fold jump in a single month. That shareIBMMicrosoft describes it as the most significant change in phishing the company observed in the past year. IBM’s X-Force research found generative AI has cut the time required to draft a convincing phishing email fromKnowBe4Polymorphic campaigns make detection harder by changing subject lines, sender names, and message structures across different emails. KnowBe4’s 2025 phishing research found this type of variation across observed attacks.MicrosoftMicrosoft’s 2025 Digital Defense Report found that AI-generated phishing emails achieve a 54% click-through rate, compared to 12% for manually written phishing. Microsoft describes it as the most significant change in
Products
FIDO2Phishing-resistant multi-factor authentication, including FIDO2 security keys and passkeys, addresses a key weakness in credential-based attacks. Even if an employee enters their credentials on a convincing fake loginKawaiiGPTA 2026 successor called KawaiiGPT is distributed freely on GitHub.WormGPTWormGPT, first sold on underground forums in 2023, was fine-tuned for phishing templates, malware code, and exploit writeups and stripped of the safety guardrails found in commercial models. Multiple successor versions