For twenty years, security awareness training taught people to look for bad spelling, odd grammar, and generic greetings. That advice worked because most phishing was written by people working in a language they did not speak well.
Generative AI removed that tell completely, and it did it for every language at once. The training has to change, and more importantly the technical controls have to stop depending on humans noticing.
What AI Actually Improves in Phishing
Three things. The writing quality, which is now indistinguishable from a real colleague. The personalization, because public information about a target can be gathered and woven in automatically. And the volume, because producing a thousand individually tailored messages now costs about the same as producing one.
Before, an attacker chose between a generic message sent to ten thousand people or a carefully researched message sent to one executive. That tradeoff is gone. Targeted quality at bulk volume is now the default.
The rest of the attack chain is unchanged. The goal is still to get a credential, a payment, or a click, and the infrastructure behind it is still domains, hosting, and mail delivery.
Business Email Compromise, Now With Voice
Business email compromise is the highest cost category of cybercrime by dollar value in most years of the FBI Internet Crime Complaint Center reports, well ahead of ransomware. It works by convincing someone with payment authority to send money to the wrong account.
The classic version was an email that appeared to come from an executive. The current version adds a phone call in that executive's voice, because a usable voice clone can be produced from a short sample, and executives have plenty of public audio from conference talks and interviews.
The most widely reported escalation so far was a 2024 case involving the engineering firm Arup, where an employee in Hong Kong took part in a video call with what appeared to be several colleagues, all of which were synthetic, and authorized transfers reported at around twenty five million United States dollars.
Why Detection of the Content Is a Losing Game
It is tempting to solve this with a detector that spots AI generated text, audio, or video. That approach has a structural problem: detectors are trained on the output of current generators, and generators improve continuously.
Text detectors in particular have well documented false positive problems, which means they accuse real people of faking things. Audio and video detection is more tractable but still an arms race, and it puts you in the position of needing to win every round forever.
The reliable defenses do not try to judge whether the content is real. They remove the value of a successful impersonation.
What Actually Works: Process
Out of band verification for money movement. Any request to change payment details or send funds gets verified through a channel the requester did not choose, using a contact number you already had. This single control defeats almost all business email compromise, including the deepfake version, because the attacker does not control your existing phone list.
Dual authorization above a threshold. Two people, independently, for anything large.
A culture where stopping to verify is never treated as an insult. Most successful attacks include urgency and authority precisely because they push people past the point where checking feels socially acceptable.
What Actually Works: Technology
Phishing resistant multi factor authentication is the single highest value control against credential phishing. Hardware security keys and passkeys built on the FIDO2 and WebAuthn standards are bound to the real site origin, so a convincing fake login page cannot use them. Codes from an authenticator app can be relayed by a proxy in real time. The FIDO Alliance publishes the specifications, and major identity providers such as Okta, Microsoft Entra ID, and Duo support it.
Email authentication done properly. SPF, DKIM, and DMARC with an enforcing policy stop straightforward domain spoofing. They do not stop lookalike domains, which is why the next layer matters.
Modern email security that scores behavior and relationships rather than content alone. Vendors in this space include Abnormal Security, Proofpoint, and Mimecast. The useful signal is that this sender has never emailed this recipient before and is asking for a wire transfer, which does not depend on the prose being clumsy.
How Awareness Training Should Change
Stop teaching spelling and grammar as indicators. Teach the request pattern instead: urgency, secrecy, authority, a change to payment details, or a push to move the conversation to another channel.
Teach that voice and video are no longer proof of identity. This is uncomfortable and it is true.
Agree on a verification method in advance for high value requests. Some organizations use a spoken code phrase for finance approvals. It sounds theatrical and it works, because it is something an impersonator cannot obtain from public information. Vendors such as KnowBe4 have updated their simulation content to include these scenarios.
Learn This at HackRange
Module 9 of the CEH course covers social engineering hands on, including cloning a login page inside the range so you can see exactly how convincing it is.