Follow ZDNET: Add us as a preferred source on Google.
ZDNET’s key takeaways
- Deepfakes and AI clones get more sophisticated and harder to detect.
- Advanced attacks involve long-term infiltration of a company’s systems.
- Experts recommend low-tech security protocols that offer better defenses.
In January 2024, an employee at professional services firm Arup joined a video call with someone they believed included the company’s CFO. This call resulted in 15 wire transfers to third-party accounts totaling about $25m.
Every participant on the other side of the video call was an AI-generated clone cobbled together from public appearances and earnings calls of Arup executives.
Also: 6 essential strategies to defend against AI-powered threat actors
“Seeing and hearing someone is no longer proof they are real,” said Deepak Gupta, technical CEO at GrackerAI, when discussing the Arup incident with ZDNET. “Any protocol that relies on ‘I recognized their face and voice’ is now broken.”
We live in a post-truth world where employees handle millions of dollars in company funds every year. Mistakes can be expensive, even when attacks don’t lead to a direct loss of funds. Over time, the damage to customer trust and fines for legislative non-compliance alone can put an otherwise profitable company out of business.
How can you keep your organization safe when cyber fraud no longer leaves a signature? Security experts like GrackerAI’s Gupta and James Scobey, CTO at B2B cybersecurity firm S2i2, think the answer lies in the same low-tech systems once considered too simple for large-scale business operations.
There are no obvious tells
Live voice and video calls have remained the gold standard for identity verification throughout most of corporate history. These techniques have been central for authenticating high-value transactions, exchanging sensitive medical data, or discussing matters of legal importance.
Also: 43% of companies have already experienced AI-enabled cyberattacks
But in the last five years, that standard has been quietly eroded by deepfakes that have grown better at mimicking human behavior at an uncanny speed. There used to be certain tells, like background noises, synthetic voice modulation, and the distinct lack of breathing sounds. However, deepfake technology has outpaced these tried-and-tested tells, and it’s difficult to tell when your digital coworker is real.
A recent University College London study found that listeners could accurately identify deepfakes only about 73% of the time. Even with proper training and familiarization with deepfake samples, the accuracy rate improved by just 3.84%. Just a year later, researchers from the University of Duisburg-Essen and Indiana University aggregated results from 56 similar studies and found that human detection rates were actually closer to chance in terms of accuracy.
“Tells still have marginal value as a supporting signal,” explained S2i2’s Scobey. However, relying on these techniques as an effective control is no longer an option because they train employees to rely on their perception during attacks, which is precisely the sense these scammers target.
A joint information sheet released by the NSA, FBI, and the Cybersecurity and Infrastructure Security Agency (CISA) also ruled out traditional automated detection protocols that visualize evidence of manipulation in voice or video content. These methods assume that statistically significant traces of manipulation can be found and recorded, which is no longer necessarily the case.
What’s even more troubling is that as technology becomes more sophisticated, social engineering attacks are no longer restricted to one-off financial scams but extend to the long-term infiltration of a company’s systems.
Also: AI image fraud will cost $40 billion next year – can these international standards help?
Gupta pointed to a 2024 incident involving security training firm KnowBe4, which was the victim of identity fraud. KnowBe4 hired an attacker after interviews and screening that did not flag the individual as suspicious. They even supplied the employee with a company workstation before realizing they were a North Korean operative using the device to upload malware to their systems.
“It’s almost poetic,” Gupta remarked while recounting the incident. “This is a security training company; its whole business is teaching people to spot social engineering scams like this.” He also added that North Korean attackers have been running this type of operation at scale, sending off thousands of fake workers each year to pose as employees at companies in the US and in Europe.
A public notice from the US Department of Justice in 2025 also suggested that North Korean attackers operated with assistance from collaborators in the US, China, the United Arab Emirates, and Taiwan. The Google Threat Intelligence Group has been investigating similar incidents since 2022, with a significant portion of them now targeting European nations.
Old-fashioned is best
Where advanced detection models and standard security countermeasures fail, analog solutions that rely on a single controlled point of entry prove much more effective.
“The defense against the most advanced AI attack is often deliberately low-tech,” said Gupta. Deepfake technology may be much better now at replicating publicly accessible appearances and voices, but is entirely useless against anything that exists outside observable channels, and that cannot be spied on remotely.
Also: 5 security tactics your business can’t get wrong in the age of AI – and why they’re critical
More fraud consultants now recommend hardware-based security keys or verbal passphrases as security measures, based on guidance from several government agencies. CISA, which operates under the US Department of Homeland Security, now recommends FIDO2 and PIV hardware credentials as the new gold standard for multi-factor authentication (MFA). Secret verbal passphrases shared between members of a workplace (or household) are also a recurring recommendation from the FBI.
A verbal passphrase is a secret word or phrase shared between participants during a video or voice call. Employees can authenticate the person on the other end of a call by asking them to say the secret passphrase. However, these verbal passphrases are only effective if they’re used consistently by everyone in an organization. Gupta explained that, in his experience, employees often skip security checks when they feel intimidated, such as when the person on the other end of a call presents themselves as a superior at work.
Also: OpenAI’s attack agent did exactly what it was told – just more relentlessly than expected
“Make the control automatic and no-exception,” said Scobey, confirming that attackers often create a sense of urgency or social pressure to encourage company workers to skirt established protocol. He also suggested running simulated deepfake or voice-phishing calls as part of employee security training to help people practice saying no to someone who appears to speak from a position of authority.
Gupta added that it might even be better to enforce verbal passphrases as an authentication requirement directly in the company systems, which some finance platforms are already doing.
Apart from implementing a no-exception policy, low-tech solutions like passphrase authentication need to be made scalable for larger organizations that cannot afford a single point of failure. Here, Scobey recommended taking the same best practices that apply to standard company password management and applying them to verbal passphrases, for example:
- Don’t create verbal passphrases manually; use a random password generator.
- Use unrelated words separated by random numbers and symbols, for example: “harley9jedi@buddies.sinclair”.
- Maintain separate verbal passphrases for different roles and transactions to avoid a single point of failure.
- Reset your passphrases periodically, but not according to a fixed schedule.
- Combine passphrase authentication with other security protocols (hardware keys, out-of-band callbacks, dual authentication).
Scobey pointed to the Electronic Frontier Foundation’s word list for stringing together random passphrases that are difficult to guess but easy to recall. It’s better to use a random password generator, since human minds have subconscious biases and preferences that make passphrases easier to guess. “The strength comes from the randomness of the selection,” Scobey added, “not from how meaningful the words are.”
Larger organizations should also implement role- and relationship-based passphrases that are specific to certain transactions and employee roles. For example, a company could have a unique passphrase for the accounts payable department to authorize all financial transactions under $1,000, another passphrase for transactions ranging from $1,000+ to $20,000, and so on.
With that approach, if one passphrase is compromised, it can be easily reset while others stay intact. Passphrases should also be rotated frequently, but not according to a fixed calendar. NIST guidelines now suggest overturning the 90-day password reset rule because it creates too much predictability. Instead, passphrases should be replaced whenever there’s evidence of compromise, role change, or employee offboarding.
Also: The best password managers: Expert tested
Finally, a verbal passphrase alone cannot provide sufficient security for high-value transactions in large enterprises. The passphrase must be combined with other protocols, such as out-of-band callbacks and dual authorization. That approach means authenticating the request through more than one official communication channel and requiring a second authorized employee to validate the request before it’s processed.
Gupta has found that companies object to introducing too many hurdles in the security system. However, the goal isn’t to add friction everywhere, but to confine it only to high-risk exposure workflows.
“When people experience security as targeted rather than blanket, they stop trying to bypass it,” he said. If the sheer number of passphrases makes them difficult to manage, Scobey suggested using a FedRAMP-authorized password manager, such as Keeper Security, to store company credentials.
