Attackers are adept at using new technologies to enhance their attacks. Earlier this afternoon, for example, I got call from “American Express” suggesting that I needed to “verify a transaction.” The caller used a robotic voice similar to the one used by American Express’ automated systems, and only obvious signals that it was a scam were a) they didn’t have my name or card number, b) I have been using my Visa exclusively for the past few weeks, and c) I have their app, and I get push notifications from it.

Recently, there’s been some excited press about attackers abusing AI-powered “deepfake” technology to persuade users to take unsafe actions (for instance, sending money or sharing secrets) by using a trusted person’s voice or appearance to make the pitch more compelling. Attackers no longer need to limit themselves to spoofed phone numbers or names, they now can use a real-looking voice or video.

But is cloning still an advanced technique, or are we soon going to see it everywhere?

Free Voice Cloning

The best AI models require money to use and I don’t have the time or attention span to fully explore them, but as a browser lover/enthusiast, I’m intrigued to look at how convincing a spoof I can do using an in-browser speech cloning tool.

Beyond being free, such a tool will allow an attacker to generate speech without calling a web service that might keep logs that could be used to catch the attacker. Behold SoundTools.io. It’s trivial to use and is very upfront about its market position and quality vs. competitors.

To use it, you can either feed it previously recorded audio, or (preferably) a live recording of a paragraph of text:

After collecting the audio sample, the page downloads several large models (just over a gigabyte, cached for later use), to generate a cloned voice. It then allows the user to provide text that should be read aloud, and a desired quality level (lower-quality reproduction is faster):

The time required depends on the quality chosen and the performance of the browser on your device. In my quick test, the “High quality” level, which requires 60 to 150 seconds per sentence, seems to generate pretty good results.

Finally, when the generation completes, the recording is available to play or download.

Folks who have heard me speak would probably agree that the results are pretty plausible. While my real voice has more “ums”, “uhs”, variable pauses, and animation, careful preparation of the source text would yield even more compelling output. (For example, the web app doesn’t currently handle “smart quotes” properly, voicing contractions like “Can’t” as “Can Tee”.)

High Quality example (text from this post):

Ultra Quality example (text from this post):

Both of these are pretty solid, and attackers who worry about the quality of the clone would also likely simulate the speaker being in a noisy environment or suffering from a poor phone signal.

Not just voice, obviously

Don’t even get me started about how trivial photo editing has gotten:

Even today’s high schools can afford to generate sizzle videos for marching bands that would’ve cost millions of dollars to render when I was high school.

Exciting stuff! But scary in the hands of bad guys. I don’t even know what to suggest folks do to stay safe, other than slow down and think, be extra skeptical of situations where someone unexpectedly contacts you, and be cautious of all non-in-person interactions, especially when it’s not a real-time conversation (which can still be faked, but it’s harder).

For years, it’s been easy to fake an entire company, but today it’s not too hard to shallowly fake a family member or close friend. Perhaps in the future, we will all share a Shibboleth with close friends, or demand out-of-band confirmations of all requests?

Stay safe out there, and don’t believe everything you hear, or see!

-Eric