When Reality Can’t Be Trusted: Deepfakes, Digital Deception and the Next Wave of Scams

Author: R. Paul Wilson

Our modern world is filled with marvels once restricted to science fiction, but as technology evolves with staggering speed, the dangers of being deceived or manipulated by new forms of media increase daily. The speed and capability of computers to simulate reality is a prime area of concern as so-called “AI” continues to hit milestone after milestone far faster than its creators could have predicted. If you’re not spending time trying to keep up with this runaway train of unchecked capabilities, you are vulnerable to learning the breathtaking feats of current digital fakery the hard way.

Imagine your parents, grandparents, or partner receiving a video call from you in the hospital, urgently asking for help before passing the phone to a helpful nurse or police officer to guide them through a process to send funds. Meanwhile, in reality, you’re overseas, playing in a brick and mortar casino for long hours. When you head back to your room and call your partner or family back home, they are deeply concerned about you and immediately ask if you’re okay and “what happened?”

It’s an old scam with a brand-new method.

Originally, you might have received a letter from overseas, telling you a loved one was injured, imprisoned, or arrested, and that funds should be sent to liberate them somehow. Then the telephone was invented, and you would receive a call from an official, or if the scammers knew what your loved one sounded like, would apparently speak directly with whoever needed money urgently. In the film Nine Queens, a con artist plays a clever variation of this scam on a random person in a high-rise apartment using the intercom, convincing an old lady he’s her young relative and persuading her to throw money out of the window, fifteen floors up!

Facebook and other forms of social media offered new ways to convince people they were someone they knew in trouble, whether through hijacking legitimate accounts or emulating them so well that victims never questioned that they were receiving messages from someone they cared about.

The moment DeepFaceLive was released on GitHub, I knew it was only a matter of time before it would be applied to scammers on live video calls. Anyone with imagination – and a healthy degree of cynicism – knew that online criminals would have a heyday as soon as they got their hands on that kind of software, and once DeepFaceLive was available, the barn doors were wide open, and the horses long gone before ordinary people had any idea the technology even existed.

How Deepfakes Are Built and Deployed

If this were to happen to you, all the thieves would need is a decent photo, some short video, and a brief sample of your voice to construct a working deepfake mask that might be used to call someone you know and convince them you’re in trouble. In the example given above, the scammers would capture your voice and secretly record video in or around the casino, then get access to your phone through some pretext in order to steal more information, including your contacts, who will be targeted with a video call or even a video message.

If I wanted to demonstrate this scam on The Real Hustle, that’s how we would do it, so that all the steps were performed live and easier to show and remember. But sadly, the most common examples of these scams are performed remotely using the photos and videos we put online ourselves, which are more than enough to build a convincing deepfake. The trick, as always, is to create a believable but shocking story that immediately motivates the target to act, and by the time they can tell anyone, they have already lost their money.

This is one of the key dangers of social media, where scammers can glean incredibly useful information for all kinds of deception, from identity theft to simple burglary. A friend once told me how he would lie to taxi drivers taking him to the airport so they wouldn’t think his apartment would be empty while he was away, yet at the same time, his Facebook page was a daily account of his travels! He might have the right idea not to give a stranger too much information in person, but it seems a little pointless when simultaneously broadcasting his whereabouts to the rest of the world!

We’ve touched on this topic before when an AI voice and video deepfake (a simulation designed to mimic a real person) was used to steal millions from a bank in Hong Kong, in what became a high-profile deepfake fraud involving a fake executive call. This was thanks to a computer-generated version of the bank’s CFO authorising an emergency transfer. It’s not as exciting as the heist from Michael Mann’s masterpiece HEAT, but the digital bank robbers scored many millions more than Robert De Niro and his heavily armed crew, without the need for a deadly shootout.

The trick to the Hong Kong deepfake was to run a long video in a group chat that could be cued to speak pre-recorded (faked) statements when needed, but live deepfake masking has now been in the wild for over three years and is being employed by scammers to target anyone from bank CEOs to backpackers’ grandmothers.

From Financial Crime to Social Harm

Another predictable outcome of this technology has been the bullying of children by their peers, making still photos come to life and perform embarrassing acts, shared widely on social media and causing many victims to self-harm. Last year, a 13-year-old boy used a bat mitzvah photo of a classmate to generate a nude deepfake, which he then shared online. There was no complicated software or covert system – just an app, a phone, and a moment of cruelty.

The story made headlines, but it wasn’t unique. Across the country, cases like this have quietly multiplied. According to a recent survey from the Center for Democracy and Technology, fifteen percent of students said they’d seen or heard of AI-generated sexual images of a classmate.

Depressingly, the majority of the victims tend to be girls.

Deepfakes will only improve and become more prevalent in all walks of life; I previously predicted that the last US election cycle would be affected – and possibly influenced – by some kind of simulated, fake media, but as it turned out, there was enough misinformation, doubt and uncertainty without it. At that time, deepfakes were still detectable, but today it’s becoming a lot harder to recognise.

Consider how you might spot an AI-generated video, and you can be sure that AI is quickly upgrading its output to remove such ‘tells’ in the very near future – and the future is tomorrow, not next year.

Also, as an advisor on how to avoid being scammed, I strongly recommend that you pause to consider how you and your family or business might be targeted with any form of generated media and talk about it with family and friends, so the topic is elevated, and knowledge of these applications is broadened. Modern media is now a minefield of twisted truth and boldface lies, often filtered by our own personal bias, with the clear goal of manipulating our actions or influencing our decisions. The training wheels are off, the bicycle gathering speed down a steep slope towards an uncertain future where we may not be able to trust anything we see, hear, or read unless we witness it in person.

What This Means for Gambling and Live Events

Will this affect online or live gambling?

Definitely.

I am currently investigating the veracity of a story centred on one of the world’s biggest poker tournaments, where investors were convinced they were bankrolling a well-known and successful player and even receiving regular video updates of his progress, milking more money out of the victims before the real player dropped out of the tournament, unaware of what was happening with his image! Imagine the same principle and similar technology being used to fake a live sporting event that people were gambling on!

Personally, I can think of multiple ways to use that kind of deepfake for nefarious purposes.

In the meantime, understand what’s possible and how easy it is now for even the dumbest of criminals to simulate a believable – but fake – version of reality.