It is not the calm and thoughtful Gareth Southgate the nation is used to and, in the rough and ready world of internet humour, that is probably the point.
Within hours of England walking off the pitch after winning the semi-final against the Netherlands, deepfakes of the team manager cropped up on social media, offering an expletive-filled, and deeply uncharacteristic, post-match take from the England manager.
“It looks like I put them in a big fat Rizla today,” says a faux-Southgate in one video posted on TikTok after England’s dramatic semi-final victory over the Dutch. The videos use artificial intelligence tools to replicate Southgate’s voice and manipulate the lower half of his face in an amateurish attempt at lip-syncing. There will doubtless be more after the final on Sunday, whatever the result.
The England manager is widely considered to be an emblem of the nation but in this case he is emblematic of an internet trend: the deepfake meme.
Like image editing before it, AI-generated spoof video clips – deepfakes – have become a key tool in the internet jokester’s toolbox.
“Deepfakes are becoming the new Photoshopped content,” says Bahareh Heravi, a professor at the Institute for People-Centred AI at the University of Surrey. “Now people can use different tools that are AI-based to create content similar to what they did with the likes of Photoshop and video editing tools before. But now it’s enhanced with much greater video and audio generation capabilities.”
At the simple end, commodity face-swapping apps let anyone simply substitute one face for another in a video clip, making a big technical achievement in 2018 simply a matter of pushing a button. Recent examples include Rishi Sunak and Keir Starmer as Del Boy and Rodney Trotter and Noel Gallagher as a range of Irishmen including Roy Keane in an apparent banter-nod to the rock star’s references to his Irish heritage in interviews.
Paired with voice cloning tools such as those from ElevenLabs, it’s possible to go one step further and write a full script for your newly faked celebrity to read out. If you’ve seen footage of British politicians cast as Twitch Minecraft streamers, you’ll know the resulting creations can be rich with in-jokes and references, with only the odd robotic twang or skewwiff mouth pulling you out of the gag.
Experts at Faculty AI, a UK-based company that has commercial and UK government contracts to spot deepfakes, say the Southgate clip has obvious giveaways including mouth movements not lining up with the words and unnatural pauses and pacing in his speech. The most obvious sign, of course, is that England fans know Southgate would never speak about his players – Harry Kane, Phil Foden and Jack Grealish are among the players eviscerated by their fake boss – in such a way.
Faculty said AI video deepfakes probably reached a turning point last year with a video reimagining a Balenciaga catwalk with Harry Potter stars, adding that AI-based fraud probably poses a “greater threat to society”. ITV’s Deepfake Neighbour Wars, featuring simulations of stars including Idris Elba and Kim Kardashian, also showed the concept entering the mainstream.
However Faculty said generative AI – the term for systems such as ChatGPT or Midjourney that produce convincing text, audio and image from simple prompts – are improving at such a rate that “manual detection alone will soon be insufficient to prevent the spread of genuinely harmful content”.
The FA said: “As we do with all harmful content, we will take steps to have offensive videos removed.”
skip past newsletter promotion
Alex Hern’s weekly dive in to how technology is shaping our lives
Privacy Notice: Newsletters may contain info about charities, online ads, and content funded by outside parties. For more information see our Privacy Policy. We use Google reCaptcha to protect our website and the Google Privacy Policy and Terms of Service apply.
after newsletter promotion
The intention of such pranksters is rarely to mislead directly, but that doesn’t mean the jokes can’t confuse. On the internet, there’s no parody that can’t ultimately be mistaken for an example of that which it’s mocking – an occurrence so common it was named “Poe’s law” almost two decades ago, after the commenter who noted the difficulty of satirising extreme views. As with articles from the Onion, or images produced for the humour website B3ta, it’s fairly common to see viral deepfakes reshared with increasingly incredulous comments asking whether or not the clip is real. Some of the Southgate clips are labelled as being AI-made.
The Southgate videos have been heavily viewed on social media. On YouTube, where there are many fake Southgate clips, one deepfake posted after the Slovakia match has had 390,000 views while eight videos on a TikTok account dedicated to Southgate fakes have had a total of more than 1m views. The TikTok account links to an eBay page selling customised AI messages from Southgate, adding: “if you want some swear words please let us know”.
TikTok’s guidelines require the labelling of realistic-looking AI content and allows the likeness of a public figure “in certain artistic or humorous settings”. YouTube has changed its moderation policies to allow people to request removal of deepfakes, although it has said it will consider whether the video is a parody or satire before deciding to pull it.
“For public figures things are out of their hands as anything can be falsely attributed to them,” says Heravi, adding that she is particularly concerned about audio because there are no visual clues for spotting deepfake voice. “However, we can expect things to improve as we await better deepfake detection tools, and more importantly, as the public becomes more aware and educated about deepfakes.”
Fake video isn’t the only place generative AI is reshaping the memes of production. Meme generators, led by Glif, have started to automate almost the entire process of humour creation. For a meme with a set format, like “chad/virgin” or “stop doing math”, users can create whole templates that pair a carefully prompted large language model (LLM) to generate text, with an image generator for pictures, and a standardised structure to pull them all together. The result is a tool that can be instructed to make a meme about why you should “stop doing football tournaments” and receive in seconds the exhortation that “KICKING A BALL WAS NOT MEANT TO DETERMINE NATIONAL PRIDE”.
