Creating a Deepfake for ITV

The power of deepfake technology has been used creatively and legitimately by VFX studios across many films and TV shows, such as the recent Tom Hanks film ‘Here’. 

Creating a deepfake requires a huge and diverse dataset of images of the person who is being deepfaked. This is so the resulting deepfake model can accommodate a wide range of facial expressions, positions, lighting, and optics. For instance, if the deepfake training process never trains on imagery of the person smiling, it won’t be able to accurately create deepfaked imagery of that person smiling.

Lux Aeterna has been building their knowledge in deepfake technology for a number of years. As a high-end visual effects studio, it’s important that we stay on top of powerful emerging technologies and how they can be applied in delivering fresh, exciting storytelling for our clients.

In addition to this, our work with clients in the digital forensics sector means providing deep research into the tools and processes that are of concern to that sector.

For ITV’s new documentary ‘Georgia Harrison: Porn, Power, Profit’ Lux Aeterna provided expert insight into how deepfakes are made, and created a special celebrity deepfake of its presenter, Georgia Harrison. Lux Aeterna’s Creative Technologist, James Pollock, directed a shoot where footage of both Georgia and model Maddison Fox was captured, and later used deepfake software to place Georgia’s face on Maddison’s body.

By capturing our training imagery of Georgia under the exact same conditions as the film of Maddison, we ensured that the deepfaking process worked with the video of Maddison without requiring a range of lighting and optics in our dataset.

This still left the matter of facial expressions. To ensure that a wide range of Georgia’s facial expressions were captured, we used what are called “phonetic pangrams”. These are sentences that, when read, use every sound used in speaking English, and in theory, cover a wide range of facial expressions.

We asked Georgia to stand where Maddison would be filmed and read phonetic pangrams while turning and rotating her head. Here’s an example of a phonetic pangram we used with Georgia: The beige hue on the waters of the loch impressed all, including the French queen, before she heard that symphony again, just as young Arthur wanted.

Back at the studio, we prepared the images of Georgia and Maddison to be used to train the deepfake model. Once the training started, we left the computer to train the deepfake model over several days, checking in occasionally to ensure the process was proceeding smoothly.

At the end of the training process we had a deepfake model that we were able to use to create a video of Georgia’s face that aligned with Maddison’s video. We composited the two videos together, colour correcting and cleaning up any issues, resulting in the finished deepfake video.

James gave his thoughts on the project: “It was great working with Georgia and Maddison on the shoot, they brought a lot of humour and fun to what could’ve been quite an awkward process! The video we made with them is a great example of the accessibility and power of deepfake tools.”

Find out more about the work the Creative Technology team do here.