Methods of impersonating people
And how harmful it could potentially be.
The idea for this article came a few months ago, when I watched this video from Two Minute Papers. I was amazed by how convincing the deepfaked videos were, and immediately realized the terrifying possibilities that this opens up since this is completely free to use for anyone around the world. This model is very powerful and works for way more than just humans too! I felt this was an important issue everybody should know about, so I wrote this short article about a few simple experiments I did.
Here’s the link to the Google Colab for the first AI algorithm:
The first experiment I did was just with the default sample data. I created this:
This was the first time I had ever used Google Colab and also the first time I tried to use this kind of AI algorithm. As you can see, it’s not perfect, but it works astonishingly well considering the only input for the Mona Lisa was an image. It’s far from perfect though; As you can see near the end of the GIF, the Mona Lisa’s face gets distorted when Trump rotates his head.
It is a very big deal what is going on beneath the hood of this software. It is generating what it thinks the other side of the face looks like, in addition to copying mouth and eye movements with astonishing accuracy.
You can even animate statues!
As you can see, the software can recognize features from other things too. Also, a feature of the software is that you can use relative or absolute displacement for the animation. Absolute displacement modifies the features of the image to match the input video exactly, whereas relative displacement maintains the placement of the features relative to their original position.
In the future, it would be crucial to be able to identify these deepfakes. An important point that you may be able to use to identify these, is the reflection. As you may be able to see, the reflection in the real video is accurate, yet the reflection on the forehead of the statue is treated as just a texture moving with the head. I’m sure that future improved software will be able to fake reflections as well. A recent image inpainting algorithm is able to fake reflections even in mirrors, so accurate reflections in deepfakes are not far off.
Next, I experimented with a different software, called Wav2Lip. This sofware takes in a video, and an audio file, and it will sync up the video to match whatever is being said in the audio file. This requires a video of the person to animate, but only needs audio for driving the deepfake. A good voice actor could easily create very convincing deepfakes.
Here’s the code that I used:
And here’s what I managed to create:
Deep fake technology is very powerful and must be treated with caution. That concludes this short article, and I hope you found it interesting and helpful.
P.S. This article was published about 4 months late, and therefore may be very outdated. Deep fake technology has probably advanced a lot more in the past few months.
2023 Update:
Now, there’s a new AI system that has been developed that allows a three second clip of someone’s voice to be used as training, to generate speech from text. Deepfakes now are indistinguishable from reality. Truly a scary time we live in today.