Fight against deepfakes of French personalities – DeTOX

Recent challenges have shown that it is extremely difficult to develop universal detectors for fake videos - such as the deepfakes used to spoof a person's identity. When the detectors are exposed to videos generated by a new algorithm, i.e. unknown during the learning phase, the performance is still extremely limited. For the video part, the algorithms examine frames one by one, without taking into account the evolution of the facial dynamics over time. For the voice part, the voice is generated independently from the video; in particular, the audio-video synchronization between the voice and the lip movements is not taken into account. This is an important weakness of deepfake generation algorithms. The present project aims at implementing and learning customized deep fake detection algorithms on some specific individuals for which many real and fake audio-video sequences are available and/or generated. Based on basic audio and video technological modules recovered from the state of the art, the project will focus on taking into account the temporal evolution of audio-visual signals and their coherence for generation and detection. We expect to demonstrate that by using audio and video simultaneously and by focusing on a specific person during training and detection, it is possible to design efficient detectors even in presence of unseen generators. Such tools will allow to searchy and detect on the web possible deepfakes of important French personalities (president of the republic, journalists, head of the army, ...) as soon as they are published.