Detection of fake images generated by deep learning

During the last few years, the amount of audiovisual content produced is continually increasing with technology development. Along with this growth comes the availability of the same information through numerous devices that any individual holds, including smartphones, laptops, tablets, and smart TVs, in an entirely free and open manner. These type of content are considered an authenticity element since they represent a reality record. For example, in court, photos frequently determine the jury's course of action since what is available is a recorded picture that validates a narrative and usually does not leave room for doubts. However, with the advancement of Deep Learning (DL) algorithms, a new and dangerous trend known as Deepfakes begins to emerge. For example, a deepfake can be a video or an image of a person on which their face or body is totally or partially modified to appear to be someone else. This technique is often used for manipulation, blackmailing, and spreading false information. After recognizing such a dangerous problem, this study aims to uncover patterns that deepfakes show to identify authenticity as accurately as possible, using machine learning and deep learning algorithms. To get the highest level of accuracy, these algorithms were trained on datasets that included both real and phony photos. The outcomes demonstrate that deepfakes can be accurately identified and that the optimal model may be selected based on the specific requirements of the application.RESUMO: Nos últimos anos, a quantidade de conteúdo audiovisual produzido tem vindo a aumentar continuamente com o desenvolvimento da tecnologia. Juntamente com este crescimento, surge a disponibilidade da mesma informação através de inúmeros dispositivos que qualquer indivíduo possui, incluindo telemóveis, computadores, tablets e smart TVs, de uma forma totalmente livre e aberta. Este tipo de conteúdo é considerado um elemento de autenticidade, uma vez que representa um registo da realidade. Por exemplo, em tribunal, as fotografias frequentemente determinam a linha de ação do júri, uma vez que o que está disponível é uma imagem registada que valida uma narrativa e geralmente não deixa espaço para dúvidas. No entanto, com o avanço dos algoritmos de Deep Learning (DL), começa a surgir uma nova e perigosa tendência conhecida como Deepfakes. Por exemplo, um deepfake pode ser um vídeo ou uma imagem de uma pessoa na qual o rosto ou o corpo é totalmente ou parcialmente modificado para parecer ser outra pessoa. Esta técnica é frequentemente utilizada para manipulação, chantagem e disseminação de informações falsas. Após reconhecer um problema tão perigoso, este estudo tem como objetivo descobrir padrões que os Deepfakes apresentam para identificar a autenticidade da forma mais precisa possível, utilizando algoritmos de Machine Learning e Deep Learning. Estes algoritmos foram treinados utilizando conjuntos de dados que contenham tanto fotografias autênticas quanto falsas, a fim de obter o melhor nível de precisão. Os resultados obtidos mostram bons resultados na identificação de deepfakes e que a escolha do melhor modelo pode ser ajustada às necessidades da aplicação em causa

DeepFakes Detection Based on Heart Rate Estimation: Single- and Multi-frame

This chapter describes a DeepFake detection framework based on physiological measurement. In particular, we consider information related to the heart rate using remote photoplethysmography (rPPG). rPPG methods analyze video sequences looking for subtle color changes in the human skin, revealing the presence of human blood under the tissues. This chapter explores to what extent rPPG is useful for the detection of DeepFake videos. We analyze the recent fake detector named DeepFakesON-Phys that is based on a Convolutional Attention Network (CAN), which extracts spatial and temporal information from video frames, analyzing and combining both sources to better detect fake videos. DeepFakesON-Phys has been experimentally evaluated using the latest public databases in the field: Celeb-DF v2 and DFDC. The results achieved for DeepFake detection based on a single frame are over 98% AUC (Area Under the Curve) on both databases, proving the success of fake detectors based on physiological measurement to detect the latest DeepFake videos. In this chapter, we also propose and study heuristical and statistical approaches for performing continuous DeepFake detection by combining scores from consecutive frames with low latency and high accuracy (100% on the Celeb-DF v2 evaluation dataset). We show that combining scores extracted from short-time video sequences can improve the discrimination power of DeepFakesON-PhysThis work has been supported by projects: PRIMA (H2020-MSCA-ITN2019-860315), TRESPASS-ETN (H2020-MSCA-ITN-2019-860813), BIBECA (MINECO/FEDER RTI2018-101248-B-I00), and COST CA16101 (MULTI-FORESEE). J. H.-O. is supported by a PhD fellowship from UA

Deepfake Video Detection Using Generative Convolutional Vision Transformer

Deepfakes have raised significant concerns due to their potential to spread false information and compromise digital media integrity. In this work, we propose a Generative Convolutional Vision Transformer (GenConViT) for deepfake video detection. Our model combines ConvNeXt and Swin Transformer models for feature extraction, and it utilizes Autoencoder and Variational Autoencoder to learn from the latent data distribution. By learning from the visual artifacts and latent data distribution, GenConViT achieves improved performance in detecting a wide range of deepfake videos. The model is trained and evaluated on DFDC, FF++, DeepfakeTIMIT, and Celeb-DF v2 datasets, achieving high classification accuracy, F1 scores, and AUC values. The proposed GenConViT model demonstrates robust performance in deepfake video detection, with an average accuracy of 95.8% and an AUC value of 99.3% across the tested datasets. Our proposed model addresses the challenge of generalizability in deepfake detection by leveraging visual and latent features and providing an effective solution for identifying a wide range of fake videos while preserving media integrity. The code for GenConViT is available at https://github.com/erprogs/GenConViT.Comment: 11 pages, 4 figure