From deepfake to deep useful: risks and opportunities through a systematic literature review

Deepfake videos are defined as a resulting media from the synthesis of different persons images and videos, mostly faces, replacing a real one. The easy spread of such videos leads to elevated misinformation and represents a threat to society and democracy today. The present study aims to collect and analyze the relevant literature through a systematic procedure. We present 27 articles from scientific databases revealing threats to society, democracies, the political life but present as well advantages of this technology in entertainment, gaming, education, and public life. The research indicates high scientific interest in deepfake detection algorithms as well as the ethical aspect of such technology. This article covers the scientific gap since, to the best of our knowledge, this is the first systematic literature review in the field. A discussion has already started among academics and practitioners concerning the spread of fake news. The next step of fake news considers the use of artificial intelligence and machine learning algorithms that create hyper-realistic videos, called deepfake. Deepfake technology has continuously attracted the attention of scholars over the last 3 years more and more. The importance of conducting research in this field derives from the necessity to understand the theory. The first contextual approach is related to the epistemological points of view of the concept. The second one is related to the phenomenological disadvantages of the field. Despite that, the authors will try to focus not only on the disadvantages of the field but also on the positive aspects of the technology.Comment: 7 pages, IADIS International Conference e-Society (2022

Siamese Network-Based Multi-Modal Deepfake Detection

Title from PDF of title page viewed June 26, 2020Thesis advisor: Yugyung LeeVitaIncludes bibliographical references (pages 41-46)Thesis (M.S.)--School of Computing and Engineering. University of Missouri--Kansas City, 2020Deep learning widely applies to solve various problems in healthcare, robotics, and computer vision. Presently, an emerging deep learning application called "deepfake" has raised concerns about the multiple types of security threats that may pose severe harm to personal privacy and public safety. Deep convolutional neural networks like VGGNet and InceptionNet have recently set a proposal for detecting deepfake. The main challenge of these CNN-based algorithms is that they require extensive training datasets and high-end GPU resources. Furthermore, these studies mainly focus on identifying patterns in facial expressions in deepfake, and there are only very few studies on detecting audio fakeness. In this thesis, we propose a novel method for uni-modal or multi-modal deepfake detection with minimum resources. The proposed solution was designed with a Siamese network-based deepfake model with invariant of constructive loss and triplet loss. Contrastive loss uses the trained network's output for a positive example and calculates its distance to an instance of the same class and contrasts it with the range to negative samples. The triplet loss was computed by positioning the baseline that minimizes the distance to positive samples but maximizes the distance to negative samples. To test and validate our proposed model, we report our metrics like similarity score, loss, and accuracy on large-scale DFDC, Faceforensic++, and CelebDF datasets. We compared our method with state-of-the-art algorithms and confirmed that our overall accuracy is improved by 2-3% for deepfake detection.Introduction -- Background and related work -- Proposed framework -- Results and evaluations -- Conclusion, limitation and future wor