10 research outputs found
How Do the Hearts of Deep Fakes Beat? Deep Fake Source Detection via Interpreting Residuals with Biological Signals
Fake portrait video generation techniques have been posing a new threat to
the society with photorealistic deep fakes for political propaganda, celebrity
imitation, forged evidences, and other identity related manipulations.
Following these generation techniques, some detection approaches have also been
proved useful due to their high classification accuracy. Nevertheless, almost
no effort was spent to track down the source of deep fakes. We propose an
approach not only to separate deep fakes from real videos, but also to discover
the specific generative model behind a deep fake. Some pure deep learning based
approaches try to classify deep fakes using CNNs where they actually learn the
residuals of the generator. We believe that these residuals contain more
information and we can reveal these manipulation artifacts by disentangling
them with biological signals. Our key observation yields that the
spatiotemporal patterns in biological signals can be conceived as a
representative projection of residuals. To justify this observation, we extract
PPG cells from real and fake videos and feed these to a state-of-the-art
classification network for detecting the generative model per video. Our
results indicate that our approach can detect fake videos with 97.29% accuracy,
and the source model with 93.39% accuracy.Comment: To be published in the proceedings of 2020 IEEE/IAPR International
Joint Conference on Biometrics (IJCB
Analysis of Score-Level Fusion Rules for Deepfake Detection
Deepfake detection is of fundamental importance to preserve the reliability of multimedia communications. Modern deepfake detection systems are often specialized on one or more types of manipulation but are not able to generalize. On the other hand, when properly designed, ensemble learning and fusion techniques can reduce this issue. In this paper, we exploit the complementarity of different individual classifiers and evaluate which fusion rules are best suited to increase the generalization capacity of modern deepfake detection systems. We also give some insights to designers for selecting the most appropriate approach
FaceForensics++: Learning to Detect Manipulated Facial Images
The rapid progress in synthetic image generation and manipulation has now
come to a point where it raises significant concerns for the implications
towards society. At best, this leads to a loss of trust in digital content, but
could potentially cause further harm by spreading false information or fake
news. This paper examines the realism of state-of-the-art image manipulations,
and how difficult it is to detect them, either automatically or by humans. To
standardize the evaluation of detection methods, we propose an automated
benchmark for facial manipulation detection. In particular, the benchmark is
based on DeepFakes, Face2Face, FaceSwap and NeuralTextures as prominent
representatives for facial manipulations at random compression level and size.
The benchmark is publicly available and contains a hidden test set as well as a
database of over 1.8 million manipulated images. This dataset is over an order
of magnitude larger than comparable, publicly available, forgery datasets.
Based on this data, we performed a thorough analysis of data-driven forgery
detectors. We show that the use of additional domainspecific knowledge improves
forgery detection to unprecedented accuracy, even in the presence of strong
compression, and clearly outperforms human observers.Comment: Video: https://youtu.be/x2g48Q2I2Z
Media Forensics and DeepFakes: an overview
With the rapid progress of recent years, techniques that generate and
manipulate multimedia content can now guarantee a very advanced level of
realism. The boundary between real and synthetic media has become very thin. On
the one hand, this opens the door to a series of exciting applications in
different fields such as creative arts, advertising, film production, video
games. On the other hand, it poses enormous security threats. Software packages
freely available on the web allow any individual, without special skills, to
create very realistic fake images and videos. So-called deepfakes can be used
to manipulate public opinion during elections, commit fraud, discredit or
blackmail people. Potential abuses are limited only by human imagination.
Therefore, there is an urgent need for automated tools capable of detecting
false multimedia content and avoiding the spread of dangerous false
information. This review paper aims to present an analysis of the methods for
visual media integrity verification, that is, the detection of manipulated
images and videos. Special emphasis will be placed on the emerging phenomenon
of deepfakes and, from the point of view of the forensic analyst, on modern
data-driven forensic methods. The analysis will help to highlight the limits of
current forensic tools, the most relevant issues, the upcoming challenges, and
suggest future directions for research