Image and Video Forensics

Nowadays, images and videos have become the main modalities of information being exchanged in everyday life, and their pervasiveness has led the image forensics community to question their reliability, integrity, confidentiality, and security. Multimedia contents are generated in many different ways through the use of consumer electronics and high-quality digital imaging devices, such as smartphones, digital cameras, tablets, and wearable and IoT devices. The ever-increasing convenience of image acquisition has facilitated instant distribution and sharing of digital images on digital social platforms, determining a great amount of exchange data. Moreover, the pervasiveness of powerful image editing tools has allowed the manipulation of digital images for malicious or criminal ends, up to the creation of synthesized images and videos with the use of deep learning techniques. In response to these threats, the multimedia forensics community has produced major research efforts regarding the identification of the source and the detection of manipulation. In all cases (e.g., forensic investigations, fake news debunking, information warfare, and cyberattacks) where images and videos serve as critical evidence, forensic technologies that help to determine the origin, authenticity, and integrity of multimedia content can become essential tools. This book aims to collect a diverse and complementary set of articles that demonstrate new developments and applications in image and video forensics to tackle new and serious challenges to ensure media authenticity

Noise, artifact and the uncanny in large scale digital photographic practice.

This dissertation explores the question: why, when encountering the products of many new technologies delivering information via a new media, do I often experience a feeling of disquiet or estrangement? I use the example of laser-photographic printing to explore the issue through a program of practice-based research. The outcome of this line of enquiry includes an original contribution via three series of large-format digital photographic works: Presenting "The Amazing Kriels", Home At Last, and Pure. In this thesis, which supports the main body of the research, that is, the practice-based research, I will briefly review the case for artefact as noise within photographic printing, articulate a significant difference between the artefact levels of traditional analogue and Lambda prints, present original dialogical evidence for estrangement in the latter, and identify it via readings of Sigmund Freud's "The Uncanny" and McLuhan's "The Gadget Lover", as a function of the uncanny. I will propose an original rewriting of McLuhan's ideas of "hot" and "cool" media, as well as the cycles of irritation/mediation repression within McLuhan's media theory as a direction for future research, and relate them to a shift from large-scale analogue photographic printing to Lambda printing

Media Forensics and DeepFakes: an overview

With the rapid progress of recent years, techniques that generate and manipulate multimedia content can now guarantee a very advanced level of realism. The boundary between real and synthetic media has become very thin. On the one hand, this opens the door to a series of exciting applications in different fields such as creative arts, advertising, film production, video games. On the other hand, it poses enormous security threats. Software packages freely available on the web allow any individual, without special skills, to create very realistic fake images and videos. So-called deepfakes can be used to manipulate public opinion during elections, commit fraud, discredit or blackmail people. Potential abuses are limited only by human imagination. Therefore, there is an urgent need for automated tools capable of detecting false multimedia content and avoiding the spread of dangerous false information. This review paper aims to present an analysis of the methods for visual media integrity verification, that is, the detection of manipulated images and videos. Special emphasis will be placed on the emerging phenomenon of deepfakes and, from the point of view of the forensic analyst, on modern data-driven forensic methods. The analysis will help to highlight the limits of current forensic tools, the most relevant issues, the upcoming challenges, and suggest future directions for research

Noise, artefact and the uncanny in large scale digital photographic practice

This dissertation explores the question: why, when encountering the products of many new technologies delivering information via a new media, do I often experience a feeling of disquiet or estrangement? I use the example of laser-photographic printing to explore the issue through a program of practice-based research. The outcome of this line of enquiry includes an original contribution via three series of large-format digital photographic works: Presenting "The Amazing Kriels", Home At Last, and Pure. In this thesis, which supports the main body of the research, that is, the practice-based research, I will briefly review the case for artefact as noise within photographic printing, articulate a significant difference between the artefact levels of traditional analogue and Lambda prints, present original dialogical evidence for estrangement in the latter, and identify it via readings of Sigmund Freud's "The Uncanny" and McLuhan's "The Gadget Lover", as a function of the uncanny. I will propose an original rewriting of McLuhan's ideas of "hot" and "cool" media, as well as the cycles of irritation/mediation repression within McLuhan's media theory as a direction for future research, and relate them to a shift from large-scale analogue photographic printing to Lambda printing.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Human-Centric Deep Generative Models: The Blessing and The Curse

Over the past years, deep neural networks have achieved significant progress in a wide range of real-world applications. In particular, my research puts a focused lens in deep generative models, a neural network solution that proves effective in visual (re)creation. But is generative modeling a niche topic that should be researched on its own? My answer is critically no. In the thesis, I present the two sides of deep generative models, their blessing and their curse to human beings. Regarding what can deep generative models do for us, I demonstrate the improvement in performance and steerability of visual (re)creation. Regarding what can we do for deep generative models, my answer is to mitigate the security concerns of DeepFakes and improve minority inclusion of deep generative models. For the performance of deep generative models, I probe on applying attention modules and dual contrastive loss to generative adversarial networks (GANs), which pushes photorealistic image generation to a new state of the art. For the steerability, I introduce Texture Mixer, a simple yet effective approach to achieve steerable texture synthesis and blending. For the security, my research spans over a series of GAN fingerprinting solutions that enable the detection and attribution of GAN-generated image misuse. For the inclusion, I investigate the biased misbehavior of generative models and present my solution in enhancing the minority inclusion of GAN models over underrepresented image attributes. All in all, I propose to project actionable insights to the applications of deep generative models, and finally contribute to human-generator interaction

Gaze-Based Human-Robot Interaction by the Brunswick Model

We present a new paradigm for human-robot interaction based on social signal processing, and in particular on the Brunswick model. Originally, the Brunswick model copes with face-to-face dyadic interaction, assuming that the interactants are communicating through a continuous exchange of non verbal social signals, in addition to the spoken messages. Social signals have to be interpreted, thanks to a proper recognition phase that considers visual and audio information. The Brunswick model allows to quantitatively evaluate the quality of the interaction using statistical tools which measure how effective is the recognition phase. In this paper we cast this theory when one of the interactants is a robot; in this case, the recognition phase performed by the robot and the human have to be revised w.r.t. the original model. The model is applied to Berrick, a recent open-source low-cost robotic head platform, where the gazing is the social signal to be considered