Face recognition technologies for evidential evaluation of video traces

Human recognition from video traces is an important task in forensic investigations and evidence evaluations. Compared with other biometric traits, face is one of the most popularly used modalities for human recognition due to the fact that its collection is non-intrusive and requires less cooperation from the subjects. Moreover, face images taken at a long distance can still provide reasonable resolution, while most biometric modalities, such as iris and fingerprint, do not have this merit. In this chapter, we discuss automatic face recognition technologies for evidential evaluations of video traces. We first introduce the general concepts in both forensic and automatic face recognition , then analyse the difficulties in face recognition from videos . We summarise and categorise the approaches for handling different uncontrollable factors in difficult recognition conditions. Finally we discuss some challenges and trends in face recognition research in both forensics and biometrics . Given its merits tested in many deployed systems and great potential in other emerging applications, considerable research and development efforts are expected to be devoted in face recognition in the near future

Image super-resolution for outdoor digital forensics. Usability and legal aspects

This work was supported in part by the Spanish Ministry of Economy and Competitiveness (MINECO) through projects TIN2013-43880-R and DPI2016-77869-C2-2-R, the Department of Energy under Grant DE-NA0002520, ONR award N00014-15-1-2735, NSF IDEAS program, DARPA ReImagine.Digital Forensics encompasses the recovery and investigation of data, images, and recordings found in digital devices in order to provide evidence in the court of law. This paper is devoted to the assessment of digital evidence which requires not only an understanding of the scientific technique that leads to improved quality of surveillance video recordings, but also of the legal principles behind it. Emphasis is given on the special treatment of image processing in terms of its handling and explanation that would be acceptable in a court of law. In this context, we propose a variational Bayesian approach to multiple- image super-resolution based on Super-Gaussian prior models that automatically enhances the quality of outdoor video recordings and estimates all the model parameters while preserving the authenticity, credibility and reliability of video data as digital evidence. The proposed methodology is validated both quantitatively and visually on synthetic videos generated from single images and real-life videos and applied to a real-life case of damages and stealing in a private property.Spanish Ministry of Economy and Competitiveness (MINECO) TIN2013-43880-R, DPI2016-77869-C2-2-RDepartment of Energy DE-NA0002520, ONR award N00014-15-1-2735, NSF IDEAS program, DARPA ReImagin

Training methods for facial image comparison: a literature review

This literature review was commissioned to explore the psychological literature relating to facial image comparison with a particular emphasis on whether individuals can be trained to improve performance on this task. Surprisingly few studies have addressed this question directly. As a consequence, this review has been extended to cover training of face recognition and training of different kinds of perceptual comparisons where we are of the opinion that the methodologies or findings of such studies are informative. The majority of studies of face processing have examined face recognition, which relies heavily on memory. This may be memory for a face that was learned recently (e.g. minutes or hours previously) or for a face learned longer ago, perhaps after many exposures (e.g. friends, family members, celebrities). Successful face recognition, irrespective of the type of face, relies on the ability to retrieve the to-berecognised face from long-term memory. This memory is then compared to the physically present image to reach a recognition decision. In contrast, in face matching task two physical representations of a face (live, photographs, movies) are compared and so long-term memory is not involved. Because the comparison is between two present stimuli rather than between a present stimulus and a memory, one might expect that face matching, even if not an easy task, would be easier to do and easier to learn than face recognition. In support of this, there is evidence that judgment tasks where a presented stimulus must be judged by a remembered standard are generally more cognitively demanding than judgments that require comparing two presented stimuli Davies & Parasuraman, 1982; Parasuraman & Davies, 1977; Warm and Dember, 1998). Is there enough overlap between face recognition and matching that it is useful to look at the literature recognition? No study has directly compared face recognition and face matching, so we turn to research in which people decided whether two non-face stimuli were the same or different. In these studies, accuracy of comparison is not always better when the comparator is present than when it is remembered. Further, all perceptual factors that were found to affect comparisons of simultaneously presented objects also affected comparisons of successively presented objects in qualitatively the same way. Those studies involved judgments about colour (Newhall, Burnham & Clark, 1957; Romero, Hita & Del Barco, 1986), and shape (Larsen, McIlhagga & Bundesen, 1999; Lawson, Bülthoff & Dumbell, 2003; Quinlan, 1995). Although one must be cautious in generalising from studies of object processing to studies of face processing (see, e.g., section comparing face processing to object processing), from these kinds of studies there is no evidence to suggest that there are qualitative differences in the perceptual aspects of how recognition and matching are done. As a result, this review will include studies of face recognition skill as well as face matching skill. The distinction between face recognition involving memory and face matching not involving memory is clouded in many recognition studies which require observers to decide which of many presented faces matches a remembered face (e.g., eyewitness studies). And of course there are other forensic face-matching tasks that will require comparison to both presented and remembered comparators (e.g., deciding whether any person in a video showing a crowd is the target person). For this reason, too, we choose to include studies of face recognition as well as face matching in our revie

Super-recognisers in Action: Evidence from Face-matching and Face Memory Tasks

Individuals employed in forensic or security settings are often required to compare faces of ID holders to document photographs, or to recognise the faces of suspects in closed-circuit television footage. It has long been established that both tasks produce a high error rate amongst typical perceivers. This study sought to determine the performance of individuals with exceptionally good face memory ('super-recognisers') on applied facial identity matching and memory tasks. In experiment 1, super-recognisers were significantly better than controls when matching target faces to simultaneously presented line-ups. In experiment 2, super-recognisers were also better at recognising faces from video footage. These findings suggest that super-recognisers are more accurate at face matching and face memory tasks than typical perceivers, and they could be valuable expert employees in national security and forensic settings

Face comparison in forensics:A deep dive into deep learning and likelihood rations

This thesis explores the transformative potential of deep learning techniques in the field of forensic face recognition. It aims to address the pivotal question of how deep learning can advance this traditionally manual field, focusing on three key areas: forensic face comparison, face image quality assessment, and likelihood ratio estimation. Using a comparative analysis of open-source automated systems and forensic experts, the study finds that automated systems excel in identifying non-matches in low-quality images, but lag behind experts in high-quality settings. The thesis also investigates the role of calibration methods in estimating likelihood ratios, revealing that quality score-based and feature-based calibrations are more effective than naive methods. To enhance face image quality assessment, a multi-task explainable quality network is proposed that not only gauges image quality, but also identifies contributing factors. Additionally, a novel images-to-video recognition method is introduced to improve the estimation of likelihood ratios in surveillance settings. The study employs multiple datasets and software systems for its evaluations, aiming for a comprehensive analysis that can serve as a cornerstone for future research in forensic face recognition

Face Recognition Methodologies Using Component Analysis: The Contemporary Affirmation of The Recent Literature

This paper explored the contemporary affirmation of the recent literature in the context of face recognition systems, a review motivated by contradictory claims in the literature. This paper shows how the relative performance of recent claims based on methodologies such as PCA and ICA, which are depend on the task statement. It then explores the space of each model acclaimed in recent literature. In the process, this paper verifies the results of many of the face recognition models in the literature, and relates them to each other and to this work

Forgery detection from printed images: a tool in crime scene analysis

.The preliminary analysis of the genuineness of a photo is become, in the time, the first step of any forensic examination that involves images, in case there is not a certainty of its intrinsic authenticity. Digital cameras have largely replaced film based devices, till some years ago, in some areas (countries) just images made from film negatives where considered fully reliable in Court. There was a widespread prejudicial thought regarding a digital image which, according to some people, it cannot ever been considered a legal proof, since its “inconsistent digital nature”. Great efforts have been made by the forensic science community on this field and now, after all this year, different approaches have been unveiled to discover and declare possible malicious frauds, thus to establish whereas an image is authentic or not or, at least, to assess a certain degree of probability of its “pureness”. Nowadays it’s an easy practice to manipulate digital images by using powerful photo editing tools. In order to alter the original meaning of the image, copy-move forgery is the one of the most common ways of manipulating the contents. With this technique a portion of the image is copied and pasted once or more times elsewhere into the same image to hide something or change the real meaning of it. Whenever a digital image (or a printed image) will be presented as an evidence into a Court, it should be followed the criteria to analyze the document with a forensic approach to determine if it contains traces of manipulation. Image forensics literature offers several examples of detectors for such manipulation and, among them, the most recent and effective ones are those based on Zernike moments and those based on Scale Invariant Feature Transform (SIFT). In particular, the capability of SIFT to discover correspondences among similar visual contents allows the forensic analysis to detect even very accurate and realistic copy-move forgeries. In some situation, however, instead of a digital document only its analog version may be available. It is interesting to ask whether it is possible to identify tampering from a printed picture rather than its digital counterpart. Scanned documents or recaptured printed documents by a digital camera are widely used in a number of different scenarios, from medical imaging, law enforcement to banking and daily consumer use. So, in this paper, the problem of identifying copy-move forgery from a printed picture is investigated. The copy-move manipulation is detected by proving the presence of copy-move patches in the scanned image by using the tool, named CADET (Cloned Area DETector), based on our previous methodology which has been adapted in a version tailored for printed image case (e.g. choice of the minimum number of matched keypoints, size of the input image, etc.) In this paper a real case of murder is presented, where an image of a crime scene, submitted as a printed documentary evidence, had been modified by the defense advisors to reject the theory of accusation given by the Prosecutor. The goal of this paper is to experimentally investigate the requirement set under which reliable copy-move forgery detection is possible on printed images, in that way the forgery test is the very first step of an appropriate operational check list manual

Use of electric network frequency presence in video material for time estimation

In this research, the possibility of estimating the time a video was recorded at through electric network frequency is explored by examining various light sources in differentiating circumstances. This research focuses on videos made with smartphones. The smartphone cameras make use of an integrated complementary metal oxide semiconductor sensor. The filmed videos are analyzed using software, which employs a small electric network frequency (ENF) database to determine the time of recording of a video made in experimental circumstances. This research shows that in ideal circumstances, it is possible to determine the time stamp of a video recording made with a smartphone. However, it becomes clear that different light sources greatly influence the outcome. The best results are achieved with Halogen and Incandescent light sources, both of which also seem promising in less ideal circumstances. LED sources do work in ideal circumstances and, however, do not show much success in lesser circumstances. This research further demonstrates that there is potential in using ENF to determine a time stamp of recorded videos and provides validation on prior research on this topic. It proves usable in ideal circumstances with the presence of a clear light source on a white wall. With additional research, it has potential to become a feasible method to use for forensic settings in circumstances that are less ideal

Rectification and Super-Resolution Enhancements for Forensic Text Recognition

[EN] Retrieving text embedded within images is a challenging task in real-world settings. Multiple problems such as low-resolution and the orientation of the text can hinder the extraction of information. These problems are common in environments such as Tor Darknet and Child Sexual Abuse images, where text extraction is crucial in the prevention of illegal activities. In this work, we evaluate eight text recognizers and, to increase the performance of text transcription, we combine these recognizers with rectification networks and super-resolution algorithms. We test our approach on four state-of-the-art and two custom datasets (TOICO-1K and Child Sexual Abuse (CSA)-text, based on text retrieved from Tor Darknet and Child Sexual Exploitation Material, respectively). We obtained a 0.3170 score of correctly recognized words in the TOICO-1K dataset when we combined Deep Convolutional Neural Networks (CNN) and rectification-based recognizers. For the CSA-text dataset, applying resolution enhancements achieved a final score of 0.6960. The highest performance increase was achieved on the ICDAR 2015 dataset, with an improvement of 4.83% when combining the MORAN recognizer and the Residual Dense resolution approach. We conclude that rectification outperforms super-resolution when applied separately, while their combination achieves the best average improvements in the chosen datasets.SIInstituto Nacional de CiberseguridadThis research has been funded with support from the European Commission under the 4NSEEK project with Grant Agreement 821966. This publication reflects the views only of the author, and the European Commission cannot be held responsible for any use that may be made of the information contained therein.This research has been supported by the grant ’Ayudas para la realización de estudios de doctorado en el marco del programa propio de investigación de la Universidad de León Convocatoria 2018’ and by the framework agreement between Universidad de León and INCIBE (Spanish National Cybersecurity Institute) under Addendum 01. We acknowledge NVIDIA Corporation with the donation of the Titan Xp GPU used for this research