Image and Video Forensics

Nowadays, images and videos have become the main modalities of information being exchanged in everyday life, and their pervasiveness has led the image forensics community to question their reliability, integrity, confidentiality, and security. Multimedia contents are generated in many different ways through the use of consumer electronics and high-quality digital imaging devices, such as smartphones, digital cameras, tablets, and wearable and IoT devices. The ever-increasing convenience of image acquisition has facilitated instant distribution and sharing of digital images on digital social platforms, determining a great amount of exchange data. Moreover, the pervasiveness of powerful image editing tools has allowed the manipulation of digital images for malicious or criminal ends, up to the creation of synthesized images and videos with the use of deep learning techniques. In response to these threats, the multimedia forensics community has produced major research efforts regarding the identification of the source and the detection of manipulation. In all cases (e.g., forensic investigations, fake news debunking, information warfare, and cyberattacks) where images and videos serve as critical evidence, forensic technologies that help to determine the origin, authenticity, and integrity of multimedia content can become essential tools. This book aims to collect a diverse and complementary set of articles that demonstrate new developments and applications in image and video forensics to tackle new and serious challenges to ensure media authenticity

Validation of Fast Spectrochemical Screening Methods for the Identification of Counterfeit Pharmaceutical Packaging

Counterfeit pharmaceuticals are an actively developing health and economic threat worldwide. Particularly prevalent are counterfeit pharmaceuticals distributed in emerging nations and through internet pharmacies or e-pharmacies. Although technology has been developed that discourages anti-counterfeiting practices (such as optically variable devices, invisible ink, and track-and-trace technology), it remains somewhat novel and expensive to implement on a widespread scale. In this study, Laser Induced Breakdown Spectroscopy (LIBS) and Attenuated Total Reflectance-Fourier Transform Infrared Spectroscopy (ATR-FTIR) were proposed as fast and non-invasive tools for the identification of counterfeit pharmaceutical packages. The main objective of this research was to develop and evaluate the capabilities of LIBS and ATR-FTIR to determine chemical differences between counterfeit and authentic pharmaceutical packaging samples. LIBS and ATR-FTIR possess several characteristics that render them suitable for rapid on-site detection. They produce analytical results in less than one minute per sample, with high sensitivity and selectivity, limited sample preparation, and minimal destructivity. The methods were evaluated through the analysis of a dataset of 166 packages (112 counterfeits and 54 authentic sources). The dataset was divided into two main subsets. The first subset was evaluated to identify the informative value of LIBS for fast screening of black barcodes and the carton substrate (100 counterfeit and 35 authentic). The multi-color inks and paper of the second subset was investigated for variation of chemical profiles within and between sources, and the method’s capabilities to distinguish between counterfeits (112) and authentic samples (12). One hundred and twelve counterfeit pharmaceutical cartons were printed from five different sources, mimicking six authentic counterparts. The authentic subset consisted of twelve secondary packages of six common medical products, including packages from the same and different manufacturing lots. The selected products consisted of vasodilators, antivirals, steroids, and other commonly counterfeited pharmaceuticals. Intra-source variation of the counterfeit subset was investigated; it was determined to be sufficiently lower than inter-source variation. False exclusion rates were calculated to be less than 20% for samples originating from the same source (e.g., same package, intra-lots, replicate printouts). Using LIBS, a two-class classification system was used for the combined black barcode ink and paperboard carton spectra (n = 135, 100 counterfeit, 35 authentic packages). As black barcode ink is very common on pharmaceutical packaging, this system was used as a general screening technique to quickly identify a sample as authentic or counterfeit, regardless of counterfeit printing source. In general, the correct classification rates for this set were over 92%. The classification models were established using six machine learning methods: Random Forest, Naïve Bayes, Neural Networks, k-Nearest Neighbors, Quadratic Discriminant Analysis, and Linear Discriminant Analysis. A random split of 60% and 40% of the dataset was applied for training and testing of the classifier algorithms. Principal Component Analysis (PCA) was utilized on the LIBS and ATR-FTIR data for variable reduction purposes. The principal components for each ink type were combined prior to classification. Also, a six-class system was also used to classify the dataset using LIBS, ATR-FTIR, and combined data from both techniques (n = 124, 112 counterfeit, 12 authentic packages). The machine learning methods classified the samples as belonging to one of five counterfeit printing sources or their corresponding authentic counterpart. Seven ink colors (red, blue, yellow, green, brown, pink, black) were analyzed; additionally, in ATR-FTIR, the paperboard substrate was also analyzed. In most comparisons, LIBS had a successful classification rate of over 70% and ATR-FTIR had a successful classification rate of over 85%. When the data from both techniques were combined, the discrimination power of the system increased to 93% correct classification. Although LIBS and ATR-FTIR had a low misclassification rate when used in isolation, the misclassification rate could be reduced even further through data combination. The results of this study are encouraging for the inclusion of LIBS and ATR-FTIR as a screening method for the detection of counterfeit pharmaceutical packaging. The utilization of combined data to discover chemical signatures addresses an urgent need in the investigation of counterfeit pharmaceuticals. Also, the classification of counterfeit samples into their specific counterfeit source may benefit investigators as they make determinations in the counterfeit pharmaceutical packaging supply chain. This study is anticipated to offer relevant tools to both government and pharmaceutical industry in the detection and fight against counterfeit pharmaceuticals

Image statistical frameworks for digital image forensics

The advances of digital cameras, scanners, printers, image editing tools, smartphones, tablet personal computers as well as high-speed networks have made a digital image a conventional medium for visual information. Creation, duplication, distribution, or tampering of such a medium can be easily done, which calls for the necessity to be able to trace back the authenticity or history of the medium. Digital image forensics is an emerging research area that aims to resolve the imposed problem and has grown in popularity over the past decade. On the other hand, anti-forensics has emerged over the past few years as a relatively new branch of research, aiming at revealing the weakness of the forensic technology. These two sides of research move digital image forensic technologies to the next higher level. Three major contributions are presented in this dissertation as follows. First, an effective multi-resolution image statistical framework for digital image forensics of passive-blind nature is presented in the frequency domain. The image statistical framework is generated by applying Markovian rake transform to image luminance component. Markovian rake transform is the applications of Markov process to difference arrays which are derived from the quantized block discrete cosine transform 2-D arrays with multiple block sizes. The efficacy and universality of the framework is then evaluated in two major applications of digital image forensics: 1) digital image tampering detection; 2) classification of computer graphics and photographic images. Second, a simple yet effective anti-forensic scheme is proposed, capable of obfuscating double JPEG compression artifacts, which may vital information for image forensics, for instance, digital image tampering detection. Shrink-and-zoom (SAZ) attack, the proposed scheme, is simply based on image resizing and bilinear interpolation. The effectiveness of SAZ has been evaluated over two promising double JPEG compression schemes and the outcome reveals that the proposed scheme is effective, especially in the cases that the first quality factor is lower than the second quality factor. Third, an advanced textural image statistical framework in the spatial domain is proposed, utilizing local binary pattern (LBP) schemes to model local image statistics on various kinds of residual images including higher-order ones. The proposed framework can be implemented either in single- or multi-resolution setting depending on the nature of application of interest. The efficacy of the proposed framework is evaluated on two forensic applications: 1) steganalysis with emphasis on HUGO (Highly Undetectable Steganography), an advanced steganographic scheme embedding hidden data in a content-adaptive manner locally into some image regions which are difficult for modeling image statics; 2) image recapture detection (IRD). The outcomes of the evaluations suggest that the proposed framework is effective, not only for detecting local changes which is in line with the nature of HUGO, but also for detecting global difference (the nature of IRD)

De-Duplication of Person's Identity Using Multi-Modal Biometrics

The objective of this work is to explore approaches to create unique identities by the de-duplication process using multi-modal biometrics. Various government sectors in the world provide different services and welfare schemes for the beneffit of the people in the society using an identity number. A unique identity (UID) number assigned for every person would obviate the need for a person to produce multiple documentary proofs of his/her identity for availing any government/private services. In the process of creating unique identity of a person, there is a possibility of duplicate identities as the same person might want to get multiple identities in order to get extra beneffits from the Government. These duplicate identities can be eliminated by the de-duplication process using multi-modal biometrics, namely, iris, ngerprint, face and signature. De-duplication is the process of removing instances of multiple enrollments of the same person using the person's biometric data. As the number of people enrolledinto the biometric system runs into billions, the time complexity increases in the de duplication process. In this thesis, three different case studies are presented to address the performance issues of de-duplication process in order to create unique identity of a person

Análise de propriedades intrínsecas e extrínsecas de amostras biométricas para detecção de ataques de apresentação

Orientadores: Anderson de Rezende Rocha, Hélio PedriniTese (doutorado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Os recentes avanços nas áreas de pesquisa em biometria, forense e segurança da informação trouxeram importantes melhorias na eficácia dos sistemas de reconhecimento biométricos. No entanto, um desafio ainda em aberto é a vulnerabilidade de tais sistemas contra ataques de apresentação, nos quais os usuários impostores criam amostras sintéticas, a partir das informações biométricas originais de um usuário legítimo, e as apresentam ao sensor de aquisição procurando se autenticar como um usuário válido. Dependendo da modalidade biométrica, os tipos de ataque variam de acordo com o tipo de material usado para construir as amostras sintéticas. Por exemplo, em biometria facial, uma tentativa de ataque é caracterizada quando um usuário impostor apresenta ao sensor de aquisição uma fotografia, um vídeo digital ou uma máscara 3D com as informações faciais de um usuário-alvo. Em sistemas de biometria baseados em íris, os ataques de apresentação podem ser realizados com fotografias impressas ou com lentes de contato contendo os padrões de íris de um usuário-alvo ou mesmo padrões de textura sintéticas. Nos sistemas biométricos de impressão digital, os usuários impostores podem enganar o sensor biométrico usando réplicas dos padrões de impressão digital construídas com materiais sintéticos, como látex, massa de modelar, silicone, entre outros. Esta pesquisa teve como objetivo o desenvolvimento de soluções para detecção de ataques de apresentação considerando os sistemas biométricos faciais, de íris e de impressão digital. As linhas de investigação apresentadas nesta tese incluem o desenvolvimento de representações baseadas nas informações espaciais, temporais e espectrais da assinatura de ruído; em propriedades intrínsecas das amostras biométricas (e.g., mapas de albedo, de reflectância e de profundidade) e em técnicas de aprendizagem supervisionada de características. Os principais resultados e contribuições apresentadas nesta tese incluem: a criação de um grande conjunto de dados publicamente disponível contendo aproximadamente 17K videos de simulações de ataques de apresentações e de acessos genuínos em um sistema biométrico facial, os quais foram coletados com a autorização do Comitê de Ética em Pesquisa da Unicamp; o desenvolvimento de novas abordagens para modelagem e análise de propriedades extrínsecas das amostras biométricas relacionadas aos artefatos que são adicionados durante a fabricação das amostras sintéticas e sua captura pelo sensor de aquisição, cujos resultados de desempenho foram superiores a diversos métodos propostos na literature que se utilizam de métodos tradicionais de análise de images (e.g., análise de textura); a investigação de uma abordagem baseada na análise de propriedades intrínsecas das faces, estimadas a partir da informação de sombras presentes em sua superfície; e, por fim, a investigação de diferentes abordagens baseadas em redes neurais convolucionais para o aprendizado automático de características relacionadas ao nosso problema, cujos resultados foram superiores ou competitivos aos métodos considerados estado da arte para as diferentes modalidades biométricas consideradas nesta tese. A pesquisa também considerou o projeto de eficientes redes neurais com arquiteturas rasas capazes de aprender características relacionadas ao nosso problema a partir de pequenos conjuntos de dados disponíveis para o desenvolvimento e a avaliação de soluções para a detecção de ataques de apresentaçãoAbstract: Recent advances in biometrics, information forensics, and security have improved the recognition effectiveness of biometric systems. However, an ever-growing challenge is the vulnerability of such systems against presentation attacks, in which impostor users create synthetic samples from the original biometric information of a legitimate user and show them to the acquisition sensor seeking to authenticate themselves as legitimate users. Depending on the trait used by the biometric authentication, the attack types vary with the type of material used to build the synthetic samples. For instance, in facial biometric systems, an attempted attack is characterized by the type of material the impostor uses such as a photograph, a digital video, or a 3D mask with the facial information of a target user. In iris-based biometrics, presentation attacks can be accomplished with printout photographs or with contact lenses containing the iris patterns of a target user or even synthetic texture patterns. In fingerprint biometric systems, impostor users can deceive the authentication process using replicas of the fingerprint patterns built with synthetic materials such as latex, play-doh, silicone, among others. This research aimed at developing presentation attack detection (PAD) solutions whose objective is to detect attempted attacks considering different attack types, in each modality. The lines of investigation presented in this thesis aimed at devising and developing representations based on spatial, temporal and spectral information from noise signature, intrinsic properties of the biometric data (e.g., albedo, reflectance, and depth maps), and supervised feature learning techniques, taking into account different testing scenarios including cross-sensor, intra-, and inter-dataset scenarios. The main findings and contributions presented in this thesis include: the creation of a large and publicly available benchmark containing 17K videos of presentation attacks and bona-fide presentations simulations in a facial biometric system, whose collect were formally authorized by the Research Ethics Committee at Unicamp; the development of novel approaches to modeling and analysis of extrinsic properties of biometric samples related to artifacts added during the manufacturing of the synthetic samples and their capture by the acquisition sensor, whose results were superior to several approaches published in the literature that use traditional methods for image analysis (e.g., texture-based analysis); the investigation of an approach based on the analysis of intrinsic properties of faces, estimated from the information of shadows present on their surface; and the investigation of different approaches to automatically learning representations related to our problem, whose results were superior or competitive to state-of-the-art methods for the biometric modalities considered in this thesis. We also considered in this research the design of efficient neural networks with shallow architectures capable of learning characteristics related to our problem from small sets of data available to develop and evaluate PAD solutionsDoutoradoCiência da ComputaçãoDoutor em Ciência da Computação140069/2016-0 CNPq, 142110/2017-5CAPESCNP