Image and Video Forensics

Nowadays, images and videos have become the main modalities of information being exchanged in everyday life, and their pervasiveness has led the image forensics community to question their reliability, integrity, confidentiality, and security. Multimedia contents are generated in many different ways through the use of consumer electronics and high-quality digital imaging devices, such as smartphones, digital cameras, tablets, and wearable and IoT devices. The ever-increasing convenience of image acquisition has facilitated instant distribution and sharing of digital images on digital social platforms, determining a great amount of exchange data. Moreover, the pervasiveness of powerful image editing tools has allowed the manipulation of digital images for malicious or criminal ends, up to the creation of synthesized images and videos with the use of deep learning techniques. In response to these threats, the multimedia forensics community has produced major research efforts regarding the identification of the source and the detection of manipulation. In all cases (e.g., forensic investigations, fake news debunking, information warfare, and cyberattacks) where images and videos serve as critical evidence, forensic technologies that help to determine the origin, authenticity, and integrity of multimedia content can become essential tools. This book aims to collect a diverse and complementary set of articles that demonstrate new developments and applications in image and video forensics to tackle new and serious challenges to ensure media authenticity

A New Forged Handwriting Detection Method Based on Fourier Spectral Density and Variation

Use of handwriting words for person identification in contrast to biometric features is gaining importance in the field of forensic applications. As a result, forging handwriting is a part of crime applications and hence is challenging for the researchers. This paper presents a new work for detecting forged handwriting words because width and amplitude of spectral distributions have the ability to exhibit unique properties for forged handwriting words compared to blurred, noisy and normal handwriting words. The proposed method studies spectral density and variation of input handwriting images through clustering of high and low frequency coefficients. The extracted features, which are invariant to rotation and scaling, are passed to a neural network classifier for the classification for forged handwriting words from other types of handwriting words (like blurred, noisy and normal handwriting words). Experimental results on our own dataset, which consists of four handwriting word classes, and two benchmark datasets, namely, caption and scene text classification and forged IMEI number dataset, show that the proposed method outperforms the existing methods in terms of classification rate

Receipt Dataset for Fraud Detection

International audienceThe aim of this paper is to introduce a new dataset initially created to work on fraud detection in documents. This dataset is composed of 1969 images of receipts and the associated OCR result for each. The article details the dataset and its interest for the document analysis community. We indeed share this dataset with the community as a benchmark for the evaluation of fraud detection approaches

AirCode: Unobtrusive Physical Tags for Digital Fabrication

We present AirCode, a technique that allows the user to tag physically fabricated objects with given information. An AirCode tag consists of a group of carefully designed air pockets placed beneath the object surface. These air pockets are easily produced during the fabrication process of the object, without any additional material or postprocessing. Meanwhile, the air pockets affect only the scattering light transport under the surface, and thus are hard to notice to our naked eyes. But, by using a computational imaging method, the tags become detectable. We present a tool that automates the design of air pockets for the user to encode information. AirCode system also allows the user to retrieve the information from captured images via a robust decoding algorithm. We demonstrate our tagging technique with applications for metadata embedding, robotic grasping, as well as conveying object affordances.Comment: ACM UIST 2017 Technical Paper

Intrusion Detection for Cyber-Physical Attacks in Cyber-Manufacturing System

In the vision of Cyber-Manufacturing System (CMS) , the physical components such as products, machines, and tools are connected, identifiable and can communicate via the industrial network and the Internet. This integration of connectivity enables manufacturing systems access to computational resources, such as cloud computing, digital twin, and blockchain. The connected manufacturing systems are expected to be more efficient, sustainable and cost-effective. However, the extensive connectivity also increases the vulnerability of physical components. The attack surface of a connected manufacturing environment is greatly enlarged. Machines, products and tools could be targeted by cyber-physical attacks via the network. Among many emerging security concerns, this research focuses on the intrusion detection of cyber-physical attacks. The Intrusion Detection System (IDS) is used to monitor cyber-attacks in the computer security domain. For cyber-physical attacks, however, there is limited work. Currently, the IDS cannot effectively address cyber-physical attacks in manufacturing system: (i) the IDS takes time to reveal true alarms, sometimes over months; (ii) manufacturing production life-cycle is shorter than the detection period, which can cause physical consequences such as defective products and equipment damage; (iii) the increasing complexity of network will also make the detection period even longer. This gap leaves the cyber-physical attacks in manufacturing to cause issues like over-wearing, breakage, defects or any other changes that the original design didn’t intend. A review on the history of cyber-physical attacks, and available detection methods are presented. The detection methods are reviewed in terms of intrusion detection algorithms, and alert correlation methods. The attacks are further broken down into a taxonomy covering four dimensions with over thirty attack scenarios to comprehensively study and simulate cyber-physical attacks. A new intrusion detection and correlation method was proposed to address the cyber-physical attacks in CMS. The detection method incorporates IDS software in cyber domain and machine learning analysis in physical domain. The correlation relies on a new similarity-based cyber-physical alert correlation method. Four experimental case studies were used to validate the proposed method. Each case study focused on different aspects of correlation method performance. The experiments were conducted on a security-oriented manufacturing testbed established for this research at Syracuse University. The results showed the proposed intrusion detection and alert correlation method can effectively disclose unknown attack, known attack and attack interference that causes false alarms. In case study one, the alarm reduction rate reached 99.1%, with improvement of detection accuracy from 49.6% to 100%. The case studies also proved the proposed method can mitigate false alarms, detect attacks on multiple machines, and attacks from the supply chain. This work contributes to the security domain in cyber-physical manufacturing systems, with the focus on intrusion detection. The dataset collected during the experiments has been shared with the research community. The alert correlation methodology also contributes to cyber-physical systems, such as smart grid and connected vehicles, which requires enhanced security protection in today’s connected world

Profiling and imaging of forensic evidence – a pan-European forensic round robin study part 1: document forgery

The forensic scenario, on which the round robin study was based, simulated a suspected intentional manipulation of a real estate rental agreement consisting of a total of three pages. The aims of this study were to (i) establish the amount and reliability of information extractable from a single type of evidence and to (ii) provide suggestions on the most suitable combination of compatible techniques for a multi-modal imaging approach to forgery detection. To address these aims, seventeen laboratories from sixteen countries were invited to answer the following tasks questions: (i) which printing technique was used? (ii) were the three pages printed with the same printer? (iii) were the three pages made from the same paper? (iv) were the three pages originally stapled? (v) were the headings and signatures written with the same ink? and (vi) were headings and signatures of the same age on all pages? The methods used were classified into the following categories: Optical spectroscopy, including multispectral imaging, smartphone mapping, UV-luminescence and LIBS; Infrared spectroscopy, including Raman and FTIR (micro-)spectroscopy; X-ray spectroscopy, including SEM-EDX, PIXE and XPS; Mass spectrometry, including ICPMS, SIMS, MALDI and LDIMS; Electrostatic imaging, as well as non-imaging methods, such as non-multimodal visual inspection, (micro-)spectroscopy, physical testing and thin layer chromatography. The performance of the techniques was evaluated as the proportion of discriminated sample pairs to all possible sample pairs. For the undiscriminated sample pairs, a distinction was made between undecidability and false positive claims. It was found that none of the methods used were able to solve all tasks completely and/or correctly and that certain methods were a priori judged unsuitable by the laboratories for some tasks. Correct results were generally achieved for the discrimination of printer toners, whereas incorrect results in the discrimination of inks. For the discrimination of paper, solid state analytical methods proved to be superior to mass spectrometric methods. None of the participating laboratories deemed addressing ink age feasible. It was concluded that correct forensic statements can only be achieved by the complementary application of different methods and that the classical approach of round robin studies to send standardised subsamples to the participants is not feasible for a true multimodal approach if the techniques are not available at one location

Um método supervisionado para encontrar variáveis discriminantes na análise de problemas complexos : estudos de caso em segurança do Android e em atribuição de impressora fonte

Orientadores: Ricardo Dahab, Anderson de Rezende RochaDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: A solução de problemas onde muitos componentes atuam e interagem simultaneamente requer modelos de representação nem sempre tratáveis pelos métodos analíticos tradicionais. Embora em muitos caso se possa prever o resultado com excelente precisão através de algoritmos de aprendizagem de máquina, a interpretação do fenómeno requer o entendimento de quais são e em que proporção atuam as variáveis mais importantes do processo. Esta dissertação apresenta a aplicação de um método onde as variáveis discriminantes são identificadas através de um processo iterativo de ranqueamento ("ranking") por eliminação das que menos contribuem para o resultado, avaliando-se em cada etapa o impacto da redução de características nas métricas de acerto. O algoritmo de florestas de decisão ("Random Forest") é utilizado para a classificação e sua propriedade de importância das características ("Feature Importance") para o ranqueamento. Para a validação do método, dois trabalhos abordando sistemas complexos de natureza diferente foram realizados dando origem aos artigos aqui apresentados. O primeiro versa sobre a análise das relações entre programas maliciosos ("malware") e os recursos requisitados pelos mesmos dentro de um ecossistema de aplicações no sistema operacional Android. Para realizar esse estudo, foram capturados dados, estruturados segundo uma ontologia definida no próprio artigo (OntoPermEco), de 4.570 aplicações (2.150 malware, 2.420 benignas). O modelo complexo produziu um grafo com cerca de 55.000 nós e 120.000 arestas, o qual foi transformado usando-se a técnica de bolsa de grafos ("Bag Of Graphs") em vetores de características de cada aplicação com 8.950 elementos. Utilizando-se apenas os dados do manifesto atingiu-se com esse modelo 88% de acurácia e 91% de precisão na previsão do comportamento malicioso ou não de uma aplicação, e o método proposto foi capaz de identificar 24 características relevantes na classificação e identificação de famílias de malwares, correspondendo a 70 nós no grafo do ecosistema. O segundo artigo versa sobre a identificação de regiões em um documento impresso que contém informações relevantes na atribuição da impressora laser que o imprimiu. O método de identificação de variáveis discriminantes foi aplicado sobre vetores obtidos a partir do uso do descritor de texturas (CTGF-"Convolutional Texture Gradient Filter") sobre a imagem scaneada em 600 DPI de 1.200 documentos impressos em 10 impressoras. A acurácia e precisão médias obtidas no processo de atribuição foram de 95,6% e 93,9% respectivamente. Após a atribuição da impressora origem a cada documento, 8 das 10 impressoras permitiram a identificação de variáveis discriminantes associadas univocamente a cada uma delas, podendo-se então visualizar na imagem do documento as regiões de interesse para uma análise pericial. Os objetivos propostos foram atingidos mostrando-se a eficácia do método proposto na análise de dois problemas em áreas diferentes (segurança de aplicações e forense digital) com modelos complexos e estruturas de representação bastante diferentes, obtendo-se um modelo reduzido interpretável para ambas as situaçõesAbstract: Solving a problem where many components interact and affect results simultaneously requires models which sometimes are not treatable by traditional analytic methods. Although in many cases the result is predicted with excellent accuracy through machine learning algorithms, the interpretation of the phenomenon requires the understanding of how the most relevant variables contribute to the results. This dissertation presents an applied method where the discriminant variables are identified through an iterative ranking process. In each iteration, a classifier is trained and validated discarding variables that least contribute to the result and evaluating in each stage the impact of this reduction in the classification metrics. Classification uses the Random Forest algorithm, and the discarding decision applies using its feature importance property. The method handled two works approaching complex systems of different nature giving rise to the articles presented here. The first article deals with the analysis of the relations between \textit{malware} and the operating system resources requested by them within an ecosystem of Android applications. Data structured according to an ontology defined in the article (OntoPermEco) were captured to carry out this study from 4,570 applications (2,150 malware, 2,420 benign). The complex model produced a graph of about 55,000 nodes and 120,000 edges, which was transformed using the Bag of Graphs technique into feature vectors of each application with 8,950 elements. The work accomplished 88% of accuracy and 91% of precision in predicting malicious behavior (or not) for an application using only the data available in the application¿s manifest, and the proposed method was able to identify 24 relevant features corresponding to only 70 nodes of the entire ecosystem graph. The second article is about to identify regions in a printed document that contains information relevant to the attribution of the laser printer that printed it. The discriminant variable determination method achieved average accuracy and precision of 95.6% and 93.9% respectively in the source printer attribution using a dataset of 1,200 documents printed on ten printers. Feature vectors were obtained from the scanned image at 600 DPI applying the texture descriptor Convolutional Texture Gradient Filter (CTGF). After the assignment of the source printer to each document, eight of the ten printers allowed the identification of discriminant variables univocally associated to each one of them, and it was possible to visualize in document's image the regions of interest for expert analysis. The work in both articles accomplished the objective of reducing a complex system into an interpretable streamlined model demonstrating the effectiveness of the proposed method in the analysis of two problems in different areas (application security and digital forensics) with complex models and entirely different representation structuresMestradoCiência da ComputaçãoMestre em Ciência da Computaçã