Pattern matching of footwear Impressions

One of the most frequently secured types of evidence at crime scenes are footware impressions. Identifying the brand and model of the footware can be crucial to narrowing the search for suspects. This is done by forensic experts by comparing the evidence found at the crime scene with a huge list of reference impressions. In order to support the forensic experts an automatic retrieval of the most likely matches is desired.In this thesis different techniques are evaluated to recognize and match footwear impressions, using reference and real crime scene shoeprint images. Due to the conditions in which the shoeprints are found (partial occlusions, variation in shape) a translation, rotation and scale invariant system is needed. A VLAD (Vector of Locally Aggregated Descriptors) encoder is used to clustering descriptors obtained using different approaches, such as SIFT (Scale-Invariant Feature Transform), Dense SIFT in a Triplet CNN (Convolutional Neural Network). These last two approaches provide the best performance results when the parameters are correctly adjusted, using the Cumulative Matching Characteristic curve to evaluate it.En esta tesis se evalúan diferentes técnicas para reconocer y emparejar impresiones de calzado, utilizando imágenes de referencia y de escenas reales de crimen. Debido a las condiciones en que se encuentran las impresiones (oclusiones parciales, variaciones de forma) se necesita un sistema invariante ante translación, rotación y escalado. Para ello se utiliza un codificador VLAD (Vector of Locally Aggregated Descriptors) para agrupar descriptores obtenidos en diferentes enfoques, como SIFT (Scale-Invariant Feature Transform), Dense SIFT y Triplet CNN (Convolutional Neural Network). Estos dos últimos enfoques proporcionan los mejores resultados una vez los parámetros se han ajustado correctamente, utilizando la curva CMC (Characteristic Matching Curve) para realizar la evaluación.En aquesta tesi s'avaluen diferents tècniques per reconèixer i aparellar impressions de calçat, utilitzant imatges de referència i d'escenes reals de crim. Degut a les condicions en què es troben les impressions (oclusions parcials, variació de forma ) es necessita un sistema invariant davant translació, rotació i escalat. Per això s'utilitza un codificador VLAD (Vector of Locally Aggregated Descriptors) per agrupar descriptors obtinguts en diferents enfocaments, com SIFT (Scale-Invariant Feature Transform), Dense SIFT i Triplet CNN (Convolutional Neural Network). Aquests dos últims enfocaments proporcionen els millors resultats un cop els paràmetres s'han ajustat correctament, utilitzant la corba CMC (Characteristic Matching Curve) per realitzar l'avaluació

An algorithm to compare two‐dimensional footwear outsole images using maximum cliques and speeded‐up robust feature

Footwear examiners are tasked with comparing an outsole impression (Q) left at a crime scene with an impression (K) from a database or from the suspect\u27s shoe. We propose a method for comparing two shoe outsole impressions that relies on robust features (speeded‐up robust feature; SURF) on each impression and aligns them using a maximum clique (MC). After alignment, an algorithm we denote MC‐COMP is used to extract additional features that are then combined into a univariate similarity score using a random forest (RF). We use a database of shoe outsole impressions that includes images from two models of athletic shoes that were purchased new and then worn by study participants for about 6 months. The shoes share class characteristics such as outsole pattern and size, and thus the comparison is challenging. We find that the RF implemented on SURF outperforms other methods recently proposed in the literature in terms of classification precision. In more realistic scenarios where crime scene impressions may be degraded and smudged, the algorithm we propose—denoted MC‐COMP‐SURF—shows the best classification performance by detecting unique features better than other methods. The algorithm can be implemented with the R‐package shoeprintr

Computational Shoeprint Analysis for Forensic Science

Shoeprints are a common type of evidence found at crime scenes and are regularly used in forensic investigations. However, their utility is limited by the lack of reference footwear databases that cover the large and growing number of distinct shoe models. Additionally, existing methods for matching crime-scene shoeprints to reference databases cannot effectively employ deep learning techniques due to a lack of training data. Moreover, these methods typically rely on comparing crime-scene shoeprints with clean reference prints instead of more detailed tread depth maps. To address these challenges, we break down the problem into two parts. First, we leverage shoe tread images sourced from online retailers to predict their corresponding depth maps, which are then thresholded to generate prints, thus constructing a comprehensive reference database. Next, we use a section of this database to train a retrieval network that matches query crime-scene shoeprints to tread depth maps. Extensive experimentation across multiple datasets demonstrates the state-of-the-art performance achieved by both the database creation and retrieval steps, validating the effectiveness of our proposed methodology

Model-based image analysis for forensic shoe print recognition

This thesis is about automated forensic shoe print recognition. Recognizing a shoe print in an image is an inherently difficult task. Shoe prints vary in their pose, shape and appearance. They are surrounded and partially occluded by other objects and may be left on a wide range of diverse surfaces. We propose to formulate this task in a model-based image analysis framework. Our framework is based on the Active Basis Model. A shoe print is represented as hierarchical composition of basis filters. The individual filters encode local information about the geometry and appearance of the shoe print pattern. The hierarchical com- position encodes mid- and long-range geometric properties of the object. A statistical distribution is imposed on the parameters of this representation, in order to account for the variation in a shoe print‘s geometry and appearance. Our work extends the Active Basis Model in various ways, in order to make it robustly applicable to the analysis of shoe print images. We propose an algorithm that automat- ically infers an efficient hierarchical dependency structure between the basis filters. The learned hierarchical dependencies are beneficial for our further extensions, while at the same time permitting an efficient optimization process. We introduce an occlusion model and propose to leverage the hierarchical dependencies to integrate contextual informa- tion efficiently into the reasoning process about occlusions. Finally, we study the effect of the basis filter on the discrimination of the object from the background. In this con- text, we highlight the role of the hierarchical model structure in terms of combining the locally ambiguous filter response into a sophisticated discriminator. The main contribution of this work is a model-based image analysis framework which represents a planar object‘s variation in shape and appearance, it‘s partial occlusion as well as background clutter. The model parameters are optimized jointly in an efficient optimization scheme. Our extensions to the Active Basis Model lead to an improved discriminative ability and permit coherent occlusions and hierarchical deformations. The experimental results demonstrate a new state of the art performance at the task of forensic shoe print recognition

Crowd Abnormal Behaviour Detection and Analysis

The analysis and understanding of abnormal behaviours in human crowds is a challenging task in pattern recognition and computer vision. First of all, the semantic definition of the term “crowd” is ambiguous. Secondly, the taxonomy of crowd behaviours is usually rudimentary and intrinsically complicated. How to identify and construct effective features for crowd behaviour classification is a prominent challenge. Thirdly, the acquisition of suitable video for crowd analysis is another critical problem. In order to address those issues, a categorization model for abnormal behaviour types is defined according to the state-of-the-art. In the novel taxonomy of crowd behaviour, eight types of crowd behaviours are defined based on the key visual patterns. An enhanced social force-based model is proposed to achieve the visual realism in crowd simulation, hence to generate customizable videos for crowd analysis. The proposed model consists of a long-term behavior control model based on A-star path finding algorithm and a short-term interaction handling model based on the enhanced social force. The proposed simulation approach produced all the crowd behaviours in the new taxonomy for the training and testing of the detection procedure. On the aspect of feature engineering, an innovative signature is devised for assisting the segmentation of crowd in both low and high density. The signature is modelled with derived features from Grey-Level Co-occurrence Matrix. Another major breakthrough is an effective approach for efficiently extracting spatial temporal information based on the information entropy theory and Gabor background subtraction. The extraction approach is capable of obtaining the texture with most motion information, which could help the detection approach to achieve the real-time processing. Overall, these contributions have supported the crucial components in a pipeline of abnormal crowd behaviour detecting process. This process is consisted of crowd behaviour taxonomy, crowd video generation, crowd segmentation and crowd abnormal behaviour detection. Experiments for each component show promising results, and proved the accessibility of the proposed approaches

Geotecnologías láser y fotogramétricas aplicadas a la modelización 3D de escenarios complejos en infografía forense

[ES]El estudio de la reconstrucción tridimensional de escenas y objetos para su posterior análisis es un tema objeto de investigación por diferentes disciplinas. Una de las disciplinas en las que se hace necesaria la obtención de modelos 3D es en la ingeniería forense, y más concretamente en el campo de la infografía. La infografía forense es una técnica que permite la reconstrucción virtual de diferentes hechos a través de la informática y el manejo de imágenes digitales. La gran ventaja que ofrecen las geotecnologías láser y fotogramétricas para la modelización de escenarios complejos en infografía forense es que son técnicas no invasivas y no destructivas. Es decir, a través de ellas quedará constancia documental de los indicios y evidencias presentes en el escenario, sin alterar en ningún momento sus posiciones espaciales ni sus propiedades físicas, además de dotar de rigor, exhaustividad y realismo a la reconstrucción del suceso. En esta Tesis Doctoral se ha demostrado que la aplicación de diversas geotecnologías tales como, las cámaras digitales convencionales (incluyendo los propios ¿Smartphones¿), los escáneres ¿Gaming Sensor¿ y los sistemas de cartografiado de interiores móviles (¿Indoor Mapping¿), son idóneas en la inspección ocular del delito para su posterior representación gráfica tridimensional. Más concretamente, en esta Tesis Doctoral se presentan las siguientes contribuciones: - Se propone una solución, basada en la integración de la fotogrametría de rango cercano y la visión computacional, como una alternativa eficiente a la reconstrucción 3D de objetos y escenarios complejos para infografía forense, garantizando flexibilidad (trabajar con cualquier tipo de cámara), automatismo (paso del 2d-imágenes al 3d-nubes de puntos) y calidad (resoluciones superiores a los sistemas láser) en los resultados. - Se desarrolla y valida una solución tecnológica sencilla y de bajo coste basada en los dispositivos activos de escaneado ¿Gaming Sensors¿ que permite el análisis dimensional y el modelado tridimensional de la escena forense a pequeñas distancias. - Se testea y valida un sistema novedoso de cartografiado de espacios interiores mediante láser móvil (indoor mapping), ideal en aquellos escenarios forenses complejos y de grandes dimensiones. - Se avanza en una estrategia que permite progresar en el paso de las nubes de puntos, ya sean láser y/o fotogramétricas, a los modelos CAD (Computer Aided Design), a través de la segmentación de dichas nubes de puntos en base al análisis de componentes principales (PCA-Principal Component Analysis), lo que supone una contribución directa al campo de la infografía forense