Infrared face recognition: a comprehensive review of methodologies and databases

Automatic face recognition is an area with immense practical potential which includes a wide range of commercial and law enforcement applications. Hence it is unsurprising that it continues to be one of the most active research areas of computer vision. Even after over three decades of intense research, the state-of-the-art in face recognition continues to improve, benefitting from advances in a range of different research fields such as image processing, pattern recognition, computer graphics, and physiology. Systems based on visible spectrum images, the most researched face recognition modality, have reached a significant level of maturity with some practical success. However, they continue to face challenges in the presence of illumination, pose and expression changes, as well as facial disguises, all of which can significantly decrease recognition accuracy. Amongst various approaches which have been proposed in an attempt to overcome these limitations, the use of infrared (IR) imaging has emerged as a particularly promising research direction. This paper presents a comprehensive and timely review of the literature on this subject. Our key contributions are: (i) a summary of the inherent properties of infrared imaging which makes this modality promising in the context of face recognition, (ii) a systematic review of the most influential approaches, with a focus on emerging common trends as well as key differences between alternative methodologies, (iii) a description of the main databases of infrared facial images available to the researcher, and lastly (iv) a discussion of the most promising avenues for future research.Comment: Pattern Recognition, 2014. arXiv admin note: substantial text overlap with arXiv:1306.160

Unifying the Visible and Passive Infrared Bands: Homogeneous and Heterogeneous Multi-Spectral Face Recognition

Face biometrics leverages tools and technology in order to automate the identification of individuals. In most cases, biometric face recognition (FR) can be used for forensic purposes, but there remains the issue related to the integration of technology into the legal system of the court. The biggest challenge with the acceptance of the face as a modality used in court is the reliability of such systems under varying pose, illumination and expression, which has been an active and widely explored area of research over the last few decades (e.g. same-spectrum or homogeneous matching). The heterogeneous FR problem, which deals with matching face images from different sensors, should be examined for the benefit of military and law enforcement applications as well. In this work we are concerned primarily with visible band images (380-750 nm) and the infrared (IR) spectrum, which has become an area of growing interest.;For homogeneous FR systems, we formulate and develop an efficient, semi-automated, direct matching-based FR framework, that is designed to operate efficiently when face data is captured using either visible or passive IR sensors. Thus, it can be applied in both daytime and nighttime environments. First, input face images are geometrically normalized using our pre-processing pipeline prior to feature-extraction. Then, face-based features including wrinkles, veins, as well as edges of facial characteristics, are detected and extracted for each operational band (visible, MWIR, and LWIR). Finally, global and local face-based matching is applied, before fusion is performed at the score level. Although this proposed matcher performs well when same-spectrum FR is performed, regardless of spectrum, a challenge exists when cross-spectral FR matching is performed. The second framework is for the heterogeneous FR problem, and deals with the issue of bridging the gap across the visible and passive infrared (MWIR and LWIR) spectrums. Specifically, we investigate the benefits and limitations of using synthesized visible face images from thermal and vice versa, in cross-spectral face recognition systems when utilizing canonical correlation analysis (CCA) and locally linear embedding (LLE), a manifold learning technique for dimensionality reduction. Finally, by conducting an extensive experimental study we establish that the combination of the proposed synthesis and demographic filtering scheme increases system performance in terms of rank-1 identification rate

Face recognition using infrared vision

Au cours de la dernière décennie, la reconnaissance de visage basée sur l’imagerie infrarouge (IR) et en particulier la thermographie IR est devenue une alternative prometteuse aux approches conventionnelles utilisant l’imagerie dans le spectre visible. En effet l’imagerie (visible et infrarouge) trouvent encore des contraintes à leur application efficace dans le monde réel. Bien qu’insensibles à toute variation d’illumination dans le spectre visible, les images IR sont caractérisées par des défis spécifiques qui leur sont propres, notamment la sensibilité aux facteurs qui affectent le rayonnement thermique du visage tels que l’état émotionnel, la température ambiante, la consommation d’alcool, etc. En outre, il est plus laborieux de corriger l’expression du visage et les changements de poses dans les images IR puisque leur contenu est moins riche aux hautes fréquences spatiales ce qui représente en fait une indication importante pour le calage de tout modèle déformable. Dans cette thèse, nous décrivons une nouvelle méthode qui répond à ces défis majeurs. Concrètement, pour remédier aux changements dans les poses et expressions du visage, nous générons une image synthétique frontale du visage qui est canonique et neutre vis-à-vis de toute expression faciale à partir d’une image du visage de pose et expression faciale arbitraires. Ceci est réalisé par l’application d’une déformation affine par morceaux précédée par un calage via un modèle d’apparence active (AAM). Ainsi, une de nos publications est la première publication qui explore l’utilisation d’un AAM sur les images IR thermiques ; nous y proposons une étape de prétraitement qui rehausse la netteté des images thermiques, ce qui rend la convergence de l’AAM rapide et plus précise. Pour surmonter le problème des images IR thermiques par rapport au motif exact du rayonnement thermique du visage, nous le décrivons celui-ci par une représentation s’appuyant sur des caractéristiques anatomiques fiables. Contrairement aux approches existantes, notre représentation n’est pas binaire ; elle met plutôt l’accent sur la fiabilité des caractéristiques extraites. Cela rend la représentation proposée beaucoup plus robuste à la fois à la pose et aux changements possibles de température. L’efficacité de l’approche proposée est démontrée sur la plus grande base de données publique des vidéos IR thermiques des visages. Sur cette base d’images, notre méthode atteint des performances de reconnaissance assez bonnes et surpasse de manière significative les méthodes décrites précédemment dans la littérature. L’approche proposée a également montré de très bonnes performances sur des sous-ensembles de cette base de données que nous avons montée nous-mêmes au sein de notre laboratoire. A notre connaissance, il s’agit de l’une des bases de données les plus importantes disponibles à l’heure actuelle tout en présentant certains défis.Over the course of the last decade, infrared (IR) and particularly thermal IR imaging based face recognition has emerged as a promising complement to conventional, visible spectrum based approaches which continue to struggle when applied in the real world. While inherently insensitive to visible spectrum illumination changes, IR images introduce specific challenges of their own, most notably sensitivity to factors which affect facial heat emission patterns, e.g., emotional state, ambient temperature, etc. In addition, facial expression and pose changes are more difficult to correct in IR images because they are less rich in high frequency details which is an important cue for fitting any deformable model. In this thesis we describe a novel method which addresses these major challenges. Specifically, to normalize for pose and facial expression changes we generate a synthetic frontal image of a face in a canonical, neutral facial expression from an image of the face in an arbitrary pose and facial expression. This is achieved by piecewise affine warping which follows active appearance model (AAM) fitting. This is the first work which explores the use of an AAM on thermal IR images; we propose a pre-processing step which enhances details in thermal images, making AAM convergence faster and more accurate. To overcome the problem of thermal IR image sensitivity to the exact pattern of facial temperature emissions we describe a representation based on reliable anatomical features. In contrast to previous approaches, our representation is not binary; rather, our method accounts for the reliability of the extracted features. This makes the proposed representation much more robust both to pose and scale changes. The effectiveness of the proposed approach is demonstrated on the largest public database of thermal IR images of faces on which it achieves satisfying recognition performance and significantly outperforms previously described methods. The proposed approach has also demonstrated satisfying performance on subsets of the largest video database of the world gathered in our laboratory which will be publicly available free of charge in future. The reader should note that due to the very nature of the feature extraction method in our system (i.e., anatomical based nature of it), we anticipate high robustness of our system to some challenging factors such as the temperature changes. However, we were not able to investigate this in depth due to the limits which exist in gathering realistic databases. Gathering the largest video database considering some challenging factors is one of the other contributions of this research

Evaluation and Hardware Realization for a Face Recognition System

Facial recognition from an image or a video sequence draws attention for many image processing researchers owing to its myriad applications in real world as well as in computer vision, human-computer interaction and intelligent systems. Facial structures have unique features which can be extracted using some mathematical tools. We have used Principal Component Analysis (PCA) and Local Binary Pattern (LBP) to extract them and stored them in a database. When the query image is given the facial features are extracted and compared to the previously obtained results using Sparse Face recognition. Detailed test methods have been defined and an extensive testing of the algorithm has been performed on various standard databases. The results have been tabulated with required graphs. The proposed algorithm has been compared to other different algorithms which show significant improvement in results with small number of training samples. Finally the algorithm was integrated in a hardware system so that it can be used as a self sufficient portable system

Advanced Techniques for Ground Penetrating Radar Imaging

Ground penetrating radar (GPR) has become one of the key technologies in subsurface sensing and, in general, in non-destructive testing (NDT), since it is able to detect both metallic and nonmetallic targets. GPR for NDT has been successfully introduced in a wide range of sectors, such as mining and geology, glaciology, civil engineering and civil works, archaeology, and security and defense. In recent decades, improvements in georeferencing and positioning systems have enabled the introduction of synthetic aperture radar (SAR) techniques in GPR systems, yielding GPR–SAR systems capable of providing high-resolution microwave images. In parallel, the radiofrequency front-end of GPR systems has been optimized in terms of compactness (e.g., smaller Tx/Rx antennas) and cost. These advances, combined with improvements in autonomous platforms, such as unmanned terrestrial and aerial vehicles, have fostered new fields of application for GPR, where fast and reliable detection capabilities are demanded. In addition, processing techniques have been improved, taking advantage of the research conducted in related fields like inverse scattering and imaging. As a result, novel and robust algorithms have been developed for clutter reduction, automatic target recognition, and efficient processing of large sets of measurements to enable real-time imaging, among others. This Special Issue provides an overview of the state of the art in GPR imaging, focusing on the latest advances from both hardware and software perspectives

Monitoring wetlands and water bodies in semi-arid Sub-Saharan regions

Surface water in wetlands is a critical resource in semi-arid West-African regions that are frequently exposed to droughts. Wetlands are of utmost importance for the population as well as the environment, and are subject to rapidly changing seasonal fluctuations. Dynamics of wetlands in the study area are still poorly understood, and the potential of remote sensing-derived information as a large-scale, multi-temporal, comparable and independent measurement source is not exploited. This work shows successful wetland monitoring with remote sensing in savannah and Sahel regions in Burkina Faso, focusing on the main study site Lac Bam (Lake Bam). Long-term optical time series from MODIS with medium spatial resolution (MR), and short-term synthetic aperture radar (SAR) time series from TerraSAR-X and RADARSAT-2 with high spatial resolution (HR) successfully demonstrate the classification and dynamic monitoring of relevant wetland features, e.g. open water, flooded vegetation and irrigated cultivation. Methodological highlights are time series analysis, e.g. spatio-temporal dynamics or multitemporal-classification, as well as polarimetric SAR (polSAR) processing, i.e. the Kennaugh elements, enabling physical interpretation of SAR scattering mechanisms for dual-polarized data. A multi-sensor and multi-frequency SAR data combination provides added value, and reveals that dual-co-pol SAR data is most recommended for monitoring wetlands of this type. The interpretation of environmental or man-made processes such as water areas spreading out further but retreating or evaporating faster, co-occurrence of droughts with surface water and vegetation anomalies, expansion of irrigated agriculture or new dam building, can be detected with MR optical and HR SAR time series. To capture long-term impacts of water extraction, sedimentation and climate change on wetlands, remote sensing solutions are available, and would have great potential to contribute to water management in Africa

Image Restoration

This book represents a sample of recent contributions of researchers all around the world in the field of image restoration. The book consists of 15 chapters organized in three main sections (Theory, Applications, Interdisciplinarity). Topics cover some different aspects of the theory of image restoration, but this book is also an occasion to highlight some new topics of research related to the emergence of some original imaging devices. From this arise some real challenging problems related to image reconstruction/restoration that open the way to some new fundamental scientific questions closely related with the world we interact with

A model for the detection of breast cancer using machine learning and thermal images in a mobile environment

Breast cancer is the most common cancer amongst women and one of the deadliest. Various modalities exist which image the breasts, all with a focus on early detection; thermography is one such method. It is a non-invasive test, which is safe and can be used for a wide variety of breast densities. It functions by analysing thermal patterns captured via an infrared camera of the surface of the breast. Advances in infrared and mobile technology enable this modality to be mobile based; allowing a high degree of portability at a lower cost. Furthermore, as technology has improved, machine learning has played a larger role in medical practices by offering unbiased, consistent, and timely second opinions. Machine learning algorithms are able to classify medical images automatically if offered in the correct format. This study aims to provide a model, which integrates breast cancer detection, thermal imaging, machine learning, and mobile technology. The conceptual model is theorised from three literature studies regarding: identifiable aspects of breast cancer through thermal imaging, the mobile ecosystem, and classification using machine learning algorithms. The model is implemented and evaluated using an experiment designed to classify automatically thermal breast images of the same quality that mobile attachable thermal cameras are able to capture. The experiment contrasts various combinations of segmentation methods, extracted features, and classification algorithms. Promising results were shown in the experiment with a high degree of accuracy obtained. The successful results obtained from the experimentation process validates the feasibility of the model

A model for the detection of breast cancer using machine learning and thermal images in a mobile environment

Breast cancer is the most common cancer amongst women and one of the deadliest. Various modalities exist which image the breasts, all with a focus on early detection; thermography is one such method. It is a non-invasive test, which is safe and can be used for a wide variety of breast densities. It functions by analysing thermal patterns captured via an infrared camera of the surface of the breast. Advances in infrared and mobile technology enable this modality to be mobile based; allowing a high degree of portability at a lower cost. Furthermore, as technology has improved, machine learning has played a larger role in medical practices by offering unbiased, consistent, and timely second opinions. Machine learning algorithms are able to classify medical images automatically if offered in the correct format. This study aims to provide a model, which integrates breast cancer detection, thermal imaging, machine learning, and mobile technology. The conceptual model is theorised from three literature studies regarding: identifiable aspects of breast cancer through thermal imaging, the mobile ecosystem, and classification using machine learning algorithms. The model is implemented and evaluated using an experiment designed to classify automatically thermal breast images of the same quality that mobile attachable thermal cameras are able to capture. The experiment contrasts various combinations of segmentation methods, extracted features, and classification algorithms. Promising results were shown in the experiment with a high degree of accuracy obtained. The successful results obtained from the experimentation process validates the feasibility of the model