Near-infrared Image Based Face Recognition

Tez (Yüksek Lisans) -- İstanbul Teknik Üniversitesi, Fen Bilimleri Enstitüsü, 2012Thesis (M.Sc.) -- İstanbul Technical University, Institute of Science and Technology, 2012İnsanların yüzleri hatırlama, tanıma ve ayrıştırma yetenekleri doğuştandır. Yüz tanıma alanındaki gelişmeler ve çeşitli ticari yüz tanıma uygulamaları birbirlerine paralel ilerlemiştir. Yine de, daha hatasız ve doğru sistemlere olan ihtiyaç devam etmektedir. Yüz tanımanın kullanıldığı bazı uygulama örnekleri aşağıdaki gibidir:  Yüz tabanlı video dizinleme ve arama motorları  Multimedya yönetimi  İnsan-bilgisayar etkileşimi  Biyometrik kimlik tanıma  Takip sistemleri Yüz tanımada işbirliği içinde ve işbirliği etmeyen olarak iki tip senaryo bulunmaktadır. Takip sistemleri, işbirliği etmeyen kullanıcı uygulamaları için iyi bir örnektir. İşbirliği içinde olan kullanıcı uygulamalarına da geçiş control makinelerinde okunabilen seyahat dökümanları, ATM, bilgisayarda oturum açma, e-ticaret ve e-devlet uygulamaları örnek verilebilir. Kullanıcının sistemle işbirliği içinde olduğu uygulamalarda, sistemin kabulu için yüzün kameraya uygun bir şekilde konumlandırıldıktan sonra yüz resminin elde edilmelidir. Aslında çoğu yüz tanıma sistemleri bu tip uygulamalar için geliştirilmiştir. Yüze ait iç ve dış faktörler yüz tanıma işleminin performansını etkilemektedir. Yüz tanıma yüz yüzeyinin 3D şekil yansıması gibi, yalnızca yüze ait iç faktörlere dayandırılmalıdır. Dış faktörler gözlük, saç modeli, yüz ifadesi, poz ve çevresel ışıklandırma gibi özellikleri içerir. Güvenilir bir yüz tanıma için etkileri en aza indirgenmelidir. Biyometrik bir sistem çevreye uyum sağlamalıdır, bu durumun tam tersi düşünülemez. Çeşitli dış faktörlerin arasından kontrolsüz çevresel ışıklandırma en önemli konudur. Işıklandırma koşulları, özellikle ışığın açısı, yüzün görünümünü öyle çok değiştirmektedir ki; farklı ışıklandırma altında aynı kişiye ait görüntüler ile aynı ışıklandırma altında iki ayrı kişiye ait görüntüler arasında hesaplanan farklılık daha fazladır. Üzerinde çalışılmakta olan bölgesel filtrelerin çoğu kendi başlarına ışıklandırma yönünün sebep olduğu değişimlerin üstesinden gelmekte yetersizdir. Bu sebeple, yakın kızılötesi görüntüleme önerilmiştir. Son zamanlarda, görünür spectrum ardı görüntüleme üzerine çalışmalar yürütülmektedir. Ancak, termal görüntülemenin üstünlükleri yanısıra birçok dezavantajı vardır. Çevresel sıcaklık, fiziksel ve duygusal durum, alkol alımı sistemin başarısını çok fazla etkilemektedir. Çalışmalar, thermal görüntüleme ile yapılan tanıma işlemlerinin, görünür ışık tabanlı görüntüleme işlemlerinden daha iyi bir performans sergilemediklerini göstermiştir. 3D görüntüleme de kullanılan yöntemler arasındadır; fakat işlem yükü, görüntüleme sırasında gözlük takılması veya ağzın açık olma durumu sistemi başarısız kılabilir. Yakın kızıl-ötesi için aktif ışıklandırmada dikkat edilmesi gereken iki önemli husus vardır. • Işıklar net bir önden aydınlatılmış yüz resmi sağlayacak şiddette olmalı; fakat göze rahatsızlık vermemelidir. • Elde edilen yüz resmi çevresel ışıklandırmadan minimum derecede etkilenmiş olmalıdır. Bu çalışmada, diğer metotlarla karşılaştırma amacıyla, yakın kızılötesi (YKÖ) imajları üzerinde öncelikle PCA, LDA ve LBP gibi geleneksel yüz tanıma metotları uygulanmıştır. Eigenfaces yaklaşımında, “öz yüzler” PCA yardımıyla yüz imajlarından oluşturulmuştur. PCA’in amacı yüksek boyutlu veri uzayını, daha az boyuta sahip içsel özellik uzayına dönüştürmektir. LDA’in PCA’den sonra uygulandığı Fisherfaces yaklaşımında, projeksiyon yönü bulunur böylece farklı id’li, farklı sınıflara ait imajlar azami ölçüde ayrıştırılacaktır. Diğer bir deyişle, sınıflar arası dağılım matrisi ve sınıf içi dağılım matrisi oranını maksimum yapan projeksiyon matrisi bulunur. Gabor ve LBP gibi yerel görüntü temsilleri ile ilgili çalışmalar da merak uyandırmaktadır. Başarılı bir yüz tanıma için yüzün dışsal özellikleri ile uğraşmak önemli bir konudur. LBP doku operatörü, ışıklandırma gibi özellikler nedeniyle oluşan değişimlerle başa çıkabilmektedir; bu yüzden çeşitli uygulamalarda popular bir yaklaşım haline gelmiştir. Kapalı mekan için yapılan yüz tanıma uygulamalarında, ışıklandırma bağımsız yüz temsilinde, gri tonlamadaki monotonik dönüşümün serbestlik derecesini telafi etmek amacıyla LBP gösterimi kullanılmaktadır. İmaja ait pixeller, komşu piksellerin eşik değeri olarak ilgili pikselle karşılaştırılması ile 0 veya 1 olarak etiketlenir. LBP operatörü, tamsayı olmayan piksel koordinatlarında çift doğrusal interpolasyon uygulayarak, farklı boyut ve çaplardaki komşuluklarda kullanılabilmesi için geliştirilmiştir. Başka bir değişik kullanımı ise tek biçim dokulardır. Yerel bir ikili değer dokusu, 0’dan 1’e veya tersi şeklinde en fazla iki bitsel geçiş içeriyorsa tek biçim olarak adlandırılır. Bu çalışmada, (8,1), (8,2) ve (16,2) komşu sayısı ve çap için tek biçim LBP’leri hesaplanmıştır. LBP+LDA metotu da bu çalışmada kullanılmıştır. İmajlara ait ek biçim (8,1)’lik LBP görüntü temsilleri elde edildikten sonra, bellek kısıtlarından ötürü alt örnekleme ile boyutu düşürülür. Tekil olmayan sınıf içi dağılım matrisi için PCA işleminden sonra, alt örneklenmiş özellik sınıfları üzerinde LDA uygulanır. Yüz tanıma performansını daha da arttırmak için Zernike momentleri kullanılmıştır. Global Zernike momentleri, LBP gibi bir yerel görüntü temsilleri eldesi için değiştirilmiştir. Komşuluklar ve her bir piksel etrafındaki mikro yapıyı yakalamak için bulunan moment bileşenleri dikkate alınarak, momentler her bir piksel için hesaplanmıştır. Asıl yüz imajı boyutlarına sahip kompleks moment imajı, her bir moment bileşeni için elde edilir. Daha sonra, her moment imajı, üst üste denk gelmeyecek şekilde alt bölgelere bölünür ve her bir alt bölgeden faz-büyüklük histogramları çıkartılır. Bu histogramlar peşi sıra birbirine eklenerek yüz temsili elde edilir. LBP ve LDA metotlarının birlikte kullanımı yüz tanıma başarısını olumlu bir şekilde etkilemektedir. Bu yüzden LZM ile LDA de birlikte kullanılarak, başarısı test edilmiştir. LDA’in LZM üzerine uygulanma şekli LBP+LDA işlemindekinin aynısıdır. Faz-büyüklük histogramlarının alt örnekleme ile boyutu düşürülmüştür. Daha sonar, LDA projeksiyonları hesaplanmış ve cosine benzerliği formülü ile eşleşme operasyonu gerçekleştirilmiştir. Sonuçlardan anlaşıldığı üzere, LZM+LDA’in LZM üzerinde belirgin bir üstünlüğü vardır. Bu çalışmada aşağıdaki metotlar kullanılmıştır: 1. Mahalanobis mesafesi ile PCA 2. Cosine benzerliği ile LDA 3. Ki-kare mesafesi ile tek biçim LBP (original (8,1), (8,2) ve (8,16)) 4. Cosine benzerliği ile LBP+LDA 5. Manhattan mesafesi ile LZM 6. Cosine benzerliği ile LBP+LZM Bu çalışma için oluşturulan yazılım hem kimlik tanımlama hem de kimlik doğrulama için test edilmiştir. Kimlik tanımlamada, sistem kullanıcının kim olduğunu bulmaya çalışır. Kimlik doğrulamada ise, kullanıcı belirli bir kimlik olduğunu iddia eder ve sistem bunun doğruluğunu kontrol eder. Testler için OTCBVS kalite testi veri kümesi koleksiyonundan CBSR NIR yüz veritabanı kullanılmıştır. Veritabanında 197 farklı kişiye ait toplam 3,940 YKÖ yüz imajı bulunmaktadır. Görüntüler, aktif yakın kızıl-ötesi ışıklandırma ile yakın kızıl-ötesi kamera kullanarak çekilmiştir. Kameranın üstüne konumlandırılmış 18 adet yakın kızıl-ötesi led bulunmaktadır. Bu çalışma için yapılan testler sonucunda, LZM’in başarısı, hem orijinal tek biçim LBP hem de farklı komşuluk sayısı ve çapta kullanım için geliştirilmiş olan tek biçim LBP metotlarından daha yüksek çıkmıştır. Metotların LDA ile birlikte kullanımı ise yüz tanıma işleminin başarısını daha üst seviyelere taşımaktadır. Kimlik doğrulama adımında, LBP operatörlerinin başarısı tek başına LDA’in başarısından daha fazladır; ancak kimlik tanımlama adımında LDA’in başarısı, LBP’nin üstünde çıkmıştır. PCA kullanımı ise hem tanımlama hem doğrulama için diğer metotların başarımlarını yakalayamamış; güvenilir bir yüz tanıma için yetersiz kalmıştır. Bir YKÖ yüz imajı, yüz tanıma sistemleri için sorunsuz bir girdi oluşturmaktadır; çünkü tanıma aşamasından önceki ağır ön işleme adımlarını azaltmaktadır. LZM işleminin de yardımlarıyla, YKÖ görüntüleme sisteminden elde edilmiş yüz imajları ile hızlı ve yüksek başarımlı yüz tanıma sistemleri gerçekleştirilebilir. Yalnız, YKÖ görüntüleme, işbirliği etmeyen kullanıcı uygulamaları için henüz uygun değildir. Ayrıca, dış mekan kullanımı da özellikle görünür ışığın, güneşli havalar gibi baskın olacağı yerlerde başarılı olamayabilir. Gelecekte, YKÖ görüntüleme sistemlerinde yapılacak çalışmalar ile bu tür kısıtların üzerinden gelinebilir.Humans have the ability to remember, recognize and distinguish faces and the scientists have been working on systems that can establish the same facility. The improvements in face recognition and numerous commercial face recognition systems has increased in a parallel way. Yet the need for more accurate systems still remains. Some examples of the applications in which face recognition is being used are:  Face-based video indexing and browsing engines  Multimedia management  Human-computer interaction  Biometric identity authentication  Surveillance systems There are two kinds of scenarios in face recognition, namely cooperative and uncooperative. Survellience systems can be a good example for uncooperative user applications. Cooperative user applications are such as access control machine readable traveling documents, ATM, computer login, e-commerce and e-government systems. In cooperative user scenarios, a user is required provide his/her face in a proper position for the camera to have the face image captured properly, in order to be granted for the access. In fact, many face recognition systems have been developed for such applications. The intrinsic and extrinsic factors of the face affect the performance of the face recognition. Face recognition should be performed based on intrinsic factors of the face only, like 3D shape reflectance of the facial surface. Extrinsic factors include eyeglasses, hairstyle, expression, posture, environmental lighting. They should be minimized for reliable face recognition. A biometric system should adapt to the environment, not vice versa. Among several extrinsic factors, problems with uncontrolled environmental lighting is the topmost issue. Lighting conditions, especially the light angle, change the appearance of a face so much that the changes calculated between the images of a person under different illumination conditions are larger than those between the images of two different people under the same illumination conditions. All of the local filters under study are insufficient by themselves to overcome variations due to changes in illumination direction. So, therefore, near infrared imaging is proposed. Studies on imaging beyond visible spectrum has been carried on recently. However, thermal imaging has many disadvantages as well as its advantages. Enviromental temperature, physical and emotional conditions, drinking alcohol can affect the system’s success drastically. Studies have shown they have not performed better than visible image based systems. 3D visible imaging had also been tried but the load created during its process and wearing sunglasses or an open mouth can fail the system’s success. There are two principles for the active lighting in near-infrared imaging: • The lights should be strong enough to produce clear frontal-lighted face image but not cause disturbance to human eyes • The resulting face image should be affected as little as possible after minimizing the environmental lighting. In this work, firstly, traditional face recognition methods such as PCA, LDA and LBP have been tried on NIR images for comparison with other methods. In Eigenfaces approach, “eigenfaces” are constructed from the face images, by means of PCA. The purpose of PCA is to reduce the large dimensionality of the data space to the smaller intrinsic dimensionality of feature space. In Fisherfaces approach, where LDA is applied after PCA, the projection direction is found so that the images belonging to different class, here the different ids, are separated maximally. In other words, the projection matrix that makes the ratio of the between-class scatter matrix and within-class scatter matrix of the images maximum, is found. Local image representations such as Gabor and LBP has arisen great interest. For robust face recognition, dealing with extrinsic properties of face is an important issue. LBP texture operator can handle the variations caused by these properties, such as illumination, so it has become a popular approach in various applications. LBP representation is used to compensate for the degree of freedom in a monotonic transform in the gray tone to achieve an illumination invariant representation of faces for indoor face recognition applications. The pixels of an image are labeled as 0 or 1, by thresholding the neighborhood of each pixel, considering the result as a binary number. The LBP operator was extended for neighborhood of different sizes and radius by bilinearly interpolating values at non-integer pixel coordinates. Another extension is the uniform patterns. A local binary pattern is called uniform if the binary pattern contains at most two bitwise transitions from 0 to 1 or vice versa. Uniform LBPs that have (8,1), (8,2) and (16,2) neighborhood and radius size are computed. LBP+LDA is also used in this work. After uniform LBP(8,1) representations of the images are obtained, they are downsampled because of the memory limitations. Then LDA is performed on the downsampled feature sets after PCA is applied to make the within-class scatter matrix nonsingular. Zernike moments are used to further improve the face recognition performance. Global Zernike moments are modified to obtain a local representation, such as LBP, called Local Zernike moments (LZM). The moments are computed at each pixel, considering their neighborhood and moment components obtained to capture the micro structure around each pixel. A complex moment image, which has the same size of the original face image, is obtained for each moment component. Later, each moment image is divided into non-overlapping subregions and phase-magnitude histograms are extracted from each subregion. Finally, the phase-magnitude histograms are concatenated and the face representation is built. Since the use of LDA on LBP has positive effects on the success of the recognition, LZM+LDA is implemented for this study. The process of applying LDA on LZM is the same as the process in LBP+LDA. The phase-magnitude moments are downsampled and PCA is applied before LDA operation. Afterwards, the LDA projections are calculated and cosine distance is used for the matching operation. It is found out that the success of LZM+LDA over LZM is significant. The tests in this study are performed with the following methods: 1. PCA with Mahalanobis distance 2. LDA with cosine distance 3. LBP with chi-square distance (original uniform (8,1), (8,2) and (16,2)) 4. LBP+LDA with cosine distance 5. LZM with Manhattan distance 6. LZM+LDA with cosine distance Both identification and verification have been tested for the methods. In face identification, a system tries to figure who the person is. In face verification, the system verifies whether the identity a person claims to be is true. CBSR NIR Face Dataset of OTCBVS Benchmark Dataset Collection is used. The database contains 3,940 NIR face images of 197 people. The images were taken by an NIR camera with active NIR lighting. 18 NIR LEDs are mounted on the camera. It is found that LZM performs better than both the original and extended uniform LBP methods in verification and identification tests. A method’s combination with LDA carries the success of face recognition to higher levels. In identification step, however, the extended LBP operators are more successful than LDA itself but in verification step, LDA is more successful than all the LBP operators. The success rate of PCA is not good enough to catch up with the other methods in face recognition. Using NIR face images for face recognition saves the system from the load of the preprocessing steps before the recognition. With the help of LZM on NIR images, robust and highly accurate systems can be built. Yet, NIR imaging is not improved enough to handle outdoor and uncooperative user applications. Future works on this context can help the system’s success carry to a higher level.Yüksek LisansM.Sc

Infrared face recognition: a comprehensive review of methodologies and databases

Automatic face recognition is an area with immense practical potential which includes a wide range of commercial and law enforcement applications. Hence it is unsurprising that it continues to be one of the most active research areas of computer vision. Even after over three decades of intense research, the state-of-the-art in face recognition continues to improve, benefitting from advances in a range of different research fields such as image processing, pattern recognition, computer graphics, and physiology. Systems based on visible spectrum images, the most researched face recognition modality, have reached a significant level of maturity with some practical success. However, they continue to face challenges in the presence of illumination, pose and expression changes, as well as facial disguises, all of which can significantly decrease recognition accuracy. Amongst various approaches which have been proposed in an attempt to overcome these limitations, the use of infrared (IR) imaging has emerged as a particularly promising research direction. This paper presents a comprehensive and timely review of the literature on this subject. Our key contributions are: (i) a summary of the inherent properties of infrared imaging which makes this modality promising in the context of face recognition, (ii) a systematic review of the most influential approaches, with a focus on emerging common trends as well as key differences between alternative methodologies, (iii) a description of the main databases of infrared facial images available to the researcher, and lastly (iv) a discussion of the most promising avenues for future research.Comment: Pattern Recognition, 2014. arXiv admin note: substantial text overlap with arXiv:1306.160

Analyzing Near-infrared Images for Utility Assessment

Visual cognition is of significant importance in certain imaging applications, such as security and surveillance. In these applications, an important issue is to determine the cognition threshold, which is the maximum distortion level that can be applied to the images while still ensuring that enough information is conveyed to recognize the scene. The cognition task is usually studied with images that represent the scene in the visible part of the spectrum. In this paper, our goal is to evaluate the usefulness of another scene representation. To this end, we study the performance of near-infrared (NIR) images in cognition. Since surface reflections in the NIR part of the spectrum is material dependent, an object made of a specific material is more probable to have uniform response in the NIR images. Consequently, edges in the NIR images are likely to correspond to the physical boundaries of the objects, which are considered to be the most useful information for cognition. This feature of the NIR images leads to the hypothesis that NIR is better than a visible scene representation to be used in cognition tasks. To test this hypothesis, we compared the cognition thresholds of NIR and visible images performing a subjective study on 11 scenes. The images were compressed with different compression factors using JPEG2000 compression. The results of this subjective test show that recognizing 8 out of the 11 scenes is significantly easier based on the NIR images when compared to their visible counterparts

Eye Detection and Face Recognition Across the Electromagnetic Spectrum

Biometrics, or the science of identifying individuals based on their physiological or behavioral traits, has increasingly been used to replace typical identifying markers such as passwords, PIN numbers, passports, etc. Different modalities, such as face, fingerprint, iris, gait, etc. can be used for this purpose. One of the most studied forms of biometrics is face recognition (FR). Due to a number of advantages over typical visible to visible FR, recent trends have been pushing the FR community to perform cross-spectral matching of visible images to face images from higher spectra in the electromagnetic spectrum.;In this work, the SWIR band of the EM spectrum is the primary focus. Four main contributions relating to automatic eye detection and cross-spectral FR are discussed. First, a novel eye localization algorithm for the purpose of geometrically normalizing a face across multiple SWIR bands for FR algorithms is introduced. Using a template based scheme and a novel summation range filter, an extensive experimental analysis show that this algorithm is fast, robust, and highly accurate when compared to other available eye detection methods. Also, the eye locations produced by this algorithm provides higher FR results than all other tested approaches. This algorithm is then augmented and updated to quickly and accurately detect eyes in more challenging unconstrained datasets, spanning the EM spectrum. Additionally, a novel cross-spectral matching algorithm is introduced that attempts to bridge the gap between the visible and SWIR spectra. By fusing multiple photometric normalization combinations, the proposed algorithm is not only more efficient than other visible-SWIR matching algorithms, but more accurate in multiple challenging datasets. Finally, a novel pre-processing algorithm is discussed that bridges the gap between document (passport) and live face images. It is shown that the pre-processing scheme proposed, using inpainting and denoising techniques, significantly increases the cross-document face recognition performance

3D face recognition using photometric stereo

Automatic face recognition has been an active research area for the last four decades. This thesis explores innovative bio-inspired concepts aimed at improved face recognition using surface normals. New directions in salient data representation are explored using data captured via a photometric stereo method from the University of the West of England’s “Photoface” device. Accuracy assessments demonstrate the advantage of the capture format and the synergy offered by near infrared light sources in achieving more accurate results than under conventional visible light. Two 3D face databases have been created as part of the thesis – the publicly available Photoface database which contains 3187 images of 453 subjects and the 3DE-VISIR dataset which contains 363 images of 115 people with different expressions captured simultaneously under near infrared and visible light. The Photoface database is believed to be the ?rst to capture naturalistic 3D face models. Subsets of these databases are then used to show the results of experiments inspired by the human visual system. Experimental results show that optimal recognition rates are achieved using surprisingly low resolution of only 10x10 pixels on surface normal data, which corresponds to the spatial frequency range of optimal human performance. Motivated by the observed increase in recognition speed and accuracy that occurs in humans when faces are caricatured, novel interpretations of caricaturing using outlying data and pixel locations with high variance show that performance remains disproportionately high when up to 90% of the data has been discarded. These direct methods of dimensionality reduction have useful implications for the storage and processing requirements for commercial face recognition systems. The novel variance approach is extended to recognise positive expressions with 90% accuracy which has useful implications for human-computer interaction as well as ensuring that a subject has the correct expression prior to recognition. Furthermore, the subject recognition rate is improved by removing those pixels which encode expression. Finally, preliminary work into feature detection on surface normals by extending Haar-like features is presented which is also shown to be useful for correcting the pose of the head as part of a fully operational device. The system operates with an accuracy of 98.65% at a false acceptance rate of only 0.01 on front facing heads with neutral expressions. The work has shown how new avenues of enquiry inspired by our observation of the human visual system can offer useful advantages towards achieving more robust autonomous computer-based facial recognition

Face recognition using infrared vision

Au cours de la dernière décennie, la reconnaissance de visage basée sur l’imagerie infrarouge (IR) et en particulier la thermographie IR est devenue une alternative prometteuse aux approches conventionnelles utilisant l’imagerie dans le spectre visible. En effet l’imagerie (visible et infrarouge) trouvent encore des contraintes à leur application efficace dans le monde réel. Bien qu’insensibles à toute variation d’illumination dans le spectre visible, les images IR sont caractérisées par des défis spécifiques qui leur sont propres, notamment la sensibilité aux facteurs qui affectent le rayonnement thermique du visage tels que l’état émotionnel, la température ambiante, la consommation d’alcool, etc. En outre, il est plus laborieux de corriger l’expression du visage et les changements de poses dans les images IR puisque leur contenu est moins riche aux hautes fréquences spatiales ce qui représente en fait une indication importante pour le calage de tout modèle déformable. Dans cette thèse, nous décrivons une nouvelle méthode qui répond à ces défis majeurs. Concrètement, pour remédier aux changements dans les poses et expressions du visage, nous générons une image synthétique frontale du visage qui est canonique et neutre vis-à-vis de toute expression faciale à partir d’une image du visage de pose et expression faciale arbitraires. Ceci est réalisé par l’application d’une déformation affine par morceaux précédée par un calage via un modèle d’apparence active (AAM). Ainsi, une de nos publications est la première publication qui explore l’utilisation d’un AAM sur les images IR thermiques ; nous y proposons une étape de prétraitement qui rehausse la netteté des images thermiques, ce qui rend la convergence de l’AAM rapide et plus précise. Pour surmonter le problème des images IR thermiques par rapport au motif exact du rayonnement thermique du visage, nous le décrivons celui-ci par une représentation s’appuyant sur des caractéristiques anatomiques fiables. Contrairement aux approches existantes, notre représentation n’est pas binaire ; elle met plutôt l’accent sur la fiabilité des caractéristiques extraites. Cela rend la représentation proposée beaucoup plus robuste à la fois à la pose et aux changements possibles de température. L’efficacité de l’approche proposée est démontrée sur la plus grande base de données publique des vidéos IR thermiques des visages. Sur cette base d’images, notre méthode atteint des performances de reconnaissance assez bonnes et surpasse de manière significative les méthodes décrites précédemment dans la littérature. L’approche proposée a également montré de très bonnes performances sur des sous-ensembles de cette base de données que nous avons montée nous-mêmes au sein de notre laboratoire. A notre connaissance, il s’agit de l’une des bases de données les plus importantes disponibles à l’heure actuelle tout en présentant certains défis.Over the course of the last decade, infrared (IR) and particularly thermal IR imaging based face recognition has emerged as a promising complement to conventional, visible spectrum based approaches which continue to struggle when applied in the real world. While inherently insensitive to visible spectrum illumination changes, IR images introduce specific challenges of their own, most notably sensitivity to factors which affect facial heat emission patterns, e.g., emotional state, ambient temperature, etc. In addition, facial expression and pose changes are more difficult to correct in IR images because they are less rich in high frequency details which is an important cue for fitting any deformable model. In this thesis we describe a novel method which addresses these major challenges. Specifically, to normalize for pose and facial expression changes we generate a synthetic frontal image of a face in a canonical, neutral facial expression from an image of the face in an arbitrary pose and facial expression. This is achieved by piecewise affine warping which follows active appearance model (AAM) fitting. This is the first work which explores the use of an AAM on thermal IR images; we propose a pre-processing step which enhances details in thermal images, making AAM convergence faster and more accurate. To overcome the problem of thermal IR image sensitivity to the exact pattern of facial temperature emissions we describe a representation based on reliable anatomical features. In contrast to previous approaches, our representation is not binary; rather, our method accounts for the reliability of the extracted features. This makes the proposed representation much more robust both to pose and scale changes. The effectiveness of the proposed approach is demonstrated on the largest public database of thermal IR images of faces on which it achieves satisfying recognition performance and significantly outperforms previously described methods. The proposed approach has also demonstrated satisfying performance on subsets of the largest video database of the world gathered in our laboratory which will be publicly available free of charge in future. The reader should note that due to the very nature of the feature extraction method in our system (i.e., anatomical based nature of it), we anticipate high robustness of our system to some challenging factors such as the temperature changes. However, we were not able to investigate this in depth due to the limits which exist in gathering realistic databases. Gathering the largest video database considering some challenging factors is one of the other contributions of this research