New Multimodal Biometric Systems with Feature-Level and Score-Level Fusions

In recent years, biometric-based authentication systems have become very important in view of their ability to prevent identity theft by identifying an individual with high accuracy and reliability. Multimodal biometric systems have now drawn some attention in view of their ability to provide a performance superior to that provided by the corresponding unimodal biometric systems by utilizing more than one biometric modality. The existing multimodal biometric systems fuse multiple modalities at a single level, such as sensor, feature, score, rank or decision, and no study to fuse the modalities at more than one level that may lead to a further improvement in the performance of multimodal biometric systems, has been hitherto undertaken. In this thesis, multimodal biometric systems, wherein fusions of the modalities are carried out at more than one level, are investigated. In order to improve the performance of multimodal biometric systems over unimodal biometric systems, normalization and weighting of scores from multiple matchers are essential tasks. In view of this, in the first part of the thesis, a number of normalization and weighting techniques under the score level fusion are investigated. Unlike the existing normalization techniques that are based only on the genuine scores, four new techniques based on both the genuine and impostor scores, are proposed. Two weighting techniques that are based on confidence of the scores, are proposed. Extensive experiments are conducted to evaluate the performance of the multimodal biometric system under the score-level fusion (MBS-SL) using the proposed normalization and weighting techniques. The focus of the second part of this thesis is on the development of multimodal biometric systems, wherein fusions of the modalities are carried out at multiple levels. Specifically, two multimodal biometric systems, in which three modalities are used for their fusion both at the feature level and the score level, are proposed. In the first multimodal biometric system, referred to as the multimodal biometric system with feature level and score level (MBS-FSL) fusions, the features of the three modalities are encoded using the binary hash encoding technique. Unlike the existing techniques for feature level fusion that use unencoded features, this encoding technique allows the neighbourhood feature information to be taken into account. The score-level fusion is carried out on the score obtained from the feature-level fusion and the score from the matching module of the modality that has the lowest equal error rate. In the proposed MBS-FSL, the border values of raw features could not participate in the encoding in view 4-connected neighbors not being available. In order to take both the border and non-border information as well as the neighbourhood information into consideration, a second multimodal biometric system, referred to as the multimodal biometric system with modified feature level and score level (MBS-MFSL) fusions, is proposed, wherein both the raw and encoded features are taken into account. In this system, the feature-level fusion is carried out in a manner similar to that for the MBS-FSL system. The score-level fusion is then carried out between the score obtained from the feature-level fusion, the score from the matching module of the modality that was not utilized in the feature-level fusion, and the scores from individual modalities by using their raw features. Extensive experiments are performed to evaluate the performance of the two proposed multimodal biometric systems. The results of these experiments demonstrate that both of the proposed multimodal biometric systems provide performance superior to that provided by the existing multimodal biometric systems in which fusion of modalities is carried out at a single level, namely, the score level. Experimental results also show that, in view of both the border and neighbourhood feature information being considered in the proposed MBS-MFSL system, it provides a performance superior to that provided by MBS-FSL system. The investigation undertaken in this thesis is aimed at advancing the present knowledge in the field of human biometric identification by considering, for the first time, the fusion of the modalities at two levels, namely, the feature and score levels, and it is hoped that the findings of this study would pave the way for further research in the development of new multimodal biometric systems employing fusion of modalities at multiple levels

Multimodal person recognition for human-vehicle interaction

Next-generation vehicles will undoubtedly feature biometric person recognition as part of an effort to improve the driving experience. Today's technology prevents such systems from operating satisfactorily under adverse conditions. A proposed framework for achieving person recognition successfully combines different biometric modalities, borne out in two case studies

Multiple classifiers in biometrics. part 1: Fundamentals and review

We provide an introduction to Multiple Classifier Systems (MCS) including basic nomenclature and describing key elements: classifier dependencies, type of classifier outputs, aggregation procedures, architecture, and types of methods. This introduction complements other existing overviews of MCS, as here we also review the most prevalent theoretical framework for MCS and discuss theoretical developments related to MCS The introduction to MCS is then followed by a review of the application of MCS to the particular field of multimodal biometric person authentication in the last 25 years, as a prototypical area in which MCS has resulted in important achievements. This review includes general descriptions of successful MCS methods and architectures in order to facilitate the export of them to other information fusion problems. Based on the theory and framework introduced here, in the companion paper we then develop in more technical detail recent trends and developments in MCS from multimodal biometrics that incorporate context information in an adaptive way. These new MCS architectures exploit input quality measures and pattern-specific particularities that move apart from general population statistics, resulting in robust multimodal biometric systems. Similarly as in the present paper, methods in the companion paper are introduced in a general way so they can be applied to other information fusion problems as well. Finally, also in the companion paper, we discuss open challenges in biometrics and the role of MCS to advance themThis work was funded by projects CogniMetrics (TEC2015-70627-R) from MINECO/FEDER and RiskTrakc (JUST-2015-JCOO-AG-1). Part of thisthis work was conducted during a research visit of J.F. to Prof. Ludmila Kuncheva at Bangor University (UK) with STSM funding from COST CA16101 (MULTI-FORESEE

Multimodal biometrics score level fusion using non-confidence information

Multimodal biometrics refers to automatic authentication methods that depend on multiple modalities of measurable physical characteristics. It alleviates most of the restrictions of single biometrics. To combine the multimodal biometrics scores, three different categories of fusion approaches including rule based, classification based and density based approaches are available. When choosing an approach, one has to consider not only the fusion performance, but also system requirements and other circumstances. In the context of verification, classification errors arise from samples in the overlapping region (or non- confidence region) between genuine users and impostors. In score space, a further separation of the samples outside the non-confidence region does not result in further verification improvements. Therefore, information contained in the non-confidence region might be useful for improving the fusion process. Up to this point, no attempts are reported in the literature that tries to enhance the fusion process using this additional information. In this work, the use of this information is explored in rule based and density based approaches mentioned above

Longitudinal Study of Child Face Recognition

We present a longitudinal study of face recognition performance on Children Longitudinal Face (CLF) dataset containing 3,682 face images of 919 subjects, in the age group [2, 18] years. Each subject has at least four face images acquired over a time span of up to six years. Face comparison scores are obtained from (i) a state-of-the-art COTS matcher (COTS-A), (ii) an open-source matcher (FaceNet), and (iii) a simple sum fusion of scores obtained from COTS-A and FaceNet matchers. To improve the performance of the open-source FaceNet matcher for child face recognition, we were able to fine-tune it on an independent training set of 3,294 face images of 1,119 children in the age group [3, 18] years. Multilevel statistical models are fit to genuine comparison scores from the CLF dataset to determine the decrease in face recognition accuracy over time. Additionally, we analyze both the verification and open-set identification accuracies in order to evaluate state-of-the-art face recognition technology for tracing and identifying children lost at a young age as victims of child trafficking or abduction

Curvelet and Ridgelet-based Multimodal Biometric Recognition System using Weighted Similarity Approach

Biometric security artifacts for establishing the identity of a person with high confidence have evoked enormous interest in security and access control applications for the past few years. Biometric systems based solely on unimodal biometrics often suffer from problems such as noise, intra-class variations and spoof attacks. This paper presents a novel multimodal biometric recognition system by integrating three biometric traits namely iris, fingerprint and face using weighted similarity approach. In this work, the multi-resolution features are extracted independently from query images using curvelet and ridgelet transforms, and are then compared to the enrolled templates stored in the database containing features of each biometric trait. The final decision is made by normalizing the feature vectors, assigning different weights to the modalities and fusing the computed scores using score combination techniques. This system is tested with the public unimodal databases such as CASIA–Iris-V3-Interval, FVC2004, ORL and self-built multimodal databases. Experimental results obtained shows that the designed system achieves an excellent recognition rate of 98.75 per cent and 100 per cent for the public and self-built databases respectively and provides ultra high security than unimodal biometric systems.Defence Science Journal, 2014, 64(2), pp. 106-114. DOI: http://dx.doi.org/10.14429/dsj.64.346