Multimodal biometric authentication based on voice, fingerprint and face recognition

openNew decison module to combine the score of voice, fingerprint and face recognition in a multimodal biometric system.New decison module to combine the score of voice, fingerprint and face recognition in a multimodal biometric system

A Novel Approach to Combining Client-Dependent and Confidence Information in Multimodal Biometric

The issues of fusion with client-dependent and confidence information have been well studied separately in biometric authentication. In this study, we propose to take advantage of both sources of information in a discriminative framework. Initially, each source of information is processed on a per expert basis (plus on a per client basis for the first information and on a per example basis for the second information). Then, both sources of information are combined using a second-level classifier, across different experts. Although the formulation of such two-step solution is not new, the novelty lies in the way the sources of prior knowledge are incorporated prior to fusion using the second-level classifier. Because these two sources of information are of very different nature, one often needs to devise special algorithms to combine both information sources. Our framework that we call ``Prior Knowledge Incorporation'' has the advantage of using the standard machine learning algorithms. Based on $10 \times 32=320$ intramodal and multimodal fusion experiments carried out on the publicly available XM2VTS score-level fusion benchmark database, it is found that the generalisation performance of combining both information sources improves over using either or none of them, thus achieving a new state-of-the-art performance on this database

Compensating User-Specific Information with User-Independent Information in Biometric Authentication Tasks

Biometric authentication is a process of verifying an identity claim using a person's behavioral and physiological characteristics. This is in general a binary classification task because a system either accepts or rejects an identity claim. However, a biometric authentication system contains many users. By recognizing this fact, better decision can be made if user-specific information can be exploited. In this study, we propose to combine user-specific information with user-independent information such that the performance due to exploiting both information sources does not perform worse than either one and in some situations can improve significantly over either one. We show that this technique, motivated by a standard Bayesian framework, is applicable in two levels, i.e., fusion level where multiple (multimodal or intramodal) systems are involved, or, score normalization level, where only a single system is involved. The second approach can be considered a novel score normalization technique that combines both information sources. The fusion technique was tested on 32 fusion experiments whereas the normalization technique was tested on 13 single-system experiments. Both techniques that are originated from the same principal share a major advantage, i.e., due to prior knowledge as supported by experimental evidences, few or almost no free parameter are actually needed in order to employ the mentioned techniques. Previous works in this direction require at least 6 to 10 user-specific client accesses. However, in this work, as few as two user-specific client accesses are needed, hence overcoming the learning problem with extremely few user-specific client samples. Finally, but not the least, a non-exhaustive survey on the state-of-the-arts of incorporating user-specific information in biometric authentication is also presented

Generic multimodal biometric fusion

Biometric systems utilize physiological or behavioral traits to automatically identify individuals. A unimodal biometric system utilizes only one source of biometric information and suffers from a variety of problems such as noisy data, intra-class variations, restricted degrees of freedom, non-universality, spoof attacks and unacceptable error rates. Multimodal biometrics refers to a system which utilizes multiple biometric information sources and can overcome some of the limitation of unimodal system. Biometric information can be combined at 4 different levels: (i) Raw data level; (ii) Feature level; (iii) Match-score level; and (iv) Decision level. Match score fusion and decision fusion have received significant attention due to convenient information representation and raw data fusion is extremely challenging due to large diversity of representation. Feature level fusion provides a good trade-off between fusion complexity and loss of information due to subsequent processing. This work presents generic feature information fusion techniques for fusion of most of the commonly used feature representation schemes. A novel concept of Local Distance Kernels is introduced to transform the available information into an arbitrary common distance space where they can be easily fused together. Also, a new dynamic learnable noise removal scheme based on thresholding is used to remove shot noise in the distance vectors. Finally we propose the use of AdaBoost and Support Vector Machines for learning the fusion rules to obtain highly reliable final matching scores from the transformed local distance vectors. The integration of the proposed methods leads to large performance improvement over match-score or decision level fusion

Towards Explaining the Success (Or Failure) of Fusion in Biometric Authentication

Biometric authentication is a process of verifying an identity claim using a person's behavioral and physiological characteristics. Due to vulnerability of the system to environmental noise and variation caused by the user, fusion of several biometric-enabled systems is identified as a promising solution. In the literature, various fixed rules (e.g. min, max, median, mean) and trainable classifiers (e.g. linear combination of scores or weighted sum) are used to combine the scores of several base-systems. Despite many empirical experiments being reported in the literature, few works are targeted at studying a wide range of factors that can affect the fusion performance, but most undertook these factors in isolation. Some of these factors are: 1) dependency among features to be combined, 2) the choice of fusion classifier/operator, 3) the choice of decision threshold, 4) the relative base-system performance, 5) the presence of noise (or the degree of robustness of classifiers to noise), and 6) the type of classifier output. To understand these factors, we propose to model Equal Error Rate (EER), a commonly used performance measure in biometric authentication. Tackling factors 1--5 implies that the use of class conditional Gaussian distribution is imperative, at least to begin with. When the class conditional scores (client or impostor) to be combined are based on a multivariate Gaussian, factors 1, 3, 4 and 5 can be readily modeled. The challenge now lies in establishing the missing link between EER and the fusion classifier mentioned above. Based on the EER framework, we can even derive such missing link with non-linear fusion classifiers, a proposal that, to the best of our knowledge, has not been investigated before. The principal difference between the theoretical EER model proposed here and previous studies in this direction is that scores are considered log-likelihood ratios (of client versus impostor) and the decision threshold is considered a prior (or log-prior ratio). In the previous studies, scores are considered posterior probabilities whereby the role of adjustable threshold as a prior adjustment parameter is somewhat less emphasized. When the EER models (especially those on Order Statistics) cannot address adequately factors 1 and 4, we rely on simulations, which are relatively easy to carry out and whose results can be interpreted more easily. There are however several issues left untreated in the EER models, namely 1) what if the scores are known to be not approximately normally distributed (for instance those due to Multi-Layer Perceptron outputs); 2) what if scores among classifiers to be combined are not comparable in range (their distributions are different from each other); 3) how to evaluate the performance measure other than EER. For issue 1, we propose to reverse the process of the squashing function such that the data (scores) is once again approximately normal. For issue 2, some score normalization procedures are also proposed, namely F-ratio normalization (F-Norm) and margin normalization. F-Norm has the advantage that scores are made comparable while the relative shape of the distribution remains the same. Margin normalization has the advantage that no free parameter is required and such transformation relies entirely on the class conditional scores. Finally, although the Gaussian assumption is central to this work, we show that it is possible to remove such assumption by modeling the scores to be combined with a mixture of Gaussians. Some 1186 BANCA experiments verify that such approach can estimate the system performance better than using the Gaussian assumption

A Study of the Effects of Score Normalisation Prior to Fusion in Biometric Authentication Tasks

Although the subject of fusion is well studied, the effects of normalisation prior to fusion are somewhat less well investigated. In this study, four normalisation techniques and six commonly used fusion classifiers were examined. Based on 24 (fusion classifiers) as a result of pairing the normalisation techniques and classifiers applied on 32 fusion data sets, 4x6x32 means 768 fusion experiments were carried out on the XM2VTS score-level fusion benchmark database, it can be concluded that trainable fusion classifiers are potentially useful. It is found that some classifiers are very sensitive (in terms of Half Total Error Rate) to normalisation techniques such as Weighted sum with weights optimised using Fisher-ratio and Decision Template. The mean fusion operator and user-specific linear weight combination are relative less sensitive. It is also found that Support Vector Machines and Gaussian Mixture Model are the least sensitive to different normalisation techniques, while achieving the best generalisation performance. For these two techniques, score normalisation is unnecessary prior to fusion

Evidences of Equal Error Rate Reduction in Biometric Authentication Fusion

Multimodal biometric authentication (BA) has shown perennial successes both in research and applications. This paper casts a light on why BA systems can be improved by fusing opinions of different experts, principally due to diversity of biometric modalities, features, classifiers and samples. These techniques are collectively called variance reduction (VR) techniques. A thorough survey was carried out and showed that these techniques have been employed in one way or another in the literature, but there was no systematic comparison of these techniques, as done here. Despite the architectural diversity, we show that the improved classification result is due to reduced (class-dependent) variance. The analysis does not assume that scores to be fused are uncorrelated. It does however assume that the class-dependent scores have Gaussian distributions. As many as 180 independent experiments from different sources show that such assumption is acceptable in practice. The theoretical explanation has its root in regression problems. Our contribution is to relate the reduced variance to a reduced classification error commonly used in BA, called Equal Error Rate. In addition to the theoretical evidence, we carried out as many as 104 fusion experiments using commonly used classifiers on the XM2VTS multimodal database to measure the gain due to fusion. This investigation leads to the conclusion that different ways of exploiting diversity incur different hardware and computation cost. In particular, higher diversity incurs higher computation and sometimes hardware cost and vice-versa. Therefore, this study can serve as an engineering guide to choosing a VR technique that will provide a good trade-off between the level of accuracy required and its associated cost

Multi-system Biometric Authentication: Optimal Fusion and User-Specific Information

Verifying a person's identity claim by combining multiple biometric systems (fusion) is a promising solution to identity theft and automatic access control. This thesis contributes to the state-of-the-art of multimodal biometric fusion by improving the understanding of fusion and by enhancing fusion performance using information specific to a user. One problem to deal with at the score level fusion is to combine system outputs of different types. Two statistically sound representations of scores are probability and log-likelihood ratio (LLR). While they are equivalent in theory, LLR is much more useful in practice because its distribution can be approximated by a Gaussian distribution, which makes it useful to analyze the problem of fusion. Furthermore, its score statistics (mean and covariance) conditioned on the claimed user identity can be better exploited. Our first contribution is to estimate the fusion performance given the class-conditional score statistics and given a particular fusion operator/classifier. Thanks to the score statistics, we can predict fusion performance with reasonable accuracy, identify conditions which favor a particular fusion operator, study the joint phenomenon of combining system outputs with different degrees of strength and correlation and possibly correct the adverse effect of bias (due to the score-level mismatch between training and test sets) on fusion. While in practice the class-conditional Gaussian assumption is not always true, the estimated performance is found to be acceptable. Our second contribution is to exploit the user-specific prior knowledge by limiting the class-conditional Gaussian assumption to each user. We exploit this hypothesis in two strategies. In the first strategy, we combine a user-specific fusion classifier with a user-independent fusion classifier by means of two LLR scores, which are then weighted to obtain a single output. We show that combining both user-specific and user-independent LLR outputs always results in improved performance than using the better of the two. In the second strategy, we propose a statistic called the user-specific F-ratio, which measures the discriminative power of a given user based on the Gaussian assumption. Although similar class separability measures exist, e.g., the Fisher-ratio for a two-class problem and the d-prime statistic, F-ratio is more suitable because it is related to Equal Error Rate in a closed form. F-ratio is used in the following applications: a user-specific score normalization procedure, a user-specific criterion to rank users and a user-specific fusion operator that selectively considers a subset of systems for fusion. The resultant fusion operator leads to a statistically significantly increased performance with respect to the state-of-the-art fusion approaches. Even though the applications are different, the proposed methods share the following common advantages. Firstly, they are robust to deviation from the Gaussian assumption. Secondly, they are robust to few training data samples thanks to Bayesian adaptation. Finally, they consider both the client and impostor information simultaneously

Benchmarking Quality-Dependent and Cost-Sensitive Score-Level Multimodal Biometric Fusion Algorithms

Automatically verifying the identity of a person by means of biometrics is an important application in day-to-day activities such as accessing banking services and security control in airports. To increase the system reliability, several biometric devices are often used. Such a combined system is known as a multimodal biometric system. This paper reports a benchmarking study carried out within the framework of the BioSecure DS2 (Access Control) evaluation campaign organized by the University of Surrey, involving face, fingerprint, and iris biometrics for person authentication, targeting the application of physical access control in a medium-size establishment with some 500 persons. While multimodal biometrics is a well-investigated subject, there exists no benchmark for a fusion algorithm comparison. Working towards this goal, we designed two sets of experiments: quality-dependent and cost-sensitive evaluation. The quality-dependent evaluation aims at assessing how well fusion algorithms can perform under changing quality of raw images principally due to change of devices. The cost-sensitive evaluation, on the other hand, investigates how well a fusion algorithm can perform given restricted computation and in the presence of software and hardware failures, resulting in errors such as failure-to-acquire and failure-to-match. Since multiple capturing devices are available, a fusion algorithm should be able to handle this nonideal but nevertheless realistic scenario. In both evaluations, each fusion algorithm is provided with scores from each biometric comparison subsystem as well as the quality measures of both template and query data. The response to the call of the campaign proved very encouraging, with the submission of 22 fusion systems. To the best of our knowledge, this is the first attempt to benchmark quality-based multimodal fusion algorithms

Optimising multimodal fusion for biometric identification systems

Biometric systems are automatic means for imitating the human brain’s ability of identifying and verifying other humans by their behavioural and physiological characteristics. A system, which uses more than one biometric modality at the same time, is known as a multimodal system. Multimodal biometric systems consolidate the evidence presented by multiple biometric sources and typically provide better recognition performance compared to systems based on a single biometric modality. This thesis addresses some issues related to the implementation of multimodal biometric identity verification systems. The thesis assesses the feasibility of using commercial offthe-shelf products to construct deployable multimodal biometric system. It also identifies multimodal biometric fusion as a challenging optimisation problem when one considers the presence of several configurations and settings, in particular the verification thresholds adopted by each biometric device and the decision fusion algorithm implemented for a particular configuration. The thesis proposes a novel approach for the optimisation of multimodal biometric systems based on the use of genetic algorithms for solving some of the problems associated with the different settings. The proposed optimisation method also addresses some of the problems associated with score normalization. In addition, the thesis presents an analysis of the performance of different fusion rules when characterising the system users as sheep, goats, lambs and wolves. The results presented indicate that the proposed optimisation method can be used to solve the problems associated with threshold settings. This clearly demonstrates a valuable potential strategy that can be used to set a priori thresholds of the different biometric devices before using them. The proposed optimisation architecture addressed the problem of score normalisation, which makes it an effective “plug-and-play” design philosophy to system implementation. The results also indicate that the optimisation approach can be used for effectively determining the weight settings, which is used in many applications for varying the relative importance of the different performance parameters