Gökberk, B.

Spreeuwers, Lieuwe Jan

Veldhuis, Raymond N.J.

English

Three-dimensional (3D) face recognition systems have started to become popular in biometric systems recently. This is due to several factors: i) facial shape characteristics contain discriminative information, ii) availability of practical 3D acquisition devices, and iii) invariance of 3D facial information to several factors such as illumination and pose changes. It has been shown that classical texture-based 2D face classifiers have difficulties in identifying faces under such conditions. Therefore, taking advantage of 3D facial shape information either alone or together with 2D modality is considered to be a viable solution under such circumstances. In this work, we propose a novel 3D face recognition system using the combination of different individual 3D face classifiers; namely Linear Discriminant-based (LDA) Likelihood Ratio classifier with the Iterative Closest Point-based (ICP) matching algorithm. Both systems operate on aligned and normalized 3D facial surfaces. Alignment phase of the proposed system carries out absolute alignment such that all faces are in a specific position and direction, with non-facial parts removed. LDA-based system uses absolutely aligned faces and produce similarity scores using Likelihood ratio-based classifier. However, the ICP-based classifier performs additional surface matching between absolutely aligned faces, which can be considered as relative alignment. After pair-wise alignment of gallery and probe faces, the ICP algorithm produces dissimilarity scores, by measuring the quality of surface registration. With the use of different matching algorithms and different alignment methods, our approach tries to minimize the shortcomings of each individual method. Finally, the scores obtained by 3D face recognizers are fused to improve the verification accuracy. Our preliminary experiments conducted on the subset of FRGC v2 3D face database show promising performance improvement in verification simulations

University of Twente Research Information

Fusion of Likelihood Ratio Classifier with ICP-based Matcher for 3D FaceRecognitionBerk Gokberk Luuk J. Spreeuwers Raymond VeldhuisUniversity of TwenteDepartment of Electrical EngineeringSignals and Systems GroupP.O. box 217, 7500 AE Enschede, The NetherlandsB.Gokberk@utwente.nl L.J.Spreeuwers@utwente.nl R.N.J.Veldhuis@utwente.nlAbstractThree-dimensional (3D) face recognition systems have started to become popu-lar in biometric systems recently. This is due to several factors: i) facial shapecharacteristics contain discriminative information, ii) availability of practical 3Dacquisition devices, and iii) invariance of 3D facial information to several fac-tors such as illumination and pose changes. It has been shown that classicaltexture-based 2D face classifiers have difficulties in identifying faces under suchconditions. Therefore, taking advantage of 3D facial shape information eitheralone or together with 2D modality is considered to be a viable solution undersuch circumstances. In this work, we propose a novel 3D face recognition systemusing the combination of different individual 3D face classifiers; namely LinearDiscriminant-based (LDA) Likelihood Ratio classifier with the Iterative ClosestPoint-based (ICP) matching algorithm. Both systems operate on aligned andnormalized 3D facial surfaces. Alignment phase of the proposed system carriesout absolute alignment such that all faces are in a specific position and direction,with non-facial parts removed. LDA-based system uses absolutely aligned facesand produce similarity scores using Likelihood ratio-based classifier. However,the ICP-based classifier performs additional surface matching between absolutelyaligned faces, which can be considered as relative alignment. After pair-wisealignment of gallery and probe faces, the ICP algorithm produces dissimilarityscores, by measuring the quality of surface registration. With the use of dif-ferent matching algorithms and different alignment methods, our approach triesto minimize the shortcomings of each individual method. Finally, the scores ob-tained by 3D face recognizers are fused to improve the verification accuracy. Ourpreliminary experiments conducted on the subset of FRGC v2 3D face databaseshow promising performance improvement in verification simulations.1 IntroductionIn recent years, three dimensional (3D) face recognition systems has caught consid-erable attention due to their advantages over classical 2D face recognition systems.Most importantly, 3D surface characteristics of a human face offers more discrimina-tory information than using only 2D facial texture information. With the availabilityof state-of-the-art 3D facial acquisition devices, 3D facial shape can be obtained ac-curately. The invariance of facial shape to varying illumination conditions and it’spotential to easily compensate for in-depth rotation variations make the use of 3Dfacial information attractive in many biometrics applications.Most of the proposed solutions for 3D face recognition systems rely on i) efficientand accurate alignment/registration of facial surfaces, and ii) surface matching tech-niques to infer the similarity of two faces. One of the standard techniques to achievethese goals is to use the Iterative Closest Point [3] (ICP) algorithm to align two facialsurfaces and to output the quality of alignment as the similarity score which then canbe used as a matching criterion. Another popular class of algorithms use the projec-tions of the aligned surfaces into 2D images, referred to as depth or range images,and then use classical image representation techniques to classify faces. Statistical fea-ture extraction techniques such as Principal Component Analysis, Linear DiscriminantAnalysis (LDA) and Independent Component Analysis are frequently used with depthimage representations [2].It has been shown that the most critical part of any 3D face recognition systemsis the accurate alignment and registration of human faces. Therefore most of thescientific research in the 3D face recognition systems has been carried out for findingprecise alignment of faces. Since human faces resemble each other, as opposed to3D object identification problem where the object classes are distinctively differentfrom each other, transforming faces into a same coordinate system is crucial to extractdiscriminative information by the pattern classifier. In this paper, we propose a 3D facerecognition system that i) encapsulates two different registration methods, absolute andrelative registration, and ii) uses two different powerful pattern classifiers, LikelihoodRatio classifier and Linear Discriminant Analysis, to compute the similarity of anygiven face pairs. Proposed system automatically carries out initial absolute alignment,i.e., transforming faces into the same coordinate system and normalizes facial surfaceinformation, i.e., removing noise, filling incomplete data, and extracting the centralfacial region. Likelihood ratio-based classifier is used to classify absolutely aligned faces.The complementary pipeline encompasses relative registration via the ICP approachto obtain a finer registration between a given pair of facial surfaces after the absolutealignment phase and Linear Discriminant Analysis to extract discriminative features.Finally, the two similarity scores obtained from these two approached are combined atthe fusion phase to achieve better recognition rates.2 Proposed 3D Face Recognition SystemThe proposed 3D face recognition system is composed of the following steps: 1) Initialabsolute alignment of faces, 2) two different facial matchers (likelihood ration classifierand ICP-based Linear Discriminant Analysis) working in parallel and 3) the score-levelfusion algorithm. The overall diagram of our 3D face recognition system is illustratedin Figure 1.2.1 Absolute AlignmentIn the first phase, face normalization and absolute alignment, the aim is to translate,rotate and scale facial surfaces to a canonical position. In order to achieve absolutealignment, central vertical symmetry plane of a facial surface is automatically foundby a multi-resolution (from low to high resolution) approach. Nose tip point is alsoautomatically localized which enables us to translate 3D facial point cloud to the originof the coordinate system. Face normalization algorithm also comprises of automaticnoise removal and hole filling. Due to the acquisition sensor, 3D human facial surfacesmostly contain perturbations such as small surface bumps and also impulse-like spikes.Local averaging filters and median filters are used to remove such surface errors. Inmost of the 3D facial sensors operating by the principle of laser beams, non-reflectivedark facial regions usually do not provide depth measurements. Due to this reason,some parts of the acquired faces such as eyes and eyebrows contain holes. If holesFigure 1: Overall diagram of the proposed approach.are relatively small, they can be filled my filtering operations, such as using medianfilter. However, when they are large, then filtering operations may not be sufficient.Therefore, in our system, these type of situations are dealt with by using the facialsymmetry property. In the proposed approach, since the vertical symmetry plane iscomputed automatically, we fill such large holes with the information present in themirror side of the facial surface. Lastly, using the automatically determined nose tiplocation, we keep the central facial region by cropping out the other parts by a 3Dspherical mask. Figure 2 shows an input raw facial surface, its cropped version, andthe final facial surface after the smoothing and hole filling operations.(a) (b) (c)Figure 2: a) Input face, b) cropped/normalized face, and c) smoothed and filled facialimage.2.2 Likelihood Ratio ClassifierBiometric verification can be considered as a two class pattern recognition problem toseparate genuine and impostor classes. Likelihood Ratio (LLR) classifier finds a lineardecision boundary under the assumption that these two classes are Gaussian[4]. Duringtraining, each user i is modeled by his/her mean feature vector, µi, and covariancematrix Σi. The impostor class is represented by a global mean and covariance matrix,µI and ΣI . Using the log-likelihood ratio, the matching score d of a given test samplex to the claimed identity i can be computed by: d = −(x − µi)T Σ−1i (x − µi) + (x −µI)T Σ−1I (x − µI). Given a verification threshold t, the user can either be acceptedas a genuine user or rejected as an impostor. In order to make the estimation of userspecific mean and covariance matrices easier, we assumed that all users share a commoncovariance matrix and applied dimensionality reduction by the Principal ComponentAnalysis and Linear Discriminant Analysis methods.2.3 Linear Discriminant Analysis after the ICP AlignmentIn our system, likelihood ratio classifier operates on absolutely aligned depth images.Although absolute alignment establishes sufficient registration, i.e., correspondencebetween the related depth measurements across faces, it is also worthwhile to carryout pairwise (relative) alignment between face pairs. The intuition here is that withpairwise registration 3D facial surface information coming from the same identity mayalign better to each other compared to the others at the expense of more computationalload. Computational load is mainly due to the need of registering the probe face toeach gallery face. However, in a typical verification scenario, only one registrationwith the enrolled template is needed as opposed to identification. In our approach, wechoose to use the Iterative Closest Point algorithm for pairwise rigid registration. TheICP algorithm finds the best rotation and translation parameters to align one surfaceto the other. If the facial surfaces do not exhibit significant deformations such asdue to extreme expressions, the ICP approach performs sufficiently well. Given a testface and the claimed identity, we use the absolutely aligned facial surfaces as inputsto the ICP algorithm. Since both the probe and gallery face of the claimed identityis coarsely aligned beforehand, the convergence of the ICP algorithm is usually fast.After aligning the probe face to the gallery face, we compute the depth image and useLinear Discriminant Analysis to extract discriminative facial features.2.4 Score-Level Fusion by the Logistic DiscriminantAs a fusion algorithm, we choose to use a linear discriminant-based approach, namelythe logistic discriminant. The attractive point of using a linear discriminant approachin fusion is to bypass the estimation of posteriors of the classes[5]. It is usually suf-ficient to estimate the parameters of the discriminant directly. Although non-lineardiscriminants can also be used, we choose to employ a linear decision boundary be-cause of its effectiveness and simplicity. In logistic discrimination, the ratio of theclass-conditional densities are are modeled, specifically, it is assumed that the log like-lihood ratio is linear. The parameters of the logistic discriminant are learned by agradient-descent mechanism. In a verification setting, the two classes to be separatedcorrespond to the genuine and impostor cases. At the training phase, a linear separat-ing line is learned by a training set of impostor and genuine scores. At the verificationphase, the output of the logistic discriminant measures how likely a given test face iscoming from the genuine and impostor classes. A typical decision line learned by thelogistic discriminant is shown in Figure 4.3 Experimental ResultsWe have tested our algorithm on a subset of the FRGC v.2 3D face database [6]. In theFRGC v.2 database, there are 4,007 face scans of 465 subjects. 3D shape data contains30,000 to 40,000 3D coordinates. Although the quality of the scanned data is high,there are two types of noise affecting 3D faces: small protrusions and impulse noise-likejumps. In our experiments, we have used the Spring 2003 part of the FRGC datasetfor the training of the Likelihood ratio classifier and Linear Discriminant Analysis-based classifier. Spring 2003 part of the FRGC database contains 943 3D scans of 275subjects. An independent test, having 977 scans of the 100 subjects, is used to test theverification accuracy of the proposed 3D verification system. We have compared everypossible pair of facial images in the test set, producing 476,776 comparisons. There are6,092 genuine and 470,684 impostor comparisons in our experimental protocol.Figure 3 shows the score distributions obtained by the LLR and ICP-based match-ers. Figure 3(a) LLR and ICP scores are shown together in a 2D plot for genuine (reddots) and impostor (blue dots) comparisons. In order to better visualize the overlap-ping regions of the genuine and impostor distributions, score distributions of the LLRand ICP matchers are shown independently in Figure 3(b) and Figure 3(c), respectively.Corresponding to this experimental setup, the Equal Error Rates (EER) obtained bythe LLR and ICP-based system are 4.45% and 2.71%, respectively. Based on this re-sults, it is seen that improving the alignment by the ICP-based relative registrationimproves the accuracy of the verification system.(a)(b) (c)Figure 3: a) LLR and ICP scores in two dimensions, b) LLR scores, and c) ICP scores.In the second part of our experiments, we investigate the effect of score-level fusionof LLR and ICP-based matchers. Since the logistic discriminant-based fusion algo-rithm needs training before the fusion at the verification phase, we have formed twoindependent sets from the test part of the FRGC v.2 experiments. To train the fu-sion algorithm, we have used half of the genuine and impostor comparisons, i.e., 3,046genuine and 235,342 comparisons are used for that purpose. The remaining genuineand impostor comparisons are used to test the fusion accuracy. In this particular fu-sion experimental protocol, LLR and ICP-based matchers obtained 4.40% and 2.62%EER which are similar to the EERs obtained by using whole genuine and impostorcomparisons. The LLR and ICP scores for this experiment together with the decisionline found by the logistic discriminant-based fusion algorithm (black line) are shownin Figure 4. As seen from Figure 4, decision line better separates the genuine andimpostor classes compared to using each matcher alone. This results is confirmed bythe EER obtained by the fusion system which is 1.54%. In order to analyze the perfor-mance of the proposed scheme under different operating points, i.e., at different FARand FRR points, it is better to look at the Receiver Operating Characteristic (ROC)curves of the verification algorithms. In Figure 5, the ROC curves of the LLR, ICP,and the fusion algorithm are shown. Visual inspection of the ROC curve reveals thatin all operating regions, fusion algorithm improves the performance of both LLR andICP-based systems.Figure 4: Decision line found by the fusion algorithm.4 ConclusionIn this paper, fusion of two different 3D face recognition methods has been proposed.The first matching engine employs Likelihood Ratio classifier where the 3D facial sur-faces are represented as depth images. Prior to verification, an automatic facial surfacealignment method is used to transform faces into a canonical position. This absolutealignment step together with the use of LLR classifier has shown to be very effectivein the verification experiments conducted on a subset of the FRGC v.2 database, ob-taining 4.45% EER. The second matching engine further improves the registration offacial image pairs by the ICP algorithm and employs LDA-based classifier to computethe similarity scores. Experimental findings showed that the second matching engineFigure 5: ROC curves for the LLR (blue), ICP (green) and the fusion system (red).performs accurately by achieving an EER of 2.71%. Combining the two matchers atthe score level by a linear discriminant-based approach finally produces the best veri-fication accuracy on the FRGC v.2 database, producing an EER of 1.54%. As futurework, it is worthwhile to investigate other fusion mechanisms different than the lineardiscriminants.References[1] Bowyer, K. W., Chang, K., Flynn, P., “A survey of approaches and challengesin 3D and multi-modal 3D + 2D face recognition,” Computer Vision and ImageUnderstanding, Vol.101, pp. 1-15, 2006.[2] Gokberk, B., Dutagaci, H., Ulas, A., Akarun, L., Sankur, B., “RepresentationPlurality and Fusion for 3D Face Recognition, ” IEEE Transactions on Systems,Man, and Cybernetics-Part B: Cybernetics, Vol.38, Issue.1, 2008.[3] Besl, P., McKay, N., “A Method for Registration of 3D Shapes,” IEEE Transactionson Pattern Analysis and Machine Intelligence, Vol.14, pp. 239-256, 1992.[4] Veldhuis, R., Bazen, A., Kauffman, J., Hartel, P., “Biometric verification based ongrip-pattern recognition,” SPIE, 2004.[5] Alpaydin, E., “Introduction to Machine Learning,” The MIT Press, October 2004,ISBN 0-262-01211-1.[6] Phillips, J. P., Flynn, P. J., Scruggs, T., Bowyer K. W., Chang, J., Hoffman,K., Marques, J., Min, J., Worek, W., “Overview of the face recognition grandchallenge,” International Conference on Computer Vision and Pattern Recognition,pp. 947-954, 2005

Fusion of likelihood ratio classifier with ICP-based matcher for 3D face recognition

https://ris.utwente.nl/ws/files/6137892/gokberk_wic2009.pdf

Fusion of likelihood ratio classifier with ICP-based matcher for 3D face recognition

Abstract

Similar works

Full text

Available Versions

University of Twente Research Information