Summary This paper presents a multimodal biometric system called &quot;handmetric&quot;, which is the fusion of hand based biometrics including palmprint, knuckleprint and hand shape. Moreover, a new framework of FLF (feature-level fusion) is put forward based on subspace analyzes. While verifies that existing feature concatenating methods are instances of the framework, a parameter optimized model using KPCA (kernel principal components analyze) is presented and applied to the verification system. The experiments testify the effectiveness of proposed method

Qiang Li

Zhengding Qiu

CiteSeerX

IJCSNS International Journal of Computer Science and Network Security, VOL.6 No.2A, February 2006 
 
 
 
164 
Manuscript received  February, 2006. 
Manuscript revised  February  , 2006. 
        Handmetric Verification Based on Feature-Level Fusion 
              Qiang Li,and Zhengding Qiu  
                                                   
                                        Institute of Information Science, Beijing Jiaotong University, Beijing, China                             
 
 
 
Summary 
This paper presents a multimodal biometric system called 
“handmetric”, which is the fusion of hand based 
biometrics including palmprint, knuckleprint and hand 
shape. Moreover, a new framework of FLF (feature-level 
fusion) is put forward based on subspace analyzes. While 
verifies that existing feature concatenating methods are 
instances of the framework, a parameter optimized model 
using KPCA (kernel principal components analyze) is 
presented and applied to the verification system. The 
experiments testify the effectiveness of proposed method. 
Key words: 
Multimodal biometric, handmetric verification, feature-level 
fusion, kernel principal components analyze, palmprint  
Introduction 
Information fusion for multimodal biometrics is a hot 
topic recently. As one of the most reliable biometric traits, 
palmprint and have got widely attention in recent years [1]. 
While fusing with other kind of biometric, palmprint 
recognition could be more accurate and more ready to 
business use. By adding hand geometry information on 
palmprint, Kumar et al. [2] suggests that fusion in decision 
level using max rule could outperform traditional serial 
concatenate feature-level fusion (FLF) method and single 
palmprint system. Ribaric et al. [3] testifies weighted sum 
rule in a similar manner. Based on these works, we name 
the outline of the hand and cuticle in the inner surface of 
the hand as “handmetric”. As to low-resolution images, 
handmetric contains palmprint, hand geometry and 
knuckleprint [4] with their geometrical correlativity. 
Without loss of generality, we fuse the identity 
information of palmprint, middle finger shape and 
knuckleprint as shown in Fig.1 to represent handmetric 
features in this paper. 
 
 
(a) palmprints from different persons 
 
(b) finger shapes from different persons 
  Fig.1. Palmprint and finger images from different persons 
Decision-level fusion and score-level fusion methods 
are most popular in the field of multimodal biometrics. 
According to Jain and Ross [5][6], FLF could keep the 
identity information to its most and is expected to perform 
better than at the above two levels, but the study on it is 
seldom reported. There are mainly two reasons of it [5]. 
First, the feature spaces of different biometric traits may 
not compatible. That is, different features may have 
different dimension and measurement, and their dynamic 
variation ranges lie in different complicated nonlinear 
spaces. Second, FLF may lead to the “curse of 
dimensionality” problem by concatenating two features as 
one. While solving these problems, we propose a new 
strategy for FLF based on the fusion of the relationship 
between samples but not the samples themselves.  
Actually, the existing serial and parallel feature 
concatenating methods are special cases of the framework. 
While implementing the framework using nonlinear 
kernels, the performance of the system would be enhanced 
greatly. In this paper we apply polynomial kernel with 
principal components analyze (KPCA) method to the 
handmetric system, the result of the evaluation shows the 
power of the framework. 
2. Idea and Algorithm 
2.1 The framework 
The traditional way of FLF can be sorted into two 
catalogues: serial concatenate and parallel concatenate, 
which are suffer from space incompatible and dimension 
curse greatly [5]. Lanckriet et al. [7] and Kuncheva et al. 
[8] suggest another way. By fusing different features using 
kernel matrix or template matrix, the problem of space 
incompatible is solved. However, dimension curse is still 
unfathomed. Based on these ideas, we employ KPCA with 
IJCSNS International Journal of Computer Science and Network Security, VOL.6 No.2A, February 2006 
 
 
165
decision level fusion operators to construct a novel 
framework for FLF. As shown in Fig.2, three steps are 
needed:  
 Step 1. Determine kernel matrixes. The kernel 
can be treated as nonlinear correlation between the 
samples. A more generalized operator A  is defined to 
describe the transform from the original sample space to 
the relation measurement space. 
 Step 2. Fusion of kernel matrixes using decision 
level fusion operator B , the fused kernel space O is 
produced by it. 
 Step 3. Feature extraction by operator C  in O, 
then the fused feature space F is determined.  
 
Fig.2. Three steps of the framework 
 
2.2 Implementation  
Given M zero-mean training samples of palmprint { pi } 
and finger samples { si } (i=1,…,M), the corresponding 
new handmetric feature {hi} could be acquired using 
operators A , B , andC .  
Kernel matrixes of palmprint and finger are 
generated using A . Fractional power polynomial model 
is evaluated in this paper, where the power is set to 0.7 
and feature dimension d is set to 60 (which reaches the 
best performance in palmprint verification task). The 
kernel matrix of palmprint Kp and that of the finger Ks 
are: 
       0.7 0.7[ ] ( ) ,[ ] ( )p ij i j s ij i jp p s sK K= ?                   (1)   (1) 
For different biometrics, the same A  is applied. 
Therefore, the kernel matrixes have the same kind of 
relation measurement. According to Kuncheva et al. [8], 
these matrixes could be fused reasonably. Operator B  
performs the fusion step. The four most popular rules in 
decision level fusion (sum, product, min and max) are 
adopted to evaluate the performance of our method. The 
new kernel matrix for handmetric Kh using different B  
could be:  
    
min
min
1
2
3
4
[ ] ([ ] [ ] ) / 2
[ ] [ ] [ ]
[ ] min([ ] [ ] )
[ ] max([ ] [ ] )
:     
:     
:     ,
:     ,
h avg ij p ij s ij
h prd ij p ij s ij
h ij p ij s ij
h ij p ij s ij
K K K
K K K
K K K
K K K
-
-
-
-
= +
=
=
=
×
B
B
B
B
                   (2) 
The function of operator C  is to find handmetric 
feature space based on Kh. KPCA can be regarded as PCA 
in nonlinear mapping space. That is, the principle 
components of Kh span the fused feature space F:  
                          h MK F FΛ=                     (3) 
where 1 2 1 2[ ], { , , , }M MdiagF Λa a a l l l= =K K . F is the 
nonlinear mapping matrix from original image sample to 
handmetric feature. 
Handmetric feature of all the samples could be 
calculated for further classification. Suppose pt, st to be a 
pair of testing palmprint and finger samples, their kernel 
projections are: 
     
0.7 0.7 0.7
1 2
0.7 0.7 0.7
1 2
[( )  ( )   ( ) ]
[( )  ( )   ( ) ]
T
p t t M t
T
s t t M t
k p p p p p p
k s s s s s s
= 利
= 利
L
L
               (4) 
The “relation” between the testing handmetric to trained 
handmetric is: 
            ( , )h p sk k k= B{                                 (5) 
And the handmetric fh could be calculated using the 
projection matrix F: 
    Th hf kF= .                      (6) 
2.3 Describe other FLF methods by the framework 
Serial concatenate is the most popular FLF method and is 
a linear instance of the framework. By concatenating 
different features serially, the ith new sample iy can be 
defined as: 
        1 2 1 2[ , , , , , ]
T
i i id i i idiy p p p s s s= L L , [1, ]i M∈         (7) 
Because y always has high dimension, PCA (principle 
component analyze) should be carried out to get the new 
feature fsc. The scatter matrix Σ  of the samples is: 
  
1
1 1M T T
i i
i
y y
M M=
Σ = =∑ YY                               (8) 
According to theorem SVD, calculating the eigenvalues 
and eigenvectors of  Σ  can be converted to diagonalize 
the matrix TY Y . Moreover, it can be proved that: 
                    T T T= +Y Y P P S S                                        (9) 
Thus, serial concatenate is a special case when 
A degrades to linear correlation operator and B  is set to 
sum rule. And, the equitation (9) verifies that using PCA 
to the concatenated feature sets has the same effect as it 
works on each feature before the fusion procedure. 
It can also be verified that parallel concatenate is 
another instance of the framework in complex domain. 
Furthermore, the framework could be more generalized if 
the implementation of operator A  is extends to more 
generic similarity measurement. 
IJCSNS International Journal of Computer Science and Network Security, VOL.6 No.2A, February 2006 
 
 
166 
3. MAP Classifier for classification 
The maximum a posteriori (MAP) classifier [9] is 
designed for handmetric verification. In the training stage, 
the feature difference should be calculated before 
classification. Every handmetric feature would minus all 
the other features to get the feature differences in the 
training set. If the difference is got form the same person, 
then it should be marked as “G (Genuine)” class. 
Otherwise, the difference is signed to “I (Impostor)” class. 
Classifier would build on the G and I classes in training 
samples. 
In the testing stage, suppose a testing handmetric 
feature ft has the identity statement Z, according to [8], the 
difference D  between ft and class center of Z is treated as 
a sample input to the classifier, that is: 
       
1
1
( ),   ,
L
t Z Z i i
i
f f f f f Z
L =
D = - = å             (10) 
The likelihood of D  to G and I are: 
     
1
1/ 2/ 2
1
1/ 2/ 2
exp{ 1/ 2( )}
( / )
(2 )
exp{ 1/ 2( )}
( / )
(2 )
T
G
D
G
T
I
D
I
P G
P I
p
p
-
-
- D S D
D =
S
- D S D
D =
S
  (11) 
Under the assumption that the prior is P(G)=P(I)=1/2 
[8], the posteriori probability P(G/ D ) should be: 
           
( / ) ( )
( / )
( / ) ( ) ( / ) ( )
( / )
( / ) ( / )
             
P G P G
P G
P G P G P I P I
P G
P G P I
D
D =
D + D
D
=
D + D
        (12) 
Finally, comparing P(G/ D ) to similarity threshold T, 
we could draw the conclusion whether the identity 
statement is true. 
4. Experiments  
To evaluate the effectiveness of proposed method, a hand 
image database is set up. 1,853 right hand images from 98 
individuals are captured using CCD camera based device. 
For each person, up to 28 images are captured in the 
period of 6 months for 4 times at most (the average is 2.7 
times), 4 samples of each person are taken out to form a 
training set, while the remaining 1,481 images are taken as 
testing set. After preprocessing, palmprint database and 
finger database derived from the original database. For 
each biometric (including palmprint, finger and 
handmetric) in the verification test, a total of 
145,138(1481 ´ 98) comparisons are performed for the 
testing images, in which 1,481(1481 ´ 1) are genuine 
matching. 
4.1 Comparison of different fusion operators 
    
   Fig. 3. ROCs using different kernel fusion operators  
Different fusion operators affect the performance of 
handmetric greatly. A close-set test is carried out. That is, 
the training set contains training samples form all the 
classes. The experimental results of 3 samples for each 
class are plotted in Fig. 3. It is evidently that B 1 is better 
result than the other three. The EER (equal error rate) of it 
reach 0.20%, while that of palmprint and finger are 0.61% 
and 1.20% respectively. 
4.2 Comparison of different fusion strategies 
The widely used score-level fusion and feature 
concatenating method are tested and compared to given 
method. Both close-set test and open-set test are 
performed in this experiment. In the open-set test, samples 
from 62 classes are trained (to ensure the dimensionality 
of handmetric feature could be greater than 60). The other 
36 classed are treated as new clients.  
Together with proposed method, 5 biometric 
verification systems are setup and compared: 
I. Palmprint verification system. The feature of 
palmprint is extracted using KPCA and proposed classifier 
is employed. Because palmprint contains most of identity 
information in handmetric, the system is optimized to 
determine the kernel parameters. Fractional power 
polynomial model with degree=0.7 and feature dimension 
60 perform the best and are applied to all the other 
systems. 
II. Finger (contains knuckleprint and hand shape) 
verification system.  
III. FC based handmetric verification system. FC 
combines the palmprint and finger features got in I and II 
serially to get corresponding handmetric feature.  
IV. SLF based handmetric verification system. The 
IJCSNS International Journal of Computer Science and Network Security, VOL.6 No.2A, February 2006 
 
 
167
output of classifiers of I and II are normalized and fused to 
implement the system. According with Ribaric [5] and 
Ross [2], sum rule performs the best in SLF method in our 
test, and employed in the final comparisons. 
V. KPCA based handmetric verification system. B 1 
is used to extract handmetric feature. While system III and 
IV depend on I and II greatly, proposed method has the 
lowest computational complexity. 
The experimental results using 4 training samples for 
each class are shown in Fig. 4 and Table. 1. First, it is 
testified that all the handmetric system are better than 
single biometric ones while KPCA based fusion method 
performs the best in all the systems. The HTER (half total 
error rate) of kernel fusion reaches 0.087% in the close-set 
test, which could meet the requirements of high security 
applications. Second, the performance of FC and SLF are 
at the same level. SLF better than FC mostly, but FC may 
reach lower HTER in the tests. Finally, it is evidently that 
the system performance degrades dramatically when 
turning close-set into open-set. Kernel fusion method may 
be the best solution of the problem through our test. 
  
    (a) open-set test 
  
                       (b) close-set test 
Fig. 4. ROCs of different verification systems  
Table. 1. HTER of different systems. 
HTER(%) I II III IV V 
Open-set 5.74 7.59 2.10 
2.28 
0.78
Close-set 0.56 1.04 0.35 0.43 0.087
4.3 Real time identification evaluation 
      
 
Fig. 5. Demo of real time identification  
A real time handmetric identification system is 
implemented based on proposed method. 100 samples 
from 10 persons are trained for identification. The speed is 
about 22fps using P4 2.6G and 512M RAM. Though it 
uses low resolution images, the result is inspiring. While 
the fusion method could identify above 80% frames in the 
testing video correctly, that of palmprint and finger only 
achieve 54% and 32% respectively. Fig.5 shows a frame 
that fusion result is right while both of the stand-along 
systems get wrong result.  
5. Conclusion 
FLF based on KPCA outperforms the traditional FC 
(feature concatenate) and SLF schemes, and could 
improve the system performance greatly. It keeps more 
identity information than SLF method and more 
reasonable theoretically than traditional FLF method. Four 
most popular rules of SLF are employed and evaluated to 
fuse different kernel matrixes, and the sum rule gives the 
best result in the handmetric verification system. Further 
study includes normalization method of kernel matrix, 
fusion using ranks or decision results in kernel space, and 
more generalized subspace methods for FLF. 
 Acknowledgments 
This work is partially supported by Zhejiang Natural 
Science Foundation of China under grant no. Y104540, 
the Key Laboratory of Advanced Information Science and 
Network Technology of Beijing, China under grant no. 
TDXX0509.  
IJCSNS International Journal of Computer Science and Network Security, VOL.6 No.2A, February 2006 
 
 
168 
 
References 
 [1] D. Zhang, W. K. Kong, J. You, “Online Palmprint 
Identification”, IEEE Trans. PAMI. 25(9), 2003, pp. 1041-
1050 
[2] A. Kumar, D. C. M. Wong, H. C. Shen, A. K. Jain, 
“Personal Verification using Palmprint and Hand 
Geometry Biometric”, Proceedings of the fourth 
International Conference on audio- and video-based 
biometric personal authentication, 2003. 
[3] S. Rabaric, D. Ribaric, N. Pavesic, “A Biometric 
Identification System Based on the Fusion of Hand and 
Palm Features”, Proceedings of The Advent of Biometrics 
on the Internet, A Cost 275 Workshop, 2002. 
[4] Q. Li, Z. Qiu, D. Sun, Personal Identification Using 
Knuckleprint, Sinobiometric04’, Lecture Notes in 
Computer Science, Vol. 3338. Springer-Verlag, 2004, pp. 
680-689. 
[5] A. K. Jain, A. Ross, “Multibiometric Systems”, 
Communication of the ACM, Special Issue on Multimodal 
Interfaces 47 (1), 2004, pp. 34-40 
[6] A. Ross, A. K. Jain, “Information Fusion in 
Biometrics”, Pattern Recognition Letters 24 (13), 2003, pp. 
2115-2125 
 [7] G. Lanckriet, M. Deng, N. Cristianini, M. I. Jordan, 
“Kernel-based Data Fusion and Its Application to Protein 
Function Prediction in Yeast”, Proceedings of the Pacific 
Symposium on Biocomputing, 2004, pp. 300-311. 
[8] L. I. Kuncheva, J. C. Bezdek, R. P. W. Duin, 
“Decision Templates for Multiple Classifier Fusion: An 
Experimental Comparison”, Pattern Recognition, 34(2)  
2001, pp. 299-314. 
[9] B. Moghaddam, “Principal Manifolds and Probabilistic 
Subspaces for Visual Recognition”, IEEE Trans. PAMI, 
24 (6), 2002, pp. 780-788. 
 
 
Qiang Li received the B.S. and M.S. 
degrees in Electrical Engineering from 
Beijing Jiaotong University, P.R. China in 
1999 and 2002, respectively. Currently, he is 
studying for his Ph.D in the Institute of 
Information Science of Beijing Jiaotong 
University. His research interests include 
biometrics, information fusion, image 
processing, pattern recognition and signal processing. 
 
Zhengding Qiu received the B.S. degree in 
Communication Engineering in 1967, and 
M.S. degree in Signal Processing in 1981 
from Beijing Jiaotong University 
respectively. Since 1981, he worked at 
Institute of Information Science, Beijing 
Jiaotong University. During 1999 to 2000, 
he was a visiting research fellow at 
Electronics Engineering Lab of Kent 
University, U.K., engaged in image compression and biometrics 
processing program. He is currently a professor at Beijing 
Jiaotong University, a fellowship of Chinese Institute of 
Communication, a senior member of  Chinese Institute of 
Electronics and Railway respectively. 
  


Handmetric verification based on feature-level fusion

http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=D125A8DB42F591A6B2FB86AE8A665217?doi=10.1.1.1055.1870&rep=rep1&type=pdf

Handmetric verification based on feature-level fusion

Abstract

Similar works

Full text

Available Versions

CiteSeerX