Many gait recognition approaches use silhouette data. Imperfections in silhouette extraction have a negative effect on the performance of a gait recognition system. In this paper we extend quality metrics for gait recognition and evaluate new ways of using quality to improve a recognition system. We demonstrate use of quality to improve silhouette data and select gait cycles of best quality. The potential of the new approaches has been demonstrated experimentally on a challenging dataset, showing how recognition capability can be dramatically improved. Our practical study also shows that acquiring samples of adequate quality in arbitrary environments is difficult and that including quality analysis can improve performance markedly

Mahmoodi, S.

Mansfield, T.

Matovski, D.

Nixon, M.

English

Southampton (e-Prints Soton)

On Including Quality in Applied Automatic Gait Recognition   D. S. Matovski, M. Nixon, S. Mahmoodi University of Southampton, UK dm08r,msn,sm3@ecs.soton.ac.uk T. Mansfield  National Physical Laboratory, UK  tony.mansfield@npl.co.uk  Abstract  Many  gait  recognition  approaches  use  silhouette data.  Imperfections  in  silhouette  extraction  have  a negative  effect  on  the  performance  of  a  gait recognition  system.  In  this  paper  we  extend  quality metrics for gait recognition and evaluate new ways of using  quality  to  improve  a  recognition  system.  We demonstrate use of quality to improve silhouette data and select gait cycles of best quality. The potential of the  new  approaches  has  been  demonstrated experimentally on a challenging dataset, showing how recognition capability can be dramatically improved. Our practical study also shows that acquiring samples of  adequate  quality  in  arbitrary  environments  is difficult  and  that  including  quality  analysis  can improve performance markedly.    1. Introduction  Quality  is  key  to  deployment  of  automated biometric recognition in real-world environments. Gait recognition analyses the way we walk, combined with our posture. Major advantages of gait include: it is non-invasive, hard to conceal and capable of being acquired at a distance. Many studies have demonstrated that gait has the potential to become a powerful biometric for surveillance applications [1].  There  are  number  of  factors  that  affect  gait recognition  performance.  Covariate  factors  can  be related to the subject (e.g. different clothing) or to the environment  (e.g.  different  background  or  lighting). There  are  studies  that  quantify  the  effect  of  various covariates  [2].  View  angle,  clothing,  footwear  and walking surface have been most commonly quoted to affect recognition.  In addition, these factors are confounded by errors in  the  silhouette  segmentation  process  common  to many recognition approaches. The silhouette extraction process separates the subject from the background and eliminates  the  color  and  texture  of  the  clothes. Background  subtraction  is  not  a  perfect  process  and poor segmentation can result even when the video is of good  quality.  Some  of  the  factors  influencing  the process  are:  subject  clothing,  change  in  illumination and background, distance from camera, and occlusion. These  factors  can  result  in  silhouette  imperfections such  as  shadows,  holes  in  the  silhouettes,  noisy contours, and undetected body parts.                                                Many  approaches  rely  on  clean  silhouette  data. However,  a  number  of  studies  have  established  the negative effect of poor silhouette quality on recognition performance [3, 4]. Approaches such as [5] propose a robust gait representation that works better with partial silhouettes. A study [6] has proposed a way of reducing the noise around the contours. We are not aware of any work  in  the  literature  that  uses  quality  metrics  to improve a pre-processing step, although all approaches could benefit from improvement in silhouette quality.   This paper extends the existing quality metrics and proposes novel ways of using the metrics to improve recognition in real-world environments. The potential of the techniques is demonstrated using a new database specifically captured for this study. The database has been recorded at two realistic locations. We show that recognition can be improved by using quality metrics to  improve  background  subtraction  which  is  an important  step  in  most  gait  recognition  approaches. The  results  also  show  the  benefit  of  using  quality metrics to select the best quality gait cycle.           2. Dataset  There  is  no  publically  available  gait  dataset  that contains  samples  of  the  same  subject  taken  at  two different  locations,  without  introducing  additional covariates  such  as  clothing  or  view  point.  Our  new database enables investigation of the effect of different environment-dependent  covariates  in  a  principled manner. It also provides a challenging environment for development and evaluation of quality metrics.                  The NPL-SOTON gait database has been recorded at  two  indoor  locations  –  a  corridor  (with  large windows at each end) and a large room as shown in Figure 1. Both locations are realistic environments for indoor surveillance with significant and uneven change in illumination.        Figure 1 – Locations for the experiments  Figure 2 – Walking directions  The setup (Figure 2) was replicated at both locations using three cameras for data collection. The fields of view of the cameras are shown in Figure 2. Camera 2 is elevated above camera 1 and shares its field of view. The instructions given to the 23 subjects were to “walk normally in a straight line between two points’. Nearly 2000 sequences were collected at different view angles to  the  camera.  Calibration  data  is  available  for  all cameras at both locations.  3. Quality metrics  Changes in the operational environment of a system can result in different quality of biometric samples. As such,  analyzing  quality  can  be  key  to  improving performance.     3.1 Quality metric 1 – Exploiting periodicity  Human  gait  is  a  periodic motion. If there is  little noise  associated  with  binary  silhouettes  (e.g.  in  the form of shadows, reflections etc.) then the total number of white pixels forms a periodic signal. Models have been developed to represent this signal. An example is shown in Eq. 1 [3].    y(t) = a + b cos(t) + c cos(2t) + z(t)       (1) where  ω  =  2π/T  and  T  is  the  gait  period,  a  is  the average area of a silhouette, b and c reflect the periodic variation in silhouette size, and z(t) is noise.  This model is good at detecting noise when it varies over the gait sequence. Distorted silhouettes are likely to  produce  noisy  foreground-sum  signals.  However, there are situations in which a signature of poor quality exists  and  the  model  is  not  able  to  detect  it.  For example, if the top part of the silhouette is missing for all frames the resulting foreground-sum signal is likely to still fit the model in Eq. 1 very closely.  Figure 3 shows an example of this which occurred in testing a fully automated system. The fit would have been even better if the subject was walking at 90 degrees to the camera.  Nevertheless,  the  periodic  model  gives  a  useful quality  metric  if  combined  with  additional  ones.  A number of new possible metrics were evaluated and the most effective ones in combination with metric 1 are presented in the next two sections.               Figure 3 – Model fitting quality metric 1  3.2 Quality metric 2 – Signature structure  This  quality  metric  focuses  on  analyzing  the structure of a gait signature. The Gait Entropy Image [1]  highlights  the  dynamic  areas  of  the  Gait  Energy Image  by  calculating  the  Shannon  entropy  at  each pixel:   Kkk k (x,y) p (x,y) p y x H12 log ) , (  pk(x,y) is the probability that a pixel takes on the kth value. A binary image corresponds to K = 2.  The  Gait  Entropy  Image  contains  pixels  of  high intensity values in the areas of the human body that are moving and low intensity values in the areas that are static. If silhouettes have been poorly segmented there will be entropy in areas that there should not be. There are a number of areas on the human body that can be tested for presence or absence of entropy. For example, there should be a small number of bright pixels in the head  and  torso  region  and  large  number  of  high intensity  value  pixels  round  the  arms  and  the  legs. Average  anthropometric  measurements  of  the  human Points (2) body [7] have been used to locate different parts of the body.  Examples  are  shown  in  Figure  4.  Equation  3 shows an example of the thorax region.   Figure 4 – Example for quality metric 2   A x B yy x H m ) , ( 2   (3) where for height h and centre c, the thorax is between A=[c − 0.174h/2, c + 0.174h/2] and at height between B=[0.182h, 0.53h].   3.3 Quality metric 3 – Use of height   To establish temporal correspondence, the height of the  bounding  box  formed  around  the  subject’s silhouette can be analyzed over a gait cycle or walking sequence  to  reveal  any  anomalies  in  the  process  of subject-background  separation.  If  calibration  data  is available the actual height of the subject can be used.  A model can be fitted to the height signal and the error of fit can reveal poor quality signatures. Similar models as Eq. 1 can be used, as in   h(t) = a + b cos(t) + c cos(2t) + z(t)       (4) Examples are shown below in Figure 5. The left side of the figure represents a sample of good quality and the right shows a sample of poor quality. The red line represents the line of best fit.     Figure 5 – Model fitting for metric 3  4 Deploying quality metrics  In the case of metrics 1 and 3, quality is evaluated by measuring the root mean square error. In the case of metric 2, the actual value is an indicator of quality. For the  thorax  region,  low  values  for  m2  indicate  a signature  of  good  quality.  Simple  normalization  by scaling between 0 and 1 was performed for each metric in order to combine the results.   Quality metrics can be used at various stages in a biometric system. The ways in which we use quality are vital if a sample is acquired in real world conditions where re-capturing is not possible. Our experience of using  an  automated  system  in  realistic  environment shows that acquiring a signature of good quality is very difficult. Conditions such as lighting constantly change in unpredictable manner. The changes are even more significant  in  the  case  of  matching  across  different locations and across time. The following two sections show the result obtained by analyzing our database.    Table 1 – Combinations of gallery and probe    Figure 6 - Signature quality improvement  4.1 Improvement in background subtraction   Matching  signatures  of  inconsistent  qualities  can result in poor recognition performance. The aim is to perform  the  background  subtraction  process  with  the parameters that produce the best quality signature. This is achieved by dynamically changing some of the key parameters. If a signature is below a pre-defined quality threshold, then a new signature is produced by varying the background subtraction parameters. In theory it is possible to find parameters that will produce an optimal gait signature.  An  experiment  was  performed  to  understand  the impact  of  quality  on  performance.  Twenty-three subjects (2 samples per subject) were used. All samples were  captured  in  the  same  day  and  no  additional subject dependent covariates were introduced.   The combination of gallery and probe is shown in Table 1. The experiments shown in the first and last two  rows  of  Table  1  have  been  performed  to Probe Loc. (Dir.) Gallery Loc. ( Dir. ) Cam No Quality used CCR(%) Corridor (IB)  Corridor (IB)  1  No  74 Room (IB)  Room (IB)  1  No  91 Corridor (IB)  Room (IB)  1  No  22 Room (IB)  Corridor (IB)  1  No  21 Corridor (IB)  Room (IB)  1  Yes  40          Corridor (HC)  Room (IHC)  3  No  0 Corridor (HC)  Room (IHC)  3  Yes  30 Corridor (HC)  Corridor (HC)  3  No  65 Corridor (HC)  Corridor (HC)  3  Yes  95  understand the effects of using quality when matching samples acquired at the same location. The rest of the experiments  focus  on  samples  acquired  at  different locations.  The  system  operated  in  recognition  mode and all three quality metrics are used.     Figure  6  shows  example  signatures  (of  the  same subject) before and after quality metrics were applied. The  figure  demonstrated  the  benefit  of  using  quality metrics in a fully automated gait recognition system for matching across locations. The  results  reveal  improvement  in  performance, both  in  the  case  of  samples  acquired  at  the  same location and at different locations when quality metrics are used. The CCR rate increases on average by 20-30%. A ROC curve for the last two entries of Table 1 is shown in Figure 7. The equal error rate decreased from 20% to 5% for matching across locations if quality is used. A further improvement in signature quality would be  achieved  by  optimizing  a  larger  number  of background subtraction parameters.    Figure 7-ROC analysis of quality improvement  4.2 Selection of the best gait cycle  Selection of a gait cycle is an important step in gait recognition  for  a  large  number  of approaches. There can be many possible gait cycles available for a given video  sequence.  More  specifically,  there  are  Lc-t potentially valid cycles, where Lc is the length of the clean signal i.e. number of frames where the subject is fully  visible  and  t  is  the  gait  period.  A  gait  cycle selection algorithm that utilizes quality metrics can lead to the best quality cycle being chosen. The cycle that produces  the  best  value  for  the  combination  of  all quality metrics can be used.  An  experiment  has  been  performed  to  show  the potential  benefits  of  deploying  quality  metrics  for selecting  the  most  appropriate  gait  cycle.  The combination of gallery and probe is shown in Table 2.  Samples  from  twenty-three  subjects  captured  by ‘Camera  1’  were  used.  The  results  reveal  an improvement in performance. Table 2 – Combinations of gallery and probe  5. Conclusions  We have extended the quality metrics and proposed a  novel  way  of  using  quality  metrics  to  improve  an important pre-processing step. Segmentation is the first processing step of most gait recognition algorithms and the  success  of  further  processing  depends  on  this process. The approach is generic and can be applied in other domains that employ background subtraction as a pre-processing step. We have also used quality metrics to select the best gait cycle.  The  benefits  of  using  quality  have  been demonstrated  on  a  new  and  challenging  dataset specifically  collected  for  this  study.  In  addition,  the advantages  have  been  validated  in  ‘real-life’  use.  Experiments  have  been  performed  over  multiple locations  and  the  results  reveal  that  quality  of  gait samples is a step in the right direction for deployment of gait recognition in real-world environments.  While  it  is  difficult  to  acquire  samples  of  good quality  in  a  realistic  environment,  the  techniques presented here show that quality of gait samples is a promising  avenue  for  future  research.  Improving  the quality  would  allow  other  techniques  that  rely  on silhouette data to generalize to unknown environments.   References  [1] K. Bashir, T. Xiang, S. Gong, Gait recognition without subject cooperation, Pattern Recog. Lett., 31(13), pp. 2052-2060, 2010. [2]  Y.  Shiqi,  T.  Daoliang,  T.  Tan,  A  Framework  for Evaluating the Effect of View Angle, Clothing and Carrying Condition on Gait Recognition, ICPR, 2006, pp. 441-444. [3]  J.  Liu,  N.  Zheng,  L.  Xiong,  Silhouette  quality quantification  for  gait  sequence  analysis  and  recognition, Signal Processing, 89(7), pp. 1417-1427, 2009. [4]  Z.  Liu,  S.  Sarkar,  Effect  of  silhouette  quality  on  hard problems  in  gait  recognition,  IEEE  Trans.  SMC,  Part  B, 35(2), pp. 170-183, 2005. [5]  C.  Chen,  J.  Liang,  H.  Zhao,  H.  Hu,  J.  Tian,  Frame difference energy image for gait recognition with incomplete silhouettes, Pattern. Recog. Lett., 30(11), pp. 977-984, 2009. [6] S. Yu, D. Tan, K. Huang, T. Tan, Reducing the Effect of Noise on Human Contour in Gait Recognition, LNCS, 4642, pp. 338-346, 2007. [7] D.A. Winter, Biomechanics and Motor Control of Human Movement Wiley, 2009.  Probe  Gallery Quality  CCR (%)  Loc.  Dir  Loc.  Dir Corridor  IB  Corridor  IB  No  60 Corridor  IB  Corridor  IB  Yes  75 

On including quality in applied automatic gait recognition

Abstract

Similar works

Full text

Available Versions

Southampton (e-Prints Soton)