Uniform face meshes are able to represent the face in 3D format and can also be used to perform 3D face recognition.However, to obtain a good recognition rate, a fine mesh which consists of many points would be needed to accurately represent the many contours of the face.Therefore, in this paper, it is proposed that a non-uniform face mesh is constructed for 3D face recognition. A non-uniform mesh consisting of fine meshes for the middle of the face and coarse meshes for the rest of the face was created. In comparison with

a uniform mesh, the proposed non-uniform face mesh consists of much fewer points and therefore saves storage space and transmission time due to a smaller file size.Besides that, the proposed mesh was able to produce recognition rates that

were only slightly lower than the uniform mesh, hence proving that important face features for recognition were retained

Ang, Li-Minn

Kah, Phooi Seng

Wei, Jen Chew

UUM Repository

 Non-uniform Face Mesh for 3D Face Recognition Wei Jen Chew, Kah Phooi Seng, Li-Minn AngFaculty of EngineeringThe University of Nottingham, Jalan Broga, 43500 Semenyih, SelangorTel : 603-8924 8358, Fax : 603-8924 8017E-mail : Wei-Jen.Chew@nottingham.edu.myABSTRACTUniform face  meshes  are  able  to  represent  the  face  in  3D format and can also be used to perform 3D face recognition.  However, to obtain a good recognition rate, a fine mesh which  consists  of  many  points  would  be  needed  to  accurately  represent  the  many contours  of  the  face.  Therefore,  in  this paper,  it  is  proposed  that  a  non-uniform  face  mesh  is  constructed  for  3D  face  recognition.  A  non-uniform  mesh consisting of fine meshes for the middle of the face and coarse  meshes for the rest of the face was created. In comparison with  a uniform mesh, the proposed non-uniform face mesh consists  of much fewer points and therefore saves storage space and transmission time due to a smaller file size. Besides that, the  proposed  mesh  was  able  to  produce  recognition  rates  that  were only slightly lower than the uniform mesh, hence proving  that important face features for recognition were retained.KeywordsFace mesh, Face recognition1.0 INTRODUCTIONFace recognition is an important area that has been researched on for many years. A face recognition system can be used to identify dangerous people in a crowd or to verify the identity of  a  person.  In  the  past,  face  recognition  systems  uses  2D images  to  perform  identification  (Turk  &  Pentland,  1991; Belhumeur,  Hespanha  &  Kriegman,  1997).  Although  2D images can produce good results, they still suffer from a few problems,  like  illumination  and  pose  changes  (Zhao, Chellappa,  Rosenfeld  &  Phillips,  2000).  Therefore, researchers started to focus on using 3D images to solve face recognition  problems.  This  is  because  3D  images  can  be rotated  in  different  angles  and  therefore  solves  the  pose changes problem. 3D images are usually captured using 3D scanners and can be kept  in  a  variety  of  formats.  Different  format  will  suit different  applications  and  methods  of  usage.  These  formats include point clouds, polygonal models and 2.5D point matrix or  range  images  (Colbry,  2006).  Point  clouds  are  a  set  of points in a space, polygonal models, or meshes, are like point cloud, but with the extra neighbour information while range image is actually  a point  cloud in a matrix  (Colbry,  2006). This  type  of  matrix  needs  a corresponding  flag  matrix  that indicates whether that position in the matrix has a valid data or not. Range image is able to represent the face accurately and has been used for 3D face recognition. They are able to show the exact  contours  of  the  face  when  the  point  matrix  is  dense enough.  However,  the  image  size  can  be  huge  due  to  the amount of data stored which takes up storage space or causes long  transmission time.  Besides that,  some area of  the face may not need a large amount of points to represent it, causing some points to be redundant. Therefore,  face  meshes  have  been  proposed  in  the  past  to represent a 3D face image for face recognition, as shown by (Xu,  Wang,  Tan  &  Quan,  2004),  since  they  require  less points.  A  typical  mesh  consists  of  many  uniform  little triangles that cover the whole face. Basically, three points of a triangle in a mesh is able to represent a face area which may contain  many  points  from a range  image.  (Xu et  al.,  2004) method converts a point cloud face into a mesh, first by using a  coarse  mesh  and  then  subsequently  refining  it  to  a  finer dense mesh to represent a face. After that, recognition is done using  the  face  meshes.  Although  recognition  usually concentrate  on  the  face  features  like  the  eyes,  nose  and mouth, the whole face was converted into a finer dense mesh for recognition.Therefore,  in  this  paper,  it  is  proposed  that  a  3D  face represented by range images be converted to a non-uniform face mesh which consists of fine meshes in the eye, nose and mouth  areas  while  the  rest  of  the  face  is  represented  by  a coarse  mesh.  This  is  because  the  middle  area  of  the  face contains  many  contours  and  is  crucial  for  face  recognition while the rest of the face like the forehead and chin area is relatively smooth.  If  a uniform face mesh is used,  then the whole face will need to be covered by a fine mesh to cater for the middle part  of the face. With a non-uniform face mesh, which face area that needs a fine mesh and which face area that needs a coarse mesh can be controlled. The proposed method is able to reduce the number of points needed to represent the face, hence saving storage space and reducing transmission time. Besides that, the recognition rate would not be severely affected, as shown in the experimental results.Section  2  discusses  about  construction  of  the  non-uniform face  mesh  while  Section  3 discusses  about  fitting  the  face mesh to the range image. Section 4 discusses the method for face  recognition,  Section  5  shows  the  experimental  results and finally the conclusion is in Section 6.2.0 FACE MESH CONSTRUCTIONInstead of using a standard face mesh and fitting it according to face features location, a simple non-uniform face mesh is constructed instead. This mesh consists of 3 sections, which are the top,  middle and bottom sections.  The top section is from the top of the head till the eye level, the middle section is  from  the  eye  level  to  the  mouth  level  and  the  bottom section is from the mouth level to the chin level. The top and bottom sections  consist  of  a coarse  mesh while  the  middle section consists  of  a fine mesh.  This is because  the middle part of the face between the eyes and mouth contains a lot of face information and many contours.  Therefore,  it is crucial that  a  finer  mesh  is  placed  in  this  area  to  give  a  better representation of the face. However, for the forehead and chin areas,  there  are  not  much  contour  changes  and  therefore  a coarser  mesh  for  these  areas  is  sufficient.  The  basic  non-uniform mesh is shown in Figure 1.Figure 1: Basic non-uniform meshThe mesh created in Figure 1 has a fine mesh size which is 3 times smaller  than the  coarse  mesh.  The  size of  the  whole mesh is the same as the size of the image. In this paper, the eye  and  mouth  level  were  determined  manually  using  the corresponding 2D images of the 3D range image but this can be  easily  be  done  automatically  using  some  face  feature detection  algorithm.  After  obtaining  this  basic  non-uniform mesh, the next step is fitting it to a face range image.3.0 FITTING FACE MESH TO RANGE IMAGEAt the moment,  the basic mesh z-coordinate is still  at zero. Therefore,  the  next  step  is  to  match  each  mesh  point  z-coordinate with the corresponding range image. This is done using  the  least  square  fitting  method  (Weisstein;  Eberly, 2008). Firstly, the mesh created is superimposed on the range image. Then, for each triangle in the mesh, it is determined if there is a group of range values within the triangle boundary. If  there are no range values,  then this individual  triangle is removed from the mesh.For those that have a group of range values, the next step is to determine the plane equation for each group of points using equation (1) (Bourke, 1989).Ax + By + Cz + D = 0 (1)where the normal to the plane is the vector (A,B,C).Equation (1) shows the plane equation obtained from a group of  range  image  points.  Therefore,  to  estimate  the  z coordinates  of  each  mesh  triangle,  the  3  pairs  of  x  and  y coordinates  are  inserted  into  the  equation  to  obtain  each  z values.  This  results  in  a  mesh  that  has  different  depth  at different points, corresponding to the range image. However, the  mesh  is  still  disjointed  since  each  triangle  depth  value may not correspond to the neighbouring triangle depth value, as shown in Figure 2.Figure 2: Disjointed face meshTo smoothen out the mesh, all the z-coordinates at the same location are recorded and the final z-coordinate value at that location is the median of all the values. To eliminate outliers in the face mesh, all the z-coordinates must be more than Q1-(1.5xIQR)  and  less  than  Q3+(1.5xIQR)  where  Q1,  Q3 and IQR are the 1st quartile,  3rd quartile and inter-quartile range respectively. The final face mesh is shown in Figure 3.-1000100-200 -150 -100 -50 0 50 100 150-100-80-60-40-20020Figure 3: Proposed non-uniform face mesh4.0 FACE RECOGNITIONFace recognition is achieved by comparing two face meshes together  and  calculating  the distance  between them.  In this paper,  the comparison method used consists  of aligning the probe and database meshes together, find the distance of each point  on the probe with the mesh and calculating the mean square value of all the distances combined. Since each point in  the  probe  might  not  have  a  corresponding  point  in  the database mesh to determine their distance, an estimate of the corresponding  point  is  needed.  This  is  obtained  by  first determining which triangle area does the probe point falls in. Once  the  triangle  is  found,  the  triangle  plane  equation  is calculated  using  the equation  (2),  (3),  (4)  and  (5)  (Bourke, 1989).A = y1(z2 - z3) + y2(z3 - z1) + y3(z1 - z2) (2)B = z1(x2 - x3) + z2(x3 - x1) + z3(x1 - x2) (3)C = x1(y2 - y3) + x2(y3 - y1) + x3 (y1 - y2) (4)D = -(x1(y2z3 - y3z2) + x2(y3z1 - y1z3) + x3(y1z2 - y2z1)) (5)where (x1,y1), (x2,y2) and (x3,y3) are the 3 corner points of the triangle.Once the above information is calculated, the plane equation of  the  triangle  is  known.  Then,  by  inserting  the  x  and  y coordinate  of  the  investigated  probe  point  into  the  plane equation,  the  z  coordinate  of  the  database  mesh  at  that location is obtained and the distance between the probe and database mesh at that point is determined.5.0 RESULTSTo prove the effectiveness of the proposed non-uniform face mesh  for  face  recognition,  the  face  recognition  experiment was performed on 3 different types of face meshes. The first type is a uniform fine mesh and contains all small triangles, the second mesh is a uniform coarse  mesh and contains  all large  triangles  while  the  third  mesh  is  the  proposed  non-uniform face  mesh,  which  consists  of  both  fine  and  coarse meshes.  The  fine mesh,  coarse  mesh  and  non-uniform face mesh  contains  about  20%,  5% and  10% of  the  amount  of range image points respectively. Figure 4 and Figure 5 shows examples of a fine mesh and a coarse mesh.Figure 4: Uniform fine face mesh-1000100-200 -150 -100 -50 0 50 100 150-100-80-60-40-20020Figure 5: Uniform coarse face meshIn  this  paper,  the  UND  face  database  was  used  (Flynn, Bowyer  & Phillips,  2003;  Chang,  Bowyer  & Flynn,  2003). This database stores their face images in range image format. A total of 50 images were used as the database and 30 images were used as the probes. Firstly, these images were converted and  fitted  to  face  meshes  and  then  face  recognition  was performed on them. The final result is shown in Table 1. Table 1: Face recognition rateRank Uniform Coarse MeshUniform Fine MeshProposed Non-uniform Mesh1 20% 60% 53%2 47% 73% 60%3 67% 87% 60%4 80% 93% 87%5 80% 93% 87%From the results obtained in Table 1, it can be observed that the coarse  mesh had the worst  recognition rate  and there is only  slight  difference  between  the  recognition  rate  of  fine mesh and proposed mesh.The recognition rate of the uniform coarse mesh is not high probably  due  to  the  coarse  mesh  is  unable  to  sufficiently represent the finer contours of the face around the eye, nose and mouth area. Since these areas help differentiate one face to another, therefore, it is crucial for this area to be properly reconstructed.For the uniform fine mesh, recognition rate is high due to the whole face is properly reconstructed by the mesh, including the  finer  contours  of  the  face.  However,  to  represent  the whole face  using only a uniform fine mesh requires a high number of points,  which means the file size of  the mesh is big. This can pose storage and transmission problems.Therefore,  it  was  proposed  in  this  paper  that  the  face  be represented by a mesh that contains a combination of coarse and fine meshes. The multiple contour areas are represented by fine meshes while the rest of the face by coarse meshes. This helps optimize the mesh used to represent the face and reduces  the  number  of  points  for  the  face  mesh  when compared to a fully fine mesh, yet does not greatly affect the face  recognition  rate.  The  results  obtained  shows  that  the proposed method is a feasible method.6.0 CONCLUSIONIn this paper, it was proposed that 3D face images represented using range images be converted to a non-uniform face mesh to perform face recognition. Firstly, a basic mesh which has a combination of coarse  and fine mesh was created.  The fine mesh  encapsulates  the  eye,  nose  and  mouth  area  while  the coarse mesh is for the rest of the face.  This mesh was then fitted onto a face range image to obtain a face mesh. After all the database and probe range images were converted to face meshes,  face  recognition  was  performed.  The  recognition method used in this paper consists of aligning the probe and database  mesh  together  and  calculating  the  mean  distance between them. From the results obtained, it is observed that the non-uniform face mesh produces recognition rates that are only slightly lower than when a uniform fine mesh was used. However,  only  half  the  amount  of  points  is  needed  to represent  the non-uniform face mesh when compared to the uniform fine face mesh. Therefore,  this shows that  the face mesh  can  be  simplified  but  yet  retains  the  important  face features  to  produce  good  recognition  rates.  The  main advantage  in using  a combination  mesh  is  reduction  in the number of points representing the face, which in turn helps save storage space and transmission time.REFERENCESBelhumeur P. N., Hespanha J. P., and Kriegman D. J.. Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Trans. Pattern Analysis and Machine Intelligence, 19(7):711-729, Jul. 1997.Bourke P. (1989). Equation of a Plane. Retrieved January 5. 2009, from http://local.wasp.uwa.edu.au/~pbourke/geometry/planeeq/Chang  K.,  Bowyer  K.  W.,  and  Flynn P.  J.,  “Face recognition  using  2D  and  3D  facial  data,”  ACM Workshop  on  MultimodalUser  Authentication,  pp.25-32, December 2003.Colbry D., “Human Face Verification by Robust 3D Surface Alignment,” Doctoral dissertation, Department of Computer Science, Michigan State University, Michigan, United States of America, 2006.Eberly D. (2008). Least Squares Fitting of Data. Retrieved January 5, 2009, from http://www.geometrictools.com/Documentation/LeastSquaresFitting.pdfFlynn  P.  J.,  Bowyer  K.  W.,  and  Phillips  P.  J., “Assessment of time dependency in face recognition: An initial study,”  Audio and Video-Based Biometric  Person Authentication, pp.44-51, 2003.Turk M. and Pentland A.. Eigenfaces for recognition. Journal  of Cognitive Neuroscience, 3(1):71-86, Mar. 1991.Weisstein, Eric W. "Least Squares Fitting." From MathWorld--A Wolfram Web Resource. http://mathworld.wolfram.com/LeastSquaresFitting.html Xu C, Wang Y, Tan T, Quan L. Face recognition based on 3D mesh model. Accepted by SPIE defense & security symposium 2004; {online}, Intelligent Recognition and Digital Security Group, China, Available from: http://www.nlpr.ia.ac.cn/english/irds/publications.htm {Accessed 13 Jan 2008}Zhao W., Chellappa R., Rosenfeld A., and Phillips P. J., Face Recognition: A Literature Survey, UMD CfAR Technical Report CAR-TR-948, 2000.

Non-uniform face mesh for 3D face recognition

http://repo.uum.edu.my/13504/1/PID103.pdf

Non-uniform face mesh for 3D face recognition

Abstract

Similar works

Full text

Available Versions

UUM Repository