Geometric Expression Invariant 3D Face Recognition using Statistical Discriminant Models

Currently there is no complete face recognition system that is invariant to all facial expressions. Although humans find it easy to identify and recognise faces regardless of changes in illumination, pose and expression, producing a computer system with a similar capability has proved to be particularly di cult. Three dimensional face models are geometric in nature and therefore have the advantage of being invariant to head pose and lighting. However they are still susceptible to facial expressions. This can be seen in the decrease in the recognition results using principal component analysis when expressions are added to a data set. In order to achieve expression-invariant face recognition systems, we have employed a tensor algebra framework to represent 3D face data with facial expressions in a parsimonious space. Face variation factors are organised in particular subject and facial expression modes. We manipulate this using single value decomposition on sub-tensors representing one variation mode. This framework possesses the ability to deal with the shortcomings of PCA in less constrained environments and still preserves the integrity of the 3D data. The results show improved recognition rates for faces and facial expressions, even recognising high intensity expressions that are not in the training datasets. We have determined, experimentally, a set of anatomical landmarks that best describe facial expression e ectively. We found that the best placement of landmarks to distinguish di erent facial expressions are in areas around the prominent features, such as the cheeks and eyebrows. Recognition results using landmark-based face recognition could be improved with better placement. We looked into the possibility of achieving expression-invariant face recognition by reconstructing and manipulating realistic facial expressions. We proposed a tensor-based statistical discriminant analysis method to reconstruct facial expressions and in particular to neutralise facial expressions. The results of the synthesised facial expressions are visually more realistic than facial expressions generated using conventional active shape modelling (ASM). We then used reconstructed neutral faces in the sub-tensor framework for recognition purposes. The recognition results showed slight improvement. Besides biometric recognition, this novel tensor-based synthesis approach could be used in computer games and real-time animation applications

Morphable Face Models - An Open Framework

In this paper, we present a novel open-source pipeline for face registration based on Gaussian processes as well as an application to face image analysis. Non-rigid registration of faces is significant for many applications in computer vision, such as the construction of 3D Morphable face models (3DMMs). Gaussian Process Morphable Models (GPMMs) unify a variety of non-rigid deformation models with B-splines and PCA models as examples. GPMM separate problem specific requirements from the registration algorithm by incorporating domain-specific adaptions as a prior model. The novelties of this paper are the following: (i) We present a strategy and modeling technique for face registration that considers symmetry, multi-scale and spatially-varying details. The registration is applied to neutral faces and facial expressions. (ii) We release an open-source software framework for registration and model-building, demonstrated on the publicly available BU3D-FE database. The released pipeline also contains an implementation of an Analysis-by-Synthesis model adaption of 2D face images, tested on the Multi-PIE and LFW database. This enables the community to reproduce, evaluate and compare the individual steps of registration to model-building and 3D/2D model fitting. (iii) Along with the framework release, we publish a new version of the Basel Face Model (BFM-2017) with an improved age distribution and an additional facial expression model

3-D Face Analysis and Identification Based on Statistical Shape Modelling

This paper presents an effective method of statistical shape representation for automatic face analysis and identification in 3-D. The method combines statistical shape modelling techniques and the non-rigid deformation matching scheme. This work is distinguished by three key contributions. The first is the introduction of a new 3-D shape registration method using hierarchical landmark detection and multilevel B-spline warping technique, which allows accurate dense correspondence search for statistical model construction. The second is the shape representation approach, based on Laplacian Eigenmap, which provides a nonlinear submanifold that links underlying structure of facial data. The third contribution is a hybrid method for matching the statistical model and test dataset which controls the levels of the model’s deformation at different matching stages and so increases chance of the successful matching. The proposed method is tested on the public database, BU-3DFE. Results indicate that it can achieve extremely high verification rates in a series of tests, thus providing real-world practicality

Dense 3D Face Correspondence

We present an algorithm that automatically establishes dense correspondences between a large number of 3D faces. Starting from automatically detected sparse correspondences on the outer boundary of 3D faces, the algorithm triangulates existing correspondences and expands them iteratively by matching points of distinctive surface curvature along the triangle edges. After exhausting keypoint matches, further correspondences are established by generating evenly distributed points within triangles by evolving level set geodesic curves from the centroids of large triangles. A deformable model (K3DM) is constructed from the dense corresponded faces and an algorithm is proposed for morphing the K3DM to fit unseen faces. This algorithm iterates between rigid alignment of an unseen face followed by regularized morphing of the deformable model. We have extensively evaluated the proposed algorithms on synthetic data and real 3D faces from the FRGCv2, Bosphorus, BU3DFE and UND Ear databases using quantitative and qualitative benchmarks. Our algorithm achieved dense correspondences with a mean localisation error of 1.28mm on synthetic faces and detected $14$ anthropometric landmarks on unseen real faces from the FRGCv2 database with 3mm precision. Furthermore, our deformable model fitting algorithm achieved 98.5% face recognition accuracy on the FRGCv2 and 98.6% on Bosphorus database. Our dense model is also able to generalize to unseen datasets.Comment: 24 Pages, 12 Figures, 6 Tables and 3 Algorithm

Low dimensional Surface Parameterisation with application in biometrics

This paper describes initial results from a novel low dimensional surface parameterisation approach based on a modified iterative closest point (ICP) registration process which uses vertex based principal component analysis (PCA) to incorporate a deformable element into registration process. Using this method a 3D surface is represented by a shape space vector of much smaller dimensionality than the dimensionality of the original data space vector. The proposed method is tested on both simulated 3D faces with different facial expressions and real face data. It is shown that the proposed surface representation can be potentially used as feature space for a facial expression recognition system

3-D Shape Matching for Face Analysis and Recognition

The aims of this paper are to introduce a 3-D shape matching scheme for automatic face recognition and to demonstrate its invariance to pose and facial expressions. The core of this scheme lies on the combination of non-rigid deformation registration and statistical shape modelling. While the former matches 3-D faces regardless of facial expression variations, the latter provides a low-dimensional feature vector that describes the deformation after the shape matching process, thereby enabling robust identification of 3-D faces. In order to assist establishment of accurate dense point correspondences, an isometric embedding shape representation is introduced, which is able to transform 3-D faces to a canonical form that retains the intrinsic geometric structure and achieve shape alignment of 3-D faces independent from individual’s facial expression. The feasibility and effectiveness of the proposed method was investigated using standard publicly available Gavab and BU-3DFE databases, which contain faces expressions and pose variations. The performance of the system was compared with the existing benchmark approaches and it demonstrates that the proposed scheme provides a competitive solution for the face recognition task with real-world practicality