Analysis of 3D Face Reconstruction

This thesis investigates the long standing problem of 3D reconstruction from a single 2D face image. Face reconstruction from a single 2D face image is an ill posed problem involving estimation of the intrinsic and the extrinsic camera parameters, light parameters, shape parameters and the texture parameters. The proposed approach has many potential applications in the law enforcement, surveillance, medicine, computer games and the entertainment industries. This problem is addressed using an analysis by synthesis framework by reconstructing a 3D face model from identity photographs. The identity photographs are a widely used medium for face identi cation and can be found on identity cards and passports. The novel contribution of this thesis is a new technique for creating 3D face models from a single 2D face image. The proposed method uses the improved dense 3D correspondence obtained using rigid and non-rigid registration techniques. The existing reconstruction methods use the optical ow method for establishing 3D correspondence. The resulting 3D face database is used to create a statistical shape model. The existing reconstruction algorithms recover shape by optimizing over all the parameters simultaneously. The proposed algorithm simplifies the reconstruction problem by using a step wise approach thus reducing the dimension of the parameter space and simplifying the opti- mization problem. In the alignment step, a generic 3D face is aligned with the given 2D face image by using anatomical landmarks. The texture is then warped onto the 3D model by using the spatial alignment obtained previously. The 3D shape is then recovered by optimizing over the shape parameters while matching a texture mapped model to the target image. There are a number of advantages of this approach. Firstly, it simpli es the optimization requirements and makes the optimization more robust. Second, there is no need to accurately recover the illumination parameters. Thirdly, there is no need for recovering the texture parameters by using a texture synthesis approach. Fourthly, quantitative analysis is used for improving the quality of reconstruction by improving the cost function. Previous methods use qualitative methods such as visual analysis, and face recognition rates for evaluating reconstruction accuracy. The improvement in the performance of the cost function occurs as a result of improvement in the feature space comprising the landmark and intensity features. Previously, the feature space has not been evaluated with respect to reconstruction accuracy thus leading to inaccurate assumptions about its behaviour. The proposed approach simpli es the reconstruction problem by using only identity images, rather than placing eff ort on overcoming the pose, illumination and expression (PIE) variations. This makes sense, as frontal face images under standard illumination conditions are widely available and could be utilized for accurate reconstruction. The reconstructed 3D models with texture can then be used for overcoming the PIE variations

The role of saliencey and error propagation in visual object recognition

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Civil and Environmental Engineering, 1995.Includes bibliographical references (p. 162-171).by Tao Daniel Alter.Ph.D

Registration of Optical Images to 3D Medical Images

The work described in this thesis deals with the registration of single and multiple 2-dimensional (2D) optical images to a single 3-dimensional (3D) medical image such as a magnetic resonance or computed tomography scan. The approach is to develop an intensity based method using an information theoretic framework, as opposed to the more typical feature or surface based methods. Relevant camera calibration and pose estimation literature is reviewed, along with medical 2D-3D image registration. An initial algorithm is developed, which performs registration by iteratively maximising the mutual information of a rendered image and a single optical image. The framework is extended to incorporate information from multiple optical and rendered images which signi cantly improves registration performance. A tracking algorithm is proposed, which augments this framework with texture mapping as a means of achieving alignment over a sequence of optical images. These methods are tested using images of skull phantoms and volunteers. A new measure based on the concept of photo-consistency, used in the surface reconstruction literature, is proposed as a measure of image alignment. The relevant theory is developed. This new method is tested using a variety of different photo-consistency based similarity measures, optical images, different numbers of images, images with varying amounts of added noise, different resolutions and different camera positions relative to the object of interest. In almost all cases, similarity measures based on this new framework perform accurately, precisely and robustly. Potential applications will be in radiotherapy patient positioning, image guided craniofacial, skull base and neurosurgery, computer vision and robotics, where the accurate alignment between a 3D image or model and multiple 2D optical images is required