Towards reconstructing a 3D face model from an uncontrolled video sequence

A pipeline for reconstructing the 3D face model from an uncontrolled video sequence is presented which involves three major steps. Firstly, a generic deformable 3D face model is built from the 3D scans of one hundred individuals. Secondly, the 3D faceshape from a video sequence is constructed by estimating poses of images using structure-from-motion technique and dense correspondences between those images by employing Huber-L1 optical flow algorithm. Finally, the generated generic deformable 3D face model can be fitted to the reconstructed 3D face-shape from a video sequence provided that the deviation from the real 3D face is less than certain thresholds. The application is developed to reconstruct the 3D face-shape in nearly uncontrolled environment so the results cannot be expected to be very accurate. We discuss the steps taken to perform the first and second steps. The factors affecting the depth estimation in face region cause major accuracy problems. They are analyzed and possible improvements to enhance the 3D face-shape reconstruction are presented