Fitting 3D Morphable Models using Local Features

In this paper, we propose a novel fitting method that uses local image features to fit a 3D Morphable Model to 2D images. To overcome the obstacle of optimising a cost function that contains a non-differentiable feature extraction operator, we use a learning-based cascaded regression method that learns the gradient direction from data. The method allows to simultaneously solve for shape and pose parameters. Our method is thoroughly evaluated on Morphable Model generated data and first results on real data are presented. Compared to traditional fitting methods, which use simple raw features like pixel colour or edge maps, local features have been shown to be much more robust against variations in imaging conditions. Our approach is unique in that we are the first to use local features to fit a Morphable Model. Because of the speed of our method, it is applicable for realtime applications. Our cascaded regression framework is available as an open source library (https://github.com/patrikhuber).Comment: Submitted to ICIP 2015; 4 pages, 4 figure

A 3D Face Modelling Approach for Pose-Invariant Face Recognition in a Human-Robot Environment

Face analysis techniques have become a crucial component of human-machine interaction in the fields of assistive and humanoid robotics. However, the variations in head-pose that arise naturally in these environments are still a great challenge. In this paper, we present a real-time capable 3D face modelling framework for 2D in-the-wild images that is applicable for robotics. The fitting of the 3D Morphable Model is based exclusively on automatically detected landmarks. After fitting, the face can be corrected in pose and transformed back to a frontal 2D representation that is more suitable for face recognition. We conduct face recognition experiments with non-frontal images from the MUCT database and uncontrolled, in the wild images from the PaSC database, the most challenging face recognition database to date, showing an improved performance. Finally, we present our SCITOS G5 robot system, which incorporates our framework as a means of image pre-processing for face analysis

Fitting a 3D Morphable Model to Edges: A Comparison Between Hard and Soft Correspondences

We propose a fully automatic method for fitting a 3D morphable model to single face images in arbitrary pose and lighting. Our approach relies on geometric features (edges and landmarks) and, inspired by the iterated closest point algorithm, is based on computing hard correspondences between model vertices and edge pixels. We demonstrate that this is superior to previous work that uses soft correspondences to form an edge-derived cost surface that is minimised by nonlinear optimisation.Comment: To appear in ACCV 2016 Workshop on Facial Informatic

Criação de modelos faciais 3D através de fotografias

Nos dias de hoje, a qualidade da semelhança entre as personagens em videojogos e o seu correspondente em carne e osso está cada vez mais elevada. A necessidade de uma modelação rápida, eficaz e económica é então cada vez mais requisitada. A solução que melhor conjuga essas 3 qualidades é sem dúvida a modelação facial através de fotografias. O primeiro artigo publicado nesta área foi em 1999, e têm sido feitas muitas melhorias aos sistemas desenvolvidos, no entanto, ainda existem muitas falhas. Falhas quer em termos de qualidade, semelhança e tempo. Com este projeto de dissertação pretendia-se então combater essas lacunas desenvolvendo uma ferramenta capaz de criar modelos faciais através de fotografias e posterior alteração de diversos parâmetros faciais. Os objetivos foram maioritariamente cumpridos, apresentando-se então uma biblioteca capaz de criar modelos 3D a partir de uma fotografia com um rosto, e alterar expressões faciais no mesmo, bem como manipular visualmente o modelo e gravá-lo para um ficheiro. Os utilizadores que testaram a aplicação gráfica apreciaram a sua rapidez na criação no modelo, no entanto não se mostraram tão satisfeitos relativamente à quantidade de ferramentas disponíveis para a sua edição.Nowadays, the similarity quality between video game characters and their real-life counterpart is getting higher and higher. The demand for fast, efficient and cheap 3D modeling is being highly required. The solution that best combines these aspects is undoubtedly the modeling through the use of face pictures. The first article published in this area was in 1999, and many improvements have been made ever since, but there were still many flaws to cover in those systems. The flaws include quality issues, low similarity, and high execution time. Therefore, with this work, it’s intended to combat those flaws, by creating a framework capable of creating such 3D models and also adjust various facial parameters. The majority of the objectives drawn were accomplished, presenting a framework capable of creating 3D models by using a facial picture, and changing their facial expressions, as well has visually manipulating the 3D model and writing it to a file. The users that tested the graphical application appreciated the fast creation of the model, but were not so convinced by the number of editing tools at their disposal