Error-Controlled Model Approximation for Gaussian Process Morphable Models

Gaussian Process Morphable Models (GPMMs) unify a variety of non-rigid deformation models for surface and image registration. Deformation models, such as B-splines, radial basis functions, and PCA models are defined as a probability distribution using a Gaussian process. The method depends heavily on the low-rank approximation of the Gaussian process, which is mandatory to obtain a parametric representation of the model. In this article, we propose the use of the pivoted Cholesky decomposition for this task, which has the following advantages: (1) Compared to the current state of the art used in GPMMs, it provides a fully controllable approximation error. The algorithm greedily computes new basis functions until the user-defined approximation accuracy is reached. (2) Unlike the currently used approach, this method can be used in a black-box-like scenario, whereas the method automatically chooses the amount of basis functions for a given model and accuracy. (3) We propose the Newton basis as an alternative basis for GPMMs. The proposed basis does not need an SVD computation and can be iteratively refined. We show that the proposed basis functions achieve competitive registration results while providing the mentioned advantages for its computation

Morphable Face Models - An Open Framework

In this paper, we present a novel open-source pipeline for face registration based on Gaussian processes as well as an application to face image analysis. Non-rigid registration of faces is significant for many applications in computer vision, such as the construction of 3D Morphable face models (3DMMs). Gaussian Process Morphable Models (GPMMs) unify a variety of non-rigid deformation models with B-splines and PCA models as examples. GPMM separate problem specific requirements from the registration algorithm by incorporating domain-specific adaptions as a prior model. The novelties of this paper are the following: (i) We present a strategy and modeling technique for face registration that considers symmetry, multi-scale and spatially-varying details. The registration is applied to neutral faces and facial expressions. (ii) We release an open-source software framework for registration and model-building, demonstrated on the publicly available BU3D-FE database. The released pipeline also contains an implementation of an Analysis-by-Synthesis model adaption of 2D face images, tested on the Multi-PIE and LFW database. This enables the community to reproduce, evaluate and compare the individual steps of registration to model-building and 3D/2D model fitting. (iii) Along with the framework release, we publish a new version of the Basel Face Model (BFM-2017) with an improved age distribution and an additional facial expression model

Efficient computation of low-rank Gaussian process models for surface and image registration

Gaussian Process Morphable Models (GPMMs) are a unifying approach to non-rigid surface and image registration, where a deformation prior is defined using a Gaussian process. By a simple exchange of the covariance function we can formulate a wide variety of different deformation priors, such as spline-based models, free-form deformations or statistical shape and deformation models. How well the method works in practical applications depends crucially on how well a low-rank approximation of the Gaussian process can be computed. In this article we propose the use of the pivoted Cholesky decomposition for this task. This method makes it possible to efficiently compute a low-rank approximation for very large point sets, such as given by 3D meshes or 3D image grids, with a rigorously controlled approximation error. Compared to the current state of the art, which is based on the Nystro ̈m method, the approximation error is controllable and can be specified by a user-defined threshold. Further we propose a computationally more efficient and greedy alternative to currently used Karhunen-Loève expansion. This makes it possible to compute more accurate model approximations at the same computational costs. Detailed experiments from the registration of high quality human face scans and medical CT images containing the forearm with Ulna and Radius demonstrate the efficiency of the method and the computational advantages over the Nyström method

{3D} Morphable Face Models -- Past, Present and Future

In this paper, we provide a detailed survey of 3D Morphable Face Models over the 20 years since they were first proposed. The challenges in building and applying these models, namely capture, modeling, image formation, and image analysis, are still active research topics, and we review the state-of-the-art in each of these areas. We also look ahead, identifying unsolved challenges, proposing directions for future research and highlighting the broad range of current and future applications

Face recognition with image sets using manifold density divergence

In many automatic face recognition applications, a set of a person\u27s face images is available rather than a single image. In this paper, we describe a novel method for face recognition using image sets. We propose a flexible, semi-parametric model for learning probability densities confined to highly non-linear but intrinsically low-dimensional manifolds. The model leads to a statistical formulation of the recognition problem in terms of minimizing the divergence between densities estimated on these manifolds. The proposed method is evaluated on a large data set, acquired in realistic imaging conditions with severe illumination variation. Our algorithm is shown to match the best and outperform other state-of-the-art algorithms in the literature, achieving 94% recognition rate on average

PhoMoH: Implicit Photorealistic 3D Models of Human Heads

We present PhoMoH, a neural network methodology to construct generative models of photo-realistic 3D geometry and appearance of human heads including hair, beards, an oral cavity, and clothing. In contrast to prior work, PhoMoH models the human head using neural fields, thus supporting complex topology. Instead of learning a head model from scratch, we propose to augment an existing expressive head model with new features. Concretely, we learn a highly detailed geometry network layered on top of a mid-resolution head model together with a detailed, local geometry-aware, and disentangled color field. Our proposed architecture allows us to learn photo-realistic human head models from relatively little data. The learned generative geometry and appearance networks can be sampled individually and enable the creation of diverse and realistic human heads. Extensive experiments validate our method qualitatively and across different metrics.Comment: To be published at the International Conference on 3D Vision 202

A global-to-local model for the representation of human faces

In the context of face modeling and face recognition, statistical models are widely used for the representation and modeling of surfaces. Most of these models are obtained by computing Principal Components Analysis (PCA) on a set of representative examples. These models represent novel faces poorly due to their holistic nature (i.e.\ each component has global support), and they suffer from overfitting when used for generalization from partial information. In this work, we present a novel analysis method that breaks the objects up into modes based on spatial frequency. The high-frequency modes are segmented into regions with respect to specific features of the object. After computing PCA on these segments individually, a hierarchy of global and local components gradually decreasing in size of their support is combined into a linear statistical model, hence the name, Global-to-Local model (G2L). We apply our methodology to build a novel G2L model of 3D shapes of human heads. Both the representation and the generalization capabilities of the models are evaluated and compared in a standardized test, and it is demonstrated that the G2L model performs better compared to traditional holistic PCA models. Furthermore, both models are used to reconstruct the 3D shape of faces from a single photograph. A novel adaptive fitting method is presented that estimates the model parameters using a multi-resolution approach. The model is first fitted to contours extracted from the image. In a second stage, the contours are kept fixed and the remaining flexibility of the model is fitted to the input image. This makes the method fast (30 sec on a standard PC), efficient, and accurate

A machine learning approach to statistical shape models with applications to medical image analysis

Statistical shape models have become an indispensable tool for image analysis. The use of shape models is especially popular in computer vision and medical image analysis, where they were incorporated as a prior into a wide range of different algorithms. In spite of their big success, the study of statistical shape models has not received much attention in recent years. Shape models are often seen as an isolated technique, which merely consists of applying Principal Component Analysis to a set of example data sets. In this thesis we revisit statistical shape models and discuss their construction and applications from the perspective of machine learning and kernel methods. The shapes that belong to an object class are modeled as a Gaussian Process whose parameters are estimated from example data. This formulation puts statistical shape models in a much wider context and makes the powerful inference tools from learning theory applicable to shape modeling. Furthermore, the formulation is continuous and thus helps to avoid discretization issues, which often arise with discrete models. An important step in building statistical shape models is to establish surface correspondence. We discuss an approach which is based on kernel methods. This formulation allows us to integrate the statistical shape model as an additional prior. It thus unifies the methods of registration and shape model fitting. Using Gaussian Process regression we can integrate shape constraints in our model. These constraints can be used to enforce landmark matching in the fitting or correspondence problem. The same technique also leads directly to a new solution for shape reconstruction from partial data. In addition to experiments on synthetic 2D data sets, we show the applicability of our methods on real 3D medical data of the human head. In particular, we build a 3D model of the human skull, and present its applications for the planning of cranio-facial surgeries