Learning Non-rigid, 3D Shape Variations using Statistical, Physical and Geometric Models

3D shape modelling is a fundamental component in computer vision and computer graphics. Applications include shape interpolation and extrapolation, shape reconstruction, motion capture and mesh editing, etc. By “modelling” we mean the process of learning a parameter-driven model. This thesis focused on the scope of statistical modelling for 3D non-rigid shapes, such as human faces and bodies. The problem is challenging due to highly non-linear deformations, high dimensionality, and data sparsity. Several new algorithms are proposed for 3D shape modelling, 3D shape matching (computing dense correspondence) and applications. First, we propose a variant of Principal Component Analysis called “Shell PCA” which provides a physically-inspired statistical shape model. This is our first attempt to use a physically plausible metric (specifically, the discrete shell model) for statistical shape modelling. Second, we further develop this line of work into a fully Riemannian approach called “Shell PGA”. We demonstrate how to perform Principal Geodesic Analysis in the space of discrete shells. To achieve this, we present an alternate formulation of PGA which avoids working in the tangent space and deals with shapes lying on the manifold directly. Unlike displacement-based methods, Shell PGA is invariant to rigid body motion, and therefore alignment preprocessing such as Procrustes analysis is not needed. Third, we propose a groupwise shape matching method using functional map representation. Targeting at near-isometric deformations, we consider groupwise optimisation of consistent functional maps over a product of Stiefel manifolds, and optimise over a minimal subset of the transformations for efficiency. Last, we show that our proposed shape model achieves state-of-the-art performance in two very challenging applications: handle-based mesh editing, and model fitting using motion capture data. We also contribute a new algorithm for human body shape estimation using clothed scan sequence, along with a new dataset “BUFF” for evaluation