HIGH QUALITY HUMAN 3D BODY MODELING, TRACKING AND APPLICATION

Geometric reconstruction of dynamic objects is a fundamental task of computer vision and graphics, and modeling human body of high fidelity is considered to be a core of this problem. Traditional human shape and motion capture techniques require an array of surrounding cameras or subjects wear reflective markers, resulting in a limitation of working space and portability. In this dissertation, a complete process is designed from geometric modeling detailed 3D human full body and capturing shape dynamics over time using a flexible setup to guiding clothes/person re-targeting with such data-driven models. As the mechanical movement of human body can be considered as an articulate motion, which is easy to guide the skin animation but has difficulties in the reverse process to find parameters from images without manual intervention, we present a novel parametric model, GMM-BlendSCAPE, jointly taking both linear skinning model and the prior art of BlendSCAPE (Blend Shape Completion and Animation for PEople) into consideration and develop a Gaussian Mixture Model (GMM) to infer both body shape and pose from incomplete observations. We show the increased accuracy of joints and skin surface estimation using our model compared to the skeleton based motion tracking. To model the detailed body, we start with capturing high-quality partial 3D scans by using a single-view commercial depth camera. Based on GMM-BlendSCAPE, we can then reconstruct multiple complete static models of large pose difference via our novel non-rigid registration algorithm. With vertex correspondences established, these models can be further converted into a personalized drivable template and used for robust pose tracking in a similar GMM framework. Moreover, we design a general purpose real-time non-rigid deformation algorithm to accelerate this registration. Last but not least, we demonstrate a novel virtual clothes try-on application based on our personalized model utilizing both image and depth cues to synthesize and re-target clothes for single-view videos of different people

Reconstruction 3D personnalisée de la colonne vertébrale à partir d'images radiographiques non-calibrées

Les systèmes de reconstruction stéréo-radiographique 3D -- La colonne vertébrale -- La scoliose idiopathique adolescente -- Évolution des systèmes de reconstruction 3D -- Filtres de rehaussement d'images -- Techniques de segmentation -- Les méthodes de calibrage -- Les méthodes de reconstruction 3D -- Problématique, hypothèses, objectifs et méthode générale -- Three-dimensional reconstruction of the scoliotic spine and pelvis from uncalibrated biplanar X-ray images -- A versatile 3D reconstruction system of the spine and pelvis for clinical assessment of spinal deformities -- Simulation experiments -- Clinical validation -- A three-dimensional retrospective analysis of the evolution of spinal instrumentation for the correction of adolescent idiopathic scoliosis -- Auto-calibrage d'un système à rayons-X à partir de primitives de haut niveau -- Segmentation de la colonne vertébrale -- Approche hiérarchique d'auto-calibrage d'un système d'acquisition à rayons-X -- Personalized 3D reconstruction of the scoliotic spine from hybrid statistical and X-ray image-based models -- Validation protocol

Understanding the Structure of 3D Shapes

Compact representations of three dimensional objects are very often used in computer graphics to create effective ways to analyse, manipulate and transmit 3D models. Their ability to abstract from the concrete shapes and expose their structure is important in a number of applications, spanning from computer animation, to medicine, to physical simulations. This thesis will investigate new methods for the generation of compact shape representations. In the first part, the problem of computing optimal PolyCube base complexes will be considered. PolyCubes are orthogonal polyhedra used in computer graphics to map both surfaces and volumes. Their ability to resemble the original models and at the same time expose a very simple and regular structure is important in a number of applications, such as texture mapping, spline fitting and hex-meshing. The second part will focus on medial descriptors. In particular, two new algorithms for the generation of curve-skeletons will be presented. These methods are completely based on the visual appearance of the input, therefore they are independent from the type, number and quality of the primitives used to describe a shape, determining, thus, an advancement to the state of the art in the field

Understanding the Structure of 3D Shapes

Compact representations of three dimensional objects are very often used in computer graphics to create effective ways to analyse, manipulate and transmit 3D models. Their ability to abstract from the concrete shapes and expose their structure is important in a number of applications, spanning from computer animation, to medicine, to physical simulations. This thesis will investigate new methods for the generation of compact shape representations. In the first part, the problem of computing optimal PolyCube base complexes will be considered. PolyCubes are orthogonal polyhedra used in computer graphics to map both surfaces and volumes. Their ability to resemble the original models and at the same time expose a very simple and regular structure is important in a number of applications, such as texture mapping, spline fitting and hex-meshing. The second part will focus on medial descriptors. In particular, two new algorithms for the generation of curve-skeletons will be presented. These methods are completely based on the visual appearance of the input, therefore they are independent from the type, number and quality of the primitives used to describe a shape, determining, thus, an advancement to the state of the art in the field

{CurveFusion}: {R}econstructing Thin Structures from {RGBD} Sequences

We introduce CurveFusion, the first approach for high quality scanning of thin structures at interactive rates using a handheld RGBD camera. Thin filament-like structures are mathematically just 1D curves embedded in R^3, and integration-based reconstruction works best when depth sequences (from the thin structure parts) are fused using the object's (unknown) curve skeleton. Thus, using the complementary but noisy color and depth channels, CurveFusion first automatically identifies point samples on potential thin structures and groups them into bundles, each being a group of a fixed number of aligned consecutive frames. Then, the algorithm extracts per-bundle skeleton curves using L1 axes, and aligns and iteratively merges the L1 segments from all the bundles to form the final complete curve skeleton. Thus, unlike previous methods, reconstruction happens via integration along a data-dependent fusion primitive, i.e., the extracted curve skeleton. We extensively evaluate CurveFusion on a range of challenging examples, different scanner and calibration settings, and present high fidelity thin structure reconstructions previously just not possible from raw RGBD sequences