High-fidelity Human Body Modelling from User-generated Data

PhD thesisBuilding high-fidelity human body models for real people benefits a variety of applications, like fashion, health, entertainment, education and ergonomics applications. The goal of this thesis is to build visually plausible human body models from two kinds of user-generated data: low-quality point clouds and low-resolution 2D images. Due to the advances in 3D scanning technology and the growing availability of cost-effective 3D scanners to general users, a full human body scan can be easily acquired within two minutes. However, due to the imperfections of scanning devices, occlusion, self-occlusion and untrained scanning operation, the acquired scans tend to be full of noise, holes (missing data), outliers and distorted parts. In this thesis, the establishment of shape correspondences for human body meshes is firstly investigated. A robust and shape-aware approach is proposed to detect accurate shape correspondences for closed human body meshes. By investigating the vertex movements of 200 human body meshes, a robust non-rigid mesh registration method is proposed which combines the human body shape model with the traditional nonrigid ICP. To facilitate the development and benchmarking of registration methods on Kinect Fusion data, a dataset of user-generated scansis built, named Kinect-based 3D Human Body (K3D-hub) Dataset, with one Microsoft Kinect for XBOX 360. Besides building 3D human body models from point clouds, the problem is also tackled which estimates accurate 3D human body models from single 2D images. A state-of-the-art parametric 3D human body model SMPL is fitted to 2D joints as well as the boundary of the human body. Fast Region based CNN and deep CNN based methods are adopted to detect the 2D joints and boundary for each human body image automatically. Considering the commonly encountered scenario where people are in stable poses at most of the time, a stable pose prior is introduced from CMU motion capture (mocap) dataset for further improving the accuracy of pose estimation

Continuous and Orientation-preserving Correspondences via Functional Maps

We propose a method for efficiently computing orientation-preserving and approximately continuous correspondences between non-rigid shapes, using the functional maps framework. We first show how orientation preservation can be formulated directly in the functional (spectral) domain without using landmark or region correspondences and without relying on external symmetry information. This allows us to obtain functional maps that promote orientation preservation, even when using descriptors, that are invariant to orientation changes. We then show how higher quality, approximately continuous and bijective pointwise correspondences can be obtained from initial functional maps by introducing a novel refinement technique that aims to simultaneously improve the maps both in the spectral and spatial domains. This leads to a general pipeline for computing correspondences between shapes that results in high-quality maps, while admitting an efficient optimization scheme. We show through extensive evaluation that our approach improves upon state-of-the-art results on challenging isometric and non-isometric correspondence benchmarks according to both measures of continuity and coverage as well as producing semantically meaningful correspondences as measured by the distance to ground truth maps.Comment: 16 pages, 22 figure

Robust Nonrigid Registration by Convex Optimization

We present an approach to nonrigid registration of 3D surfaces. We cast isometric embedding as MRF opti-mization and apply efficient global optimization algorithms based on linear programming relaxations. The Markov ran-dom field perspective suggests a natural connection with robust statistics and motivates robust forms of the intrinsic distortion functional. Our approach outperforms a large body of prior work by a significant margin, increasing reg-istration precision on real data by a factor of 3. 1

Improving Sampling-Based Motion Planning Using Library of Trajectories

Plánování pohybu je jedním z podstatných problémů robotiky. Tato práce kombinuje pokroky v plánování pohybu a hodnocení podobnosti objektů za účelem zrychlení plánování ve statických prostředích. První část této práce pojednává o současných metodách používaných pro hodnocení podobnosti objektů a plánování pohybu. Prostřední část popisuje, jak jsou tyto metody použity pro zrychlení plánování s využitím získaných znalostí o prostředí. V poslední části jsou navržené metody porovnány s ostatními plánovači v nezávislém testu. Námi navržené algoritmy se v experimentech ukázaly být často rychlejší v porovnání s ostatními plánovači. Také často nacházely cesty v prostředích, kde ostatní plánovače nebyly schopny cestu nalézt.Motion planning is one of the fundamental problems in robotics. This thesis combines the advances in motion planning and shape matching to improve planning speeds in static environments. The first part of this thesis covers current methods used in object similarity evaluation and motion planning. The middle part describes how these methods are used together to improve planning speeds by utilizing prior knowledge about the environment, along with additional modifications. In the last part, the proposed methods are tested against other state-of-the-art planners in an independent benchmarking facility. The proposed algorithms are shown to be faster than other planners in many cases, often finding paths in environments where the other planners are unable to

CCuantuMM: Cycle-Consistent Quantum-Hybrid Matching of Multiple Shapes

Jointly matching multiple, non-rigidly deformed 3D shapes is a challenging, $\mathcal{NP}$-hard problem. A perfect matching is necessarily cycle-consistent: Following the pairwise point correspondences along several shapes must end up at the starting vertex of the original shape. Unfortunately, existing quantum shape-matching methods do not support multiple shapes and even less cycle consistency. This paper addresses the open challenges and introduces the first quantum-hybrid approach for 3D shape multi-matching; in addition, it is also cycle-consistent. Its iterative formulation is admissible to modern adiabatic quantum hardware and scales linearly with the total number of input shapes. Both these characteristics are achieved by reducing the $N$-shape case to a sequence of three-shape matchings, the derivation of which is our main technical contribution. Thanks to quantum annealing, high-quality solutions with low energy are retrieved for the intermediate $\mathcal{NP}$-hard objectives. On benchmark datasets, the proposed approach significantly outperforms extensions to multi-shape matching of a previous quantum-hybrid two-shape matching method and is on-par with classical multi-matching methods.Comment: Computer Vision and Pattern Recognition (CVPR) 2023; 22 pages, 24 figures and 5 tables; Project page: https://4dqv.mpi-inf.mpg.de/CCuantuMM

Improving Descriptors for 3D Shape Matching

Ph.DDOCTOR OF PHILOSOPH