Segmentation of 3D meshes combining the artificial neural network classifier and the spectral clustering

3D mesh segmentation has become an essential step in many applications in 3D shape analysis. In this paper, a new segmentation method is proposed based on a learning approach using the artificial neural networks classifier and the spectral clustering for segmentation. Firstly, a training step is done using the artificial neural network trained on existing segmentation, taken from the ground truth segmentation (done by humane operators) available in the benchmark proposed by Chen et al. to extract the candidate boundaries of a given 3D-model based on a set of geometric criteria. Then, we use this resulted knowledge to construct a new connectivity of the mesh and use the spectral clustering method to segment the 3D mesh into significant parts. Our approach was evaluated using different evaluation metrics. The experiments confirm that the proposed method yields significantly good results and outperforms some of the competitive segmentation methods in the literature.We would first like to thank Xiaobai Chen et al. for providing the Princeton segmentation benchmark (http://segeval.cs.princeton.edu/). We also would like to thank Liu and Hao Zhang et al. for helping us by providing the binary of the AEI and helping us to evaluate our method

A Combinatorial Solution to Non-Rigid 3D Shape-to-Image Matching

We propose a combinatorial solution for the problem of non-rigidly matching a 3D shape to 3D image data. To this end, we model the shape as a triangular mesh and allow each triangle of this mesh to be rigidly transformed to achieve a suitable matching to the image. By penalising the distance and the relative rotation between neighbouring triangles our matching compromises between image and shape information. In this paper, we resolve two major challenges: Firstly, we address the resulting large and NP-hard combinatorial problem with a suitable graph-theoretic approach. Secondly, we propose an efficient discretisation of the unbounded 6-dimensional Lie group SE(3). To our knowledge this is the first combinatorial formulation for non-rigid 3D shape-to-image matching. In contrast to existing local (gradient descent) optimisation methods, we obtain solutions that do not require a good initialisation and that are within a bound of the optimal solution. We evaluate the proposed method on the two problems of non-rigid 3D shape-to-shape and non-rigid 3D shape-to-image registration and demonstrate that it provides promising results.Comment: 10 pages, 7 figure

BodyNet: Volumetric Inference of 3D Human Body Shapes

Human shape estimation is an important task for video editing, animation and fashion industry. Predicting 3D human body shape from natural images, however, is highly challenging due to factors such as variation in human bodies, clothing and viewpoint. Prior methods addressing this problem typically attempt to fit parametric body models with certain priors on pose and shape. In this work we argue for an alternative representation and propose BodyNet, a neural network for direct inference of volumetric body shape from a single image. BodyNet is an end-to-end trainable network that benefits from (i) a volumetric 3D loss, (ii) a multi-view re-projection loss, and (iii) intermediate supervision of 2D pose, 2D body part segmentation, and 3D pose. Each of them results in performance improvement as demonstrated by our experiments. To evaluate the method, we fit the SMPL model to our network output and show state-of-the-art results on the SURREAL and Unite the People datasets, outperforming recent approaches. Besides achieving state-of-the-art performance, our method also enables volumetric body-part segmentation.Comment: Appears in: European Conference on Computer Vision 2018 (ECCV 2018). 27 page