1,890 research outputs found
Neural 3D Morphable Models: Spiral Convolutional Networks for 3D Shape Representation Learning and Generation
Generative models for 3D geometric data arise in many important applications
in 3D computer vision and graphics. In this paper, we focus on 3D deformable
shapes that share a common topological structure, such as human faces and
bodies. Morphable Models and their variants, despite their linear formulation,
have been widely used for shape representation, while most of the recently
proposed nonlinear approaches resort to intermediate representations, such as
3D voxel grids or 2D views. In this work, we introduce a novel graph
convolutional operator, acting directly on the 3D mesh, that explicitly models
the inductive bias of the fixed underlying graph. This is achieved by enforcing
consistent local orderings of the vertices of the graph, through the spiral
operator, thus breaking the permutation invariance property that is adopted by
all the prior work on Graph Neural Networks. Our operator comes by construction
with desirable properties (anisotropic, topology-aware, lightweight,
easy-to-optimise), and by using it as a building block for traditional deep
generative architectures, we demonstrate state-of-the-art results on a variety
of 3D shape datasets compared to the linear Morphable Model and other graph
convolutional operators.Comment: to appear at ICCV 201
An Implementation of Fully Convolutional Network for Surface Mesh Segmentation
This thesis presents an implementation of a 3-Dimensional triangular surface mesh segmentation architecture named Shape Fully Convolutional Network, which is proposed by Pengyu Wang and Yuan Gan in 2018. They designed a deep neural network that has a similar architecture as the Fully Convolutional Network, which provides a good segmentation result for 2D images, on 3D triangular surface meshes. In their implementation, 3D surface meshes are represented as graph structures to feed the network. There are three main barriers when applying the Fully Convolutional Network to graph-based data structures.
• First, the pooling operation is much harder to apply to general graphs.
• Second, the convolution order on a graph structure is unstable.
• Third, the raw data of surface meshes cannot be directly applied to the network.
To solve these problems, first, all the nodes inside the graph are re-ordered into a 1-dimensional list based on a multi-level graph coarsening algorithm, which allows the pooling operation to be applied as easily as a 1D pooling. Second, a self-defined generating layer is added before each convolution layer in the network to generate the neighbors of each node on the graph, and at the same time, sort all neighbors based on the L2 similarity to keep the convolution operation in a stable manner. Finally, three translation and rotation free low-level geometric features are pre-processed and used as input to train the network. This Shape Fully Convolution Network can effectively learn and predict triangular face-wise labels. In the end, to achieve a better result, the final labeling is optimized by the multi-label graph cut algorithm, which gives punishment to the predicted result based on the smoothness of the surface. The experiments show that the model can effectively learn and predict triangle-wise labels on surface meshes and yields good segmentation results
Geometric deep learning: going beyond Euclidean data
Many scientific fields study data with an underlying structure that is a
non-Euclidean space. Some examples include social networks in computational
social sciences, sensor networks in communications, functional networks in
brain imaging, regulatory networks in genetics, and meshed surfaces in computer
graphics. In many applications, such geometric data are large and complex (in
the case of social networks, on the scale of billions), and are natural targets
for machine learning techniques. In particular, we would like to use deep
neural networks, which have recently proven to be powerful tools for a broad
range of problems from computer vision, natural language processing, and audio
analysis. However, these tools have been most successful on data with an
underlying Euclidean or grid-like structure, and in cases where the invariances
of these structures are built into networks used to model them. Geometric deep
learning is an umbrella term for emerging techniques attempting to generalize
(structured) deep neural models to non-Euclidean domains such as graphs and
manifolds. The purpose of this paper is to overview different examples of
geometric deep learning problems and present available solutions, key
difficulties, applications, and future research directions in this nascent
field
- …