29 research outputs found
Deformable Shape Completion with Graph Convolutional Autoencoders
The availability of affordable and portable depth sensors has made scanning
objects and people simpler than ever. However, dealing with occlusions and
missing parts is still a significant challenge. The problem of reconstructing a
(possibly non-rigidly moving) 3D object from a single or multiple partial scans
has received increasing attention in recent years. In this work, we propose a
novel learning-based method for the completion of partial shapes. Unlike the
majority of existing approaches, our method focuses on objects that can undergo
non-rigid deformations. The core of our method is a variational autoencoder
with graph convolutional operations that learns a latent space for complete
realistic shapes. At inference, we optimize to find the representation in this
latent space that best fits the generated shape to the known partial input. The
completed shape exhibits a realistic appearance on the unknown part. We show
promising results towards the completion of synthetic and real scans of human
body and face meshes exhibiting different styles of articulation and
partiality.Comment: CVPR 201
3D Shape Completion with Multi-view Consistent Inference
3D shape completion is important to enable machines to perceive the complete
geometry of objects from partial observations. To address this problem,
view-based methods have been presented. These methods represent shapes as
multiple depth images, which can be back-projected to yield corresponding 3D
point clouds, and they perform shape completion by learning to complete each
depth image using neural networks. While view-based methods lead to
state-of-the-art results, they currently do not enforce geometric consistency
among the completed views during the inference stage. To resolve this issue, we
propose a multi-view consistent inference technique for 3D shape completion,
which we express as an energy minimization problem including a data term and a
regularization term. We formulate the regularization term as a consistency loss
that encourages geometric consistency among multiple views, while the data term
guarantees that the optimized views do not drift away too much from a learned
shape descriptor. Experimental results demonstrate that our method completes
shapes more accurately than previous techniques.Comment: Accepted to AAAI 2020 as oral presentatio
Adaptive Density Estimation for Generative Models
Unsupervised learning of generative models has seen tremendous progress over
recent years, in particular due to generative adversarial networks (GANs),
variational autoencoders, and flow-based models. GANs have dramatically
improved sample quality, but suffer from two drawbacks: (i) they mode-drop,
i.e., do not cover the full support of the train data, and (ii) they do not
allow for likelihood evaluations on held-out data. In contrast,
likelihood-based training encourages models to cover the full support of the
train data, but yields poorer samples. These mutual shortcomings can in
principle be addressed by training generative latent variable models in a
hybrid adversarial-likelihood manner. However, we show that commonly made
parametric assumptions create a conflict between them, making successful hybrid
models non trivial. As a solution, we propose to use deep invertible
transformations in the latent variable decoder. This approach allows for
likelihood computations in image space, is more efficient than fully invertible
models, and can take full advantage of adversarial training. We show that our
model significantly improves over existing hybrid models: offering GAN-like
samples, IS and FID scores that are competitive with fully adversarial models,
and improved likelihood scores