2,425 research outputs found
Geometry-Aware Face Completion and Editing
Face completion is a challenging generation task because it requires
generating visually pleasing new pixels that are semantically consistent with
the unmasked face region. This paper proposes a geometry-aware Face Completion
and Editing NETwork (FCENet) by systematically studying facial geometry from
the unmasked region. Firstly, a facial geometry estimator is learned to
estimate facial landmark heatmaps and parsing maps from the unmasked face
image. Then, an encoder-decoder structure generator serves to complete a face
image and disentangle its mask areas conditioned on both the masked face image
and the estimated facial geometry images. Besides, since low-rank property
exists in manually labeled masks, a low-rank regularization term is imposed on
the disentangled masks, enforcing our completion network to manage occlusion
area with various shape and size. Furthermore, our network can generate diverse
results from the same masked input by modifying estimated facial geometry,
which provides a flexible mean to edit the completed face appearance. Extensive
experimental results qualitatively and quantitatively demonstrate that our
network is able to generate visually pleasing face completion results and edit
face attributes as well
Accurate Optical Flow via Direct Cost Volume Processing
We present an optical flow estimation approach that operates on the full
four-dimensional cost volume. This direct approach shares the structural
benefits of leading stereo matching pipelines, which are known to yield high
accuracy. To this day, such approaches have been considered impractical due to
the size of the cost volume. We show that the full four-dimensional cost volume
can be constructed in a fraction of a second due to its regularity. We then
exploit this regularity further by adapting semi-global matching to the
four-dimensional setting. This yields a pipeline that achieves significantly
higher accuracy than state-of-the-art optical flow methods while being faster
than most. Our approach outperforms all published general-purpose optical flow
methods on both Sintel and KITTI 2015 benchmarks.Comment: Published at the Conference on Computer Vision and Pattern
Recognition (CVPR 2017
Dictionary Learning-based Inpainting on Triangular Meshes
The problem of inpainting consists of filling missing or damaged regions in
images and videos in such a way that the filling pattern does not produce
artifacts that deviate from the original data. In addition to restoring the
missing data, the inpainting technique can also be used to remove undesired
objects. In this work, we address the problem of inpainting on surfaces through
a new method based on dictionary learning and sparse coding. Our method learns
the dictionary through the subdivision of the mesh into patches and rebuilds
the mesh via a method of reconstruction inspired by the Non-local Means method
on the computed sparse codes. One of the advantages of our method is that it is
capable of filling the missing regions and simultaneously removes noise and
enhances important features of the mesh. Moreover, the inpainting result is
globally coherent as the representation based on the dictionaries captures all
the geometric information in the transformed domain. We present two variations
of the method: a direct one, in which the model is reconstructed and restored
directly from the representation in the transformed domain and a second one,
adaptive, in which the missing regions are recreated iteratively through the
successive propagation of the sparse code computed in the hole boundaries,
which guides the local reconstructions. The second method produces better
results for large regions because the sparse codes of the patches are adapted
according to the sparse codes of the boundary patches. Finally, we present and
analyze experimental results that demonstrate the performance of our method
compared to the literature
Learning to Reconstruct Shapes from Unseen Classes
From a single image, humans are able to perceive the full 3D shape of an
object by exploiting learned shape priors from everyday life. Contemporary
single-image 3D reconstruction algorithms aim to solve this task in a similar
fashion, but often end up with priors that are highly biased by training
classes. Here we present an algorithm, Generalizable Reconstruction (GenRe),
designed to capture more generic, class-agnostic shape priors. We achieve this
with an inference network and training procedure that combine 2.5D
representations of visible surfaces (depth and silhouette), spherical shape
representations of both visible and non-visible surfaces, and 3D voxel-based
representations, in a principled manner that exploits the causal structure of
how 3D shapes give rise to 2D images. Experiments demonstrate that GenRe
performs well on single-view shape reconstruction, and generalizes to diverse
novel objects from categories not seen during training.Comment: NeurIPS 2018 (Oral). The first two authors contributed equally to
this paper. Project page: http://genre.csail.mit.edu
- …