6 research outputs found
Category-Specific Object Reconstruction from a Single Image
Object reconstruction from a single image -- in the wild -- is a problem
where we can make progress and get meaningful results today. This is the main
message of this paper, which introduces an automated pipeline with pixels as
inputs and 3D surfaces of various rigid categories as outputs in images of
realistic scenes. At the core of our approach are deformable 3D models that can
be learned from 2D annotations available in existing object detection datasets,
that can be driven by noisy automatic object segmentations and which we
complement with a bottom-up module for recovering high-frequency shape details.
We perform a comprehensive quantitative analysis and ablation study of our
approach using the recently introduced PASCAL 3D+ dataset and show very
encouraging automatic reconstructions on PASCAL VOC.Comment: First two authors contributed equally. To appear at CVPR 201
Learning Direct Optimization for scene understanding
We develop a Learning Direct Optimization (LiDO) method for the refinement of
a latent variable model that describes input image x. Our goal is to explain a
single image x with an interpretable 3D computer graphics model having scene
graph latent variables z (such as object appearance, camera position). Given a
current estimate of z we can render a prediction of the image g(z), which can
be compared to the image x. The standard way to proceed is then to measure the
error E(x, g(z)) between the two, and use an optimizer to minimize the error.
However, it is unknown which error measure E would be most effective for
simultaneously addressing issues such as misaligned objects, occlusions,
textures, etc. In contrast, the LiDO approach trains a Prediction Network to
predict an update directly to correct z, rather than minimizing the error with
respect to z. Experiments show that our LiDO method converges rapidly as it
does not need to perform a search on the error landscape, produces better
solutions than error-based competitors, and is able to handle the mismatch
between the data and the fitted scene model. We apply LiDO to a realistic
synthetic dataset, and show that the method also transfers to work well with
real images