24 research outputs found
ROCA: Robust CAD Model Retrieval and Alignment from a Single Image
We present ROCA, a novel end-to-end approach that retrieves and aligns 3D CAD
models from a shape database to a single input image. This enables 3D
perception of an observed scene from a 2D RGB observation, characterized as a
lightweight, compact, clean CAD representation. Core to our approach is our
differentiable alignment optimization based on dense 2D-3D object
correspondences and Procrustes alignment. ROCA can thus provide a robust CAD
alignment while simultaneously informing CAD retrieval by leveraging the 2D-3D
correspondences to learn geometrically similar CAD models. Experiments on
challenging, real-world imagery from ScanNet show that ROCA significantly
improves on state of the art, from 9.5% to 17.6% in retrieval-aware CAD
alignment accuracy
Shape Anchor Guided Holistic Indoor Scene Understanding
This paper proposes a shape anchor guided learning strategy (AncLearn) for
robust holistic indoor scene understanding. We observe that the search space
constructed by current methods for proposal feature grouping and instance point
sampling often introduces massive noise to instance detection and mesh
reconstruction. Accordingly, we develop AncLearn to generate anchors that
dynamically fit instance surfaces to (i) unmix noise and target-related
features for offering reliable proposals at the detection stage, and (ii)
reduce outliers in object point sampling for directly providing well-structured
geometry priors without segmentation during reconstruction. We embed AncLearn
into a reconstruction-from-detection learning system (AncRec) to generate
high-quality semantic scene models in a purely instance-oriented manner.
Experiments conducted on the challenging ScanNetv2 dataset demonstrate that our
shape anchor-based method consistently achieves state-of-the-art performance in
terms of 3D object detection, layout estimation, and shape reconstruction. The
code will be available at https://github.com/Geo-Tell/AncRec