9,822 research outputs found
PanoContext-Former: Panoramic Total Scene Understanding with a Transformer
Panoramic image enables deeper understanding and more holistic perception of
surrounding environment, which can naturally encode enriched scene
context information compared to standard perspective image. Previous work has
made lots of effort to solve the scene understanding task in a bottom-up form,
thus each sub-task is processed separately and few correlations are explored in
this procedure. In this paper, we propose a novel method using depth prior for
holistic indoor scene understanding which recovers the objects' shapes,
oriented bounding boxes and the 3D room layout simultaneously from a single
panorama. In order to fully utilize the rich context information, we design a
transformer-based context module to predict the representation and relationship
among each component of the scene. In addition, we introduce a real-world
dataset for scene understanding, including photo-realistic panoramas,
high-fidelity depth images, accurately annotated room layouts, and oriented
object bounding boxes and shapes. Experiments on the synthetic and real-world
datasets demonstrate that our method outperforms previous panoramic scene
understanding methods in terms of both layout estimation and 3D object
detection
Physics Inspired Optimization on Semantic Transfer Features: An Alternative Method for Room Layout Estimation
In this paper, we propose an alternative method to estimate room layouts of
cluttered indoor scenes. This method enjoys the benefits of two novel
techniques. The first one is semantic transfer (ST), which is: (1) a
formulation to integrate the relationship between scene clutter and room layout
into convolutional neural networks; (2) an architecture that can be end-to-end
trained; (3) a practical strategy to initialize weights for very deep networks
under unbalanced training data distribution. ST allows us to extract highly
robust features under various circumstances, and in order to address the
computation redundance hidden in these features we develop a principled and
efficient inference scheme named physics inspired optimization (PIO). PIO's
basic idea is to formulate some phenomena observed in ST features into
mechanics concepts. Evaluations on public datasets LSUN and Hedau show that the
proposed method is more accurate than state-of-the-art methods.Comment: To appear in CVPR 2017. Project Page:
https://sites.google.com/view/st-pio
- …