58 research outputs found
Structured Indoor Modeling
In this dissertation, we propose data-driven approaches to reconstruct 3D models for indoor scenes which are represented in a structured way (e.g., a wall is represented by a planar surface and two rooms are connected via the wall). The structured representation of models is more application ready than dense representations (e.g., a point cloud), but poses additional challenges for reconstruction since extracting structures requires high-level understanding about geometries. To address this challenging problem, we explore two common structural regularities of indoor scenes: 1) most indoor structures consist of planar surfaces (planarity), and 2) structural surfaces (e.g., walls and floor) can be represented by a 2D floorplan as a top-down view projection (orthogonality). With breakthroughs in data capturing techniques, we develop automated systems to tackle structured modeling problems, namely piece-wise planar reconstruction and floorplan reconstruction, by learning shape priors (i.e., planarity and orthogonality) from data. With structured representations and production-level quality, the reconstructed models have an immediate impact on many industrial applications
Floor-SP: Inverse CAD for Floorplans by Sequential Room-wise Shortest Path
This paper proposes a new approach for automated floorplan reconstruction
from RGBD scans, a major milestone in indoor mapping research. The approach,
dubbed Floor-SP, formulates a novel optimization problem, where room-wise
coordinate descent sequentially solves dynamic programming to optimize the
floorplan graph structure. The objective function consists of data terms guided
by deep neural networks, consistency terms encouraging adjacent rooms to share
corners and walls, and the model complexity term. The approach does not require
corner/edge detection with thresholds, unlike most other methods. We have
evaluated our system on production-quality RGBD scans of 527 apartments or
houses, including many units with non-Manhattan structures. Qualitative and
quantitative evaluations demonstrate a significant performance boost over the
current state-of-the-art. Please refer to our project website
http://jcchen.me/floor-sp/ for code and data.Comment: 10 pages, 9 figures, accepted to ICCV 201
Connecting the Dots: Floorplan Reconstruction Using Two-Level Queries
We address 2D floorplan reconstruction from 3D scans. Existing approaches
typically employ heuristically designed multi-stage pipelines. Instead, we
formulate floorplan reconstruction as a single-stage structured prediction
task: find a variable-size set of polygons, which in turn are variable-length
sequences of ordered vertices. To solve it we develop a novel Transformer
architecture that generates polygons of multiple rooms in parallel, in a
holistic manner without hand-crafted intermediate stages. The model features
two-level queries for polygons and corners, and includes polygon matching to
make the network end-to-end trainable. Our method achieves a new
state-of-the-art for two challenging datasets, Structured3D and SceneCAD, along
with significantly faster inference than previous methods. Moreover, it can
readily be extended to predict additional information, i.e., semantic room
types and architectural elements like doors and windows. Our code and models
are available at: https://github.com/ywyue/RoomFormer.Comment: CVPR 2023 camera-ready. Project page:
https://ywyue.github.io/RoomForme
Floorplan generation from 3D point clouds: A space partitioning approach
International audienceWe propose a novel approach to automatically reconstruct the floorplan of indoor environments from raw sensor data. In contrast to existing methods that generate floorplans under the form of a planar graph by detecting corner points and connecting them, our framework employs a strategy that decomposes the space into a polygonal partition and selects edges that belong to wall structures by energy minimization. By relying on a efficient space-partitioning data structure instead of a traditional and delicate corner detection task, our framework offers a high robustness to imperfect data. We demonstrate the potential of our algorithm on both RGBD and Lidar points scanned from simple to complex scenes. Experimental results indicate that our method is competitive with respect to existing methods in terms of geometric accuracy and output simplicity
PolyDiffuse: Polygonal Shape Reconstruction via Guided Set Diffusion Models
This paper presents PolyDiffuse, a novel structured reconstruction algorithm
that transforms visual sensor data into polygonal shapes with Diffusion Models
(DM), an emerging machinery amid exploding generative AI, while formulating
reconstruction as a generation process conditioned on sensor data. The task of
structured reconstruction poses two fundamental challenges to DM: 1) A
structured geometry is a ``set'' (e.g., a set of polygons for a floorplan
geometry), where a sample of elements has different but equivalent
representations, making the denoising highly ambiguous; and 2) A
``reconstruction'' task has a single solution, where an initial noise needs to
be chosen carefully, while any initial noise works for a generation task. Our
technical contribution is the introduction of a Guided Set Diffusion Model
where 1) the forward diffusion process learns guidance networks to control
noise injection so that one representation of a sample remains distinct from
its other permutation variants, thus resolving denoising ambiguity; and 2) the
reverse denoising process reconstructs polygonal shapes, initialized and
directed by the guidance networks, as a conditional generation process subject
to the sensor data. We have evaluated our approach for reconstructing two types
of polygonal shapes: floorplan as a set of polygons and HD map for autonomous
cars as a set of polylines. Through extensive experiments on standard
benchmarks, we demonstrate that PolyDiffuse significantly advances the current
state of the art and enables broader practical applications.Comment: Project page: https://poly-diffuse.github.io
CornerFormer: Boosting Corner Representation for Fine-Grained Structured Reconstruction
Structured reconstruction is a non-trivial dense prediction problem, which
extracts structural information (\eg, building corners and edges) from a raster
image, then reconstructs it to a 2D planar graph accordingly. Compared with
common segmentation or detection problems, it significantly relays on the
capability that leveraging holistic geometric information for structural
reasoning. Current transformer-based approaches tackle this challenging problem
in a two-stage manner, which detect corners in the first model and classify the
proposed edges (corner-pairs) in the second model. However, they separate
two-stage into different models and only share the backbone encoder. Unlike the
existing modeling strategies, we present an enhanced corner representation
method: 1) It fuses knowledge between the corner detection and edge prediction
by sharing feature in different granularity; 2) Corner candidates are proposed
in four heatmap channels w.r.t its direction. Both qualitative and quantitative
evaluations demonstrate that our proposed method can better reconstruct
fine-grained structures, such as adjacent corners and tiny edges. Consequently,
it outperforms the state-of-the-art model by +1.9\%@F-1 on Corner and
+3.0\%@F-1 on Edge
Online Synthesis Of Speculative Building Information Models For Robot Motion Planning
Autonomous mobile robots today still lack the necessary understanding of indoor environments for making informed decisions about the state of the world beyond their immediate field of view. As a result, they are forced to make conservative and often inaccurate assumptions about unexplored space, inhibiting the degree of performance being increasingly expected of them in the areas of high-speed navigation and mission planning. In order to address this limitation, this thesis explores the use of Building Information Models (BIMs) for providing the existing ecosystem of local and global planning algorithms with informative compact higher-level representations of indoor environments. Although BIMs have long been used in architecture, engineering, and construction for a number of different purposes, to our knowledge, this is the first instance of them being used in robotics. Given the technical constraints accompanying this domain, including a limited and incomplete set of observations which grows over time, the systems we present are designed such that together they produce BIMs capable of providing explanations of both the explored and unexplored space in an online fashion. The first is a SLAM system that uses the structural regularity of buildings in order to mitigate drift and provide the simplest explanation of architectural features such as floors, walls, and ceilings. The planar model generated is then passed to a secondary system that then reasons about their mutual relationships in order to provide a water-tight model of the observed and inferred freespace. Our experimental results demonstrate this to be an accurate and efficient approach towards this end
- …