402 research outputs found
Structured Indoor Modeling
In this dissertation, we propose data-driven approaches to reconstruct 3D models for indoor scenes which are represented in a structured way (e.g., a wall is represented by a planar surface and two rooms are connected via the wall). The structured representation of models is more application ready than dense representations (e.g., a point cloud), but poses additional challenges for reconstruction since extracting structures requires high-level understanding about geometries. To address this challenging problem, we explore two common structural regularities of indoor scenes: 1) most indoor structures consist of planar surfaces (planarity), and 2) structural surfaces (e.g., walls and floor) can be represented by a 2D floorplan as a top-down view projection (orthogonality). With breakthroughs in data capturing techniques, we develop automated systems to tackle structured modeling problems, namely piece-wise planar reconstruction and floorplan reconstruction, by learning shape priors (i.e., planarity and orthogonality) from data. With structured representations and production-level quality, the reconstructed models have an immediate impact on many industrial applications
Floor-SP: Inverse CAD for Floorplans by Sequential Room-wise Shortest Path
This paper proposes a new approach for automated floorplan reconstruction
from RGBD scans, a major milestone in indoor mapping research. The approach,
dubbed Floor-SP, formulates a novel optimization problem, where room-wise
coordinate descent sequentially solves dynamic programming to optimize the
floorplan graph structure. The objective function consists of data terms guided
by deep neural networks, consistency terms encouraging adjacent rooms to share
corners and walls, and the model complexity term. The approach does not require
corner/edge detection with thresholds, unlike most other methods. We have
evaluated our system on production-quality RGBD scans of 527 apartments or
houses, including many units with non-Manhattan structures. Qualitative and
quantitative evaluations demonstrate a significant performance boost over the
current state-of-the-art. Please refer to our project website
http://jcchen.me/floor-sp/ for code and data.Comment: 10 pages, 9 figures, accepted to ICCV 201
SeDAR: Reading Floorplans Like a Human—Using Deep Learning to Enable Human-Inspired Localisation
This is the final version. Available from Springer Verlag via the DOI in this record. The use of human-level semantic information to aid robotic tasks has recently become an important area for both Computer
Vision and Robotics. This has been enabled by advances in Deep Learning that allow consistent and robust semantic understanding. Leveraging this semantic vision of the world has allowed human-level understanding to naturally emerge from many
different approaches. Particularly, the use of semantic information to aid in localisation and reconstruction has been at the
forefront of both fields. Like robots, humans also require the ability to localise within a structure. To aid this, humans have
designed high-level semantic maps of our structures called floorplans. We are extremely good at localising in them, even with
limited access to the depth information used by robots. This is because we focus on the distribution of semantic elements,
rather than geometric ones. Evidence of this is that humans are normally able to localise in a floorplan that has not been
scaled properly. In order to grant this ability to robots, it is necessary to use localisation approaches that leverage the same
semantic information humans use. In this paper, we present a novel method for semantically enabled global localisation. Our
approach relies on the semantic labels present in the floorplan. Deep Learning is leveraged to extract semantic labels from
RGB images, which are compared to the floorplan for localisation. While our approach is able to use range measurements if
available, we demonstrate that they are unnecessary as we can achieve results comparable to state-of-the-art without them.EPSRCInnovate UKNVIDIA Corporatio
Floorplan-Aware Camera Poses Refinement
Processing large indoor scenes is a challenging task, as scan registration
and camera trajectory estimation methods accumulate errors across time. As a
result, the quality of reconstructed scans is insufficient for some
applications, such as visual-based localization and navigation, where the
correct position of walls is crucial.
For many indoor scenes, there exists an image of a technical floorplan that
contains information about the geometry and main structural elements of the
scene, such as walls, partitions, and doors. We argue that such a floorplan is
a useful source of spatial information, which can guide a 3D model
optimization.
The standard RGB-D 3D reconstruction pipeline consists of a tracking module
applied to an RGB-D sequence and a bundle adjustment (BA) module that takes the
posed RGB-D sequence and corrects the camera poses to improve consistency. We
propose a novel optimization algorithm expanding conventional BA that leverages
the prior knowledge about the scene structure in the form of a floorplan. Our
experiments on the Redwood dataset and our self-captured data demonstrate that
utilizing floorplan improves accuracy of 3D reconstructions.Comment: IROS 202
PolyDiffuse: Polygonal Shape Reconstruction via Guided Set Diffusion Models
This paper presents PolyDiffuse, a novel structured reconstruction algorithm
that transforms visual sensor data into polygonal shapes with Diffusion Models
(DM), an emerging machinery amid exploding generative AI, while formulating
reconstruction as a generation process conditioned on sensor data. The task of
structured reconstruction poses two fundamental challenges to DM: 1) A
structured geometry is a ``set'' (e.g., a set of polygons for a floorplan
geometry), where a sample of elements has different but equivalent
representations, making the denoising highly ambiguous; and 2) A
``reconstruction'' task has a single solution, where an initial noise needs to
be chosen carefully, while any initial noise works for a generation task. Our
technical contribution is the introduction of a Guided Set Diffusion Model
where 1) the forward diffusion process learns guidance networks to control
noise injection so that one representation of a sample remains distinct from
its other permutation variants, thus resolving denoising ambiguity; and 2) the
reverse denoising process reconstructs polygonal shapes, initialized and
directed by the guidance networks, as a conditional generation process subject
to the sensor data. We have evaluated our approach for reconstructing two types
of polygonal shapes: floorplan as a set of polygons and HD map for autonomous
cars as a set of polylines. Through extensive experiments on standard
benchmarks, we demonstrate that PolyDiffuse significantly advances the current
state of the art and enables broader practical applications.Comment: Project page: https://poly-diffuse.github.io
Automatic 3D building model generation using deep learning methods based on cityjson and 2D floor plans
In the past decade, a lot of effort is put into applying digital innovations to building life cycles. 3D Models have been proven to be efficient for decision making, scenario simulation and 3D data analysis during this life cycle. Creating such digital representation of a building can be a labour-intensive task, depending on the desired scale and level of detail (LOD). This research aims at creating a new automatic deep learning based method for building model reconstruction. It combines exterior and interior data sources: 1) 3D BAG, 2) archived floor plan images. To reconstruct 3D building models from the two data sources, an innovative combination of methods is proposed. In order to obtain the information needed from the floor plan images (walls, openings and labels), deep learning techniques have been used. In addition, post-processing techniques are introduced to transform the data in the required format. In order to fuse the extracted 2D data and the 3D exterior, a data fusion process is introduced. From the literature review, no prior research on automatic integration of CityGML/JSON and floor plan images has been found. Therefore, this method is a first approach to this data integration
- …