47 research outputs found
Learning to Construct 3D Building Wireframes from 3D Line Clouds
Line clouds, though under-investigated in the previous work, potentially
encode more compact structural information of buildings than point clouds
extracted from multi-view images. In this work, we propose the first network to
process line clouds for building wireframe abstraction. The network takes a
line cloud as input , i.e., a nonstructural and unordered set of 3D line
segments extracted from multi-view images, and outputs a 3D wireframe of the
underlying building, which consists of a sparse set of 3D junctions connected
by line segments. We observe that a line patch, i.e., a group of neighboring
line segments, encodes sufficient contour information to predict the existence
and even the 3D position of a potential junction, as well as the likelihood of
connectivity between two query junctions. We therefore introduce a two-layer
Line-Patch Transformer to extract junctions and connectivities from sampled
line patches to form a 3D building wireframe model. We also introduce a
synthetic dataset of multi-view images with ground-truth 3D wireframe. We
extensively justify that our reconstructed 3D wireframe models significantly
improve upon multiple baseline building reconstruction methods. The code and
data can be found at https://github.com/Luo1Cheng/LC2WF.Comment: 10 pages, 6 figure
3D Room Layout Estimation from a Cubemap of Panorama Image via Deep Manhattan Hough Transform
Significant geometric structures can be compactly described by global
wireframes in the estimation of 3D room layout from a single panoramic image.
Based on this observation, we present an alternative approach to estimate the
walls in 3D space by modeling long-range geometric patterns in a learnable
Hough Transform block. We transform the image feature from a cubemap tile to
the Hough space of a Manhattan world and directly map the feature to the
geometric output. The convolutional layers not only learn the local
gradient-like line features, but also utilize the global information to
successfully predict occluded walls with a simple network structure. Unlike
most previous work, the predictions are performed individually on each cubemap
tile, and then assembled to get the layout estimation. Experimental results
show that we achieve comparable results with recent state-of-the-art in
prediction accuracy and performance. Code is available at
https://github.com/Starrah/DMH-Net.Comment: Accepted by ECCV 202
Neural Wireframe Renderer: Learning Wireframe to Image Translations
In architecture and computer-aided design, wireframes (i.e., line-based
models) are widely used as basic 3D models for design evaluation and fast
design iterations. However, unlike a full design file, a wireframe model lacks
critical information, such as detailed shape, texture, and materials, needed by
a conventional renderer to produce 2D renderings of the objects or scenes. In
this paper, we bridge the information gap by generating photo-realistic
rendering of indoor scenes from wireframe models in an image translation
framework. While existing image synthesis methods can generate visually
pleasing images for common objects such as faces and birds, these methods do
not explicitly model and preserve essential structural constraints in a
wireframe model, such as junctions, parallel lines, and planar surfaces. To
this end, we propose a novel model based on a structure-appearance joint
representation learned from both images and wireframes. In our model,
structural constraints are explicitly enforced by learning a joint
representation in a shared encoder network that must support the generation of
both images and wireframes. Experiments on a wireframe-scene dataset show that
our wireframe-to-image translation model significantly outperforms the
state-of-the-art methods in both visual quality and structural integrity of
generated images.Comment: ECCV 202
Floor-SP: Inverse CAD for Floorplans by Sequential Room-wise Shortest Path
This paper proposes a new approach for automated floorplan reconstruction
from RGBD scans, a major milestone in indoor mapping research. The approach,
dubbed Floor-SP, formulates a novel optimization problem, where room-wise
coordinate descent sequentially solves dynamic programming to optimize the
floorplan graph structure. The objective function consists of data terms guided
by deep neural networks, consistency terms encouraging adjacent rooms to share
corners and walls, and the model complexity term. The approach does not require
corner/edge detection with thresholds, unlike most other methods. We have
evaluated our system on production-quality RGBD scans of 527 apartments or
houses, including many units with non-Manhattan structures. Qualitative and
quantitative evaluations demonstrate a significant performance boost over the
current state-of-the-art. Please refer to our project website
http://jcchen.me/floor-sp/ for code and data.Comment: 10 pages, 9 figures, accepted to ICCV 201
DeepLSD: Line Segment Detection and Refinement with Deep Image Gradients
Line segments are ubiquitous in our human-made world and are increasingly
used in vision tasks. They are complementary to feature points thanks to their
spatial extent and the structural information they provide. Traditional line
detectors based on the image gradient are extremely fast and accurate, but lack
robustness in noisy images and challenging conditions. Their learned
counterparts are more repeatable and can handle challenging images, but at the
cost of a lower accuracy and a bias towards wireframe lines. We propose to
combine traditional and learned approaches to get the best of both worlds: an
accurate and robust line detector that can be trained in the wild without
ground truth lines. Our new line segment detector, DeepLSD, processes images
with a deep network to generate a line attraction field, before converting it
to a surrogate image gradient magnitude and angle, which is then fed to any
existing handcrafted line detector. Additionally, we propose a new optimization
tool to refine line segments based on the attraction field and vanishing
points. This refinement improves the accuracy of current deep detectors by a
large margin. We demonstrate the performance of our method on low-level line
detection metrics, as well as on several downstream tasks using multiple
challenging datasets. The source code and models are available at
https://github.com/cvg/DeepLSD.Comment: Accepted at CVPR 202