Towards automatic reconstruction of indoor scenes from incomplete point clouds: door and window detection and regularization

In the last years, point clouds have become the main source of information for building modelling. Although a considerable amount of methodologies addressing the automated generation of 3D models from point clouds have been developed, indoor modelling is still a challenging task due to complex building layouts and the high presence of severe clutters and occlusions. Most of methodologies are highly dependent on data quality, often producing irregular and non-consistent models. Although manmade environments generally exhibit some regularities, they are not commonly considered. This paper presents an optimization-based approach for detecting regularities (i.e., same shape, same alignment and same spacing) in building indoor features. The methodology starts from the detection of openings based on a voxel-based visibility analysis to distinguish ‘occluded’ from ‘empty’ regions in wall surfaces. The extraction of regular patterns in windows is addressed from studying the point cloud from an outdoor perspective. The layout is regularized by minimizing deformations while respecting the detected constraints. The methodology applies for elements placed in the same planeXunta de Galicia | Ref. ED481B 2016/079-

Tunneling Appropriate Computational Models from Laser Scanning Data

Tunneling projects often require computational models of existing structures. To this end, this paper demonstrates the viability of automatically, robustly reconstructing an individual building model from laser scanning data for further computational modeling without any manual intervention. The resulting model is appropriate for immediate importation into a commercial finite element method (FEM) program. The method combines a voxel-based technique with an angle criterion. Initially, the voxelization model is used to represent the façade model, while an angle criterion is implemented to determine boundaries of the façade and its openings (doors and windows). The algorithm overcomes common problems of occlusions or artefacts that arise during data acquisition. The resulting relative errors of overall dimensions and opening areas of geometric models were less 2% and 6%, respectively, which are generally within industry standards for this type of building modeling.Science Foundation Ireland (SFI/PICA/I850); European Union Grant ERC StG 2012-307836- RETURN

TOWARDS AUTOMATIC RECONSTRUCTION OF INDOOR SCENES FROM INCOMPLETE POINT CLOUDS: DOOR AND WINDOW DETECTION AND REGULARIZATION

In the last years, point clouds have become the main source of information for building modelling. Although a considerable amount of methodologies addressing the automated generation of 3D models from point clouds have been developed, indoor modelling is still a challenging task due to complex building layouts and the high presence of severe clutters and occlusions. Most of methodologies are highly dependent on data quality, often producing irregular and non-consistent models. Although manmade environments generally exhibit some regularities, they are not commonly considered. This paper presents an optimization-based approach for detecting regularities (i.e., same shape, same alignment and same spacing) in building indoor features. The methodology starts from the detection of openings based on a voxel-based visibility analysis to distinguish ‘occluded’ from ‘empty’ regions in wall surfaces. The extraction of regular patterns in windows is addressed from studying the point cloud from an outdoor perspective. The layout is regularized by minimizing deformations while respecting the detected constraints. The methodology applies for elements placed in the same plane

Low-rank Based Algorithms for Rectification, Repetition Detection and De-noising in Urban Images

In this thesis, we aim to solve the problem of automatic image rectification and repeated patterns detection on 2D urban images, using novel low-rank based techniques. Repeated patterns (such as windows, tiles, balconies and doors) are prominent and significant features in urban scenes. Detection of the periodic structures is useful in many applications such as photorealistic 3D reconstruction, 2D-to-3D alignment, facade parsing, city modeling, classification, navigation, visualization in 3D map environments, shape completion, cinematography and 3D games. However both of the image rectification and repeated patterns detection problems are challenging due to scene occlusions, varying illumination, pose variation and sensor noise. Therefore, detection of these repeated patterns becomes very important for city scene analysis. Given a 2D image of urban scene, we automatically rectify a facade image and extract facade textures first. Based on the rectified facade texture, we exploit novel algorithms that extract repeated patterns by using Kronecker product based modeling that is based on a solid theoretical foundation. We have tested our algorithms in a large set of images, which includes building facades from Paris, Hong Kong and New York

Detailed Three-Dimensional Building Façade Reconstruction: A Review on Applications, Data and Technologies

Urban environments are regions of complex and diverse architecture. Their reconstruction and representation as three-dimensional city models have attracted the attention of many researchers and industry specialists, as they increasingly recognise the potential for new applications requiring detailed building models. Nevertheless, despite being investigated for a few decades, the comprehensive reconstruction of buildings remains a challenging task. While there is a considerable body of literature on this topic, including several systematic reviews summarising ways of acquiring and reconstructing coarse building structures, there is a paucity of in-depth research on the detection and reconstruction of façade openings (i.e., windows and doors). In this review, we provide an overview of emerging applications, data acquisition and processing techniques for building façade reconstruction, emphasising building opening detection. The use of traditional technologies from terrestrial and aerial platforms, along with emerging approaches, such as mobile phones and volunteered geography information, is discussed. The current status of approaches for opening detection is then examined in detail, separated into methods for three-dimensional and two-dimensional data. Based on the review, it is clear that a key limitation associated with façade reconstruction is process automation and the need for user intervention. Another limitation is the incompleteness of the data due to occlusion, which can be reduced by data fusion. In addition, the lack of available diverse benchmark datasets and further investigation into deep-learning methods for façade openings extraction present crucial opportunities for future research

Automated Extraction of 3D Building Windows from Mobile LiDAR Data

The three-dimensional (3D) city models have gained more and more attentions because of their considerable potential applications at present. In particular, the demands for Level of Detail (LoD) building models become urgent. Mobile Laser Scanning (MLS) has supplied a brand-new technology in the acquisition and update of 3D information in urban off-terrain features, particularly for building façade details. Accordingly, generating LoD3 building models from MLS point clouds becomes a new trend in recent studies. As a consequence, a method that can accurately and automatically extract 3D windows from raw MLS point clouds is presented in this thesis. To provide solid and credible information for LoD3 building models, this automated method endeavors to identify window frames on building facades from MLS point clouds. This algorithm can typically be regarded as a stepwise procedure to interpret MLS point clouds as semantic features. A voxel-based upward-growing method is firstly applied to distinguish non-ground points from ground points. Noise is then filtered out from non-ground points by statistical analysis. In order to segment out the building facades, all the remaining non-ground points are clustered based on conditional Euclidean clustering algorithm; clusters whose density and width are over a given threshold will be designated as points for building facades. After a building façade is successfully extracted, a volumetric box is created to contain façade points so that neighbours of each point can be operated. A manipulator is finally applied according to the structural characteristics of window frames to extract the potential window points. The experimental results demonstrate that the proposed algorithm can successfully extract the rectangular or curved windows in the test datasets with promising accuracies. The 2D validation and 3D validation were both conducted in this study. In the 2D validation, the lowest F-measure of the test datasets is 0.740, and the highest can be 0.977. While in the 3D validation, the lowest correctness of the test dataset is 79.58%, and the highest can be 97.96%. After further analysis of the experimental results, it was found that, for those windows concave on walls or with curtains drawn, the performance of the proposed method was influenced. Furthermore, big holes caused by system errors in raw point clouds also had negative impacts on the proposed method. In conclusion, this thesis makes a considerable contribution to extracting 3D rectangular, irregular and arc-rounded windows from noisy MLS point clouds with high accuracy and high efficiency. It has supplied a promising method for generating LoD3 building models

Statistical part-based models for object detection in large 3D scans

3D scanning technology has matured to a point where very large scale acquisition of high resolution geometry has become feasible. However, having large quantities of 3D data poses new technical challenges. Many applications of practical use require an understanding of semantics of the acquired geometry. Consequently scene understanding plays a key role for many applications. This thesis is concerned with two core topics: 3D object detection and semantic alignment. We address the problem of efficiently detecting large quantities of objects in 3D scans according to object categories learned from sparse user annotation. Objects are modeled by a collection of smaller sub-parts and a graph structure representing part dependencies. The thesis introduces two novel approaches: A part-based chain structured Markov model and a general part-based full correlation model. Both models come with efficient detection schemes which allow for interactive run-times.Die Technologie für 3-dimensionale bildgebende Verfahren (3D Scans) ist mittlerweile an einem Punkt angelangt, an dem hochaufglöste Geometrie-Modelle für sehr große Szenen erstellbar sind. Große Mengen dreidimensionaler Daten stellen allerdings neue technische Herausforderungen. Viele Anwendungen von praktischem Nutzen erfordern ein semantisches Verständnis der akquirierten Geometrie. Dementsprechend spielt das sogenannte “Szenenverstehen” eine Schlüsselrolle bei vielen Anwendungen. Diese Dissertation beschäftigt sich mit 2 Kernthemen: 3D Objekt-Detektion und semantische (Objekt-) Anordnung. Das Problem hierbei ist, große Mengen von Objekten effizient in 3D Scans zu detektieren, wobei die Objekte aus bestimmten Objektkategorien entstammen, welche mittels gerinfügiger Annotationen durch den Benutzer gelernt werden. Dabei werden Objekte modelliert durch eine Ansammlung kleinerer Teilstücke und einer Graph-Struktur, welche die Abhängigkeiten der Einzelteile repäsentiert. Diese Arbeit stellt zwei neuartige Ansätze vor: Ein Markov-Modell, das aus einer teilebasierten Kettenstruktur besteht und einen generellen Ansatz, der auf einem Modell mit voll korrelierten Einzelteilen beruht. Zu beiden Modellen werden effiziente Detektionsschemata aufgezeigt, die interaktive Laufzeiten ermöglichen