A Featureless Approach to 3D Polyhedral Building Modeling from Aerial Images

This paper presents a model-based approach for reconstructing 3D polyhedral building models from aerial images. The proposed approach exploits some geometric and photometric properties resulting from the perspective projection of planar structures. Data are provided by calibrated aerial images. The novelty of the approach lies in its featurelessness and in its use of direct optimization based on image rawbrightness. The proposed framework avoids feature extraction and matching. The 3D polyhedral model is directly estimated by optimizing an objective function that combines an image-based dissimilarity measure and a gradient score over several aerial images. The optimization process is carried out by the Differential Evolution algorithm. The proposed approach is intended to provide more accurate 3D reconstruction than feature-based approaches. Fast 3D model rectification and updating can take advantage of the proposed method. Several results and evaluations of performance from real and synthetic images show the feasibility and robustness of the proposed approach

Roof Segmentation Towards Digital Twin Generation in LoD2+Using Deep Learning

There is an increasing need for digital twins of cities and their base maps, 3D city models. Creating and updating these twins is not an easy task, so automating and streamlining the process is a field of active research. A significant part of the urban geometry is residential buildings and their roofs. Modeling of roofs for urban buildings can be divided into three main areas - building detection, roof recognition and building reconstruction. The building and roofs are segmented with the help of machine learning and image processing. Afterwards the extracted information is used to generate parametric models for the roofs using methods from computational geometry. The goal is to create correct virtual models of roofs belonging to many different types of buildings. In this study, a supervised deep learning approach is proposed for the segmentation of roof edges from a single orthophoto. The predicted features include the linear elements of roofs. The experiments show that, despite the small amount of training data, even in the presence of noise, the proposed method performs well on semantic segmentation of roofs with different shapes and complexities. The quality of the extracted roof elements for the test area is about 56% and 71% for mean intersection over union (IOU) and Dice metric scores, respectively. Copyright (C) 2022 The Authors

A Pipeline of 3D Scene Reconstruction from Point Clouds

3D technologies are becoming increasingly popular as their applications in industrial, consumer, entertainment, healthcare, education, and governmental increase in number. According to market predictions, the total 3D modeling and mapping market is expected to grow from $1.1 billion in 2013 to$7.7 billion by 2018. Thus, 3D modeling techniques for different data sources are urgently needed. This thesis addresses techniques for automated point cloud classification and the reconstruction of 3D scenes (including terrain models, 3D buildings and 3D road networks). First, georeferenced binary image processing techniques were developed for various point cloud classifications. Second, robust methods for the pipeline from the original point cloud to 3D model construction were proposed. Third, the reconstruction for the levels of detail (LoDs) of 1-3 (CityGML website) of 3D models was demonstrated. Fourth, different data sources for 3D model reconstruction were studied. The strengths and weaknesses of using the different data sources were addressed. Mobile laser scanning (MLS), unmanned aerial vehicle (UAV) images, airborne laser scanning (ALS), and the Finnish National Land Survey’s open geospatial data sources e.g. a topographic database, were employed as test data. Among these data sources, MLS data from three different systems were explored, and three different densities of ALS point clouds (0.8, 8 and 50 points/m2) were studied. The results were compared with reference data such as an orthophoto with a ground sample distance of 20cm or measured reference points from existing software to evaluate their quality. The results showed that 74.6% of building roofs were reconstructed with the automated process. The resulting building models provided an average height deviation of 15 cm. A total of 6% of model points had a greater than one-pixel deviation from laser points. A total of 2.5% had a deviation of greater than two pixels. The pixel size was determined by the average distance of input laser points. The 3D roads were reconstructed with an average width deviation of 22 cm and an average height deviation of 14 cm. The results demonstrated that 93.4% of building roofs were correctly classified from sparse ALS and that 93.3% of power line points are detected from the six sets of dense ALS data located in forested areas. This study demonstrates the operability of 3D model construction for LoDs of 1-3 via the proposed methodologies and datasets. The study is beneficial to future applications, such as 3D-model-based navigation applications, the updating of 2D topographic databases into 3D maps and rapid, large-area 3D scene reconstruction. 3D-teknologiat ovat tulleet yhä suositummiksi niiden sovellusalojen lisääntyessä teollisuudessa, kuluttajatuotteissa, terveydenhuollossa, koulutuksessa ja hallinnossa. Ennusteiden mukaan 3D-mallinnus- ja -kartoitusmarkkinat kasvavat vuoden 2013 1,1 miljardista dollarista 7,7 miljardiin vuoteen 2018 mennessä. Erilaisia aineistoja käyttäviä 3D-mallinnustekniikoita tarvitaankin yhä enemmän. Tässä väitöskirjatutkimuksessa kehitettiin automaattisen pistepilviaineiston luokittelutekniikoita ja rekonstruoitiin 3D-ympäristöja (maanpintamalleja, rakennuksia ja tieverkkoja). Georeferoitujen binääristen kuvien prosessointitekniikoita kehitettiin useiden pilvipisteaineistojen luokitteluun. Työssä esitetään robusteja menetelmiä alkuperäisestä pistepilvestä 3D-malliin eri CityGML-standardin tarkkuustasoilla. Myös eri aineistolähteitä 3D-mallien rekonstruointiin tutkittiin. Eri aineistolähteiden käytön heikkoudet ja vahvuudet analysoitiin. Testiaineistona käytettiin liikkuvalla keilauksella (mobile laser scanning, MLS) ja ilmakeilauksella (airborne laser scanning, ALS) saatua laserkeilausaineistoja, miehittämättömillä lennokeilla (unmanned aerial vehicle, UAV) otettuja kuvia sekä Maanmittauslaitoksen avoimia aineistoja, kuten maastotietokantaa. Liikkuvalla laserkeilauksella kerätyn aineiston osalta tutkimuksessa käytettiin kolmella eri järjestelmällä saatua dataa, ja kolmen eri tarkkuustason (0,8, 8 ja 50 pistettä/m2) ilmalaserkeilausaineistoa. Tutkimuksessa saatuja tulosten laatua arvioitiin vertaamalla niitä referenssiaineistoon, jona käytettiin ortokuvia (GSD 20cm) ja nykyisissä ohjelmistoissa olevia mitattuja referenssipisteitä. 74,6 % rakennusten katoista saatiin rekonstruoitua automaattisella prosessilla. Rakennusmallien korkeuksien keskipoikkeama oli 15 cm. 6 %:lla mallin pisteistä oli yli yhden pikselin poikkeama laseraineiston pisteisiin verrattuna. 2,5 %:lla oli yli kahden pikselin poikkeama. Pikselikoko määriteltiin kahden laserpisteen välimatkan keskiarvona. Rekonstruoitujen teiden leveyden keskipoikkeama oli 22 cm ja korkeuden keskipoikkeama oli 14 cm. Tulokset osoittavat että 93,4 % rakennuksista saatiin luokiteltua oikein harvasta ilmalaserkeilausaineistosta ja 93,3 % sähköjohdoista saatiin havaittua kuudesta tiheästä metsäalueen ilmalaserkeilausaineistosta. Tutkimus demonstroi 3D-mallin konstruktion toimivuutta tarkkuustasoilla (LoD) 1-3 esitetyillä menetelmillä ja aineistoilla. Tulokset ovat hyödyllisiä kehitettäessä tulevaisuuden sovelluksia, kuten 3D-malleihin perustuvia navigointisovelluksia, topografisten 2D-karttojen ajantasaistamista 3D-kartoiksi, ja nopeaa suurten alueiden 3D-ympäristöjen rekonstruktiota

Automatic 3-D Building Model Reconstruction from Very High Resolution Stereo Satellite Imagery

Recent advances in the availability of very high-resolution (VHR) satellite data together with efficient data acquisition and large area coverage have led to an upward trend in their applications for automatic 3-D building model reconstruction which require large-scale and frequent updates, such as disaster monitoring and urban management. Digital Surface Models (DSMs) generated from stereo satellite imagery suffer from mismatches, missing values, or blunders, resulting in rough building shape representations. To handle 3-D building model reconstruction using such low-quality DSMs, we propose a novel automatic multistage hybrid method using DSMs together with orthorectified panchromatic (PAN) and pansharpened data (PS) of multispectral (MS) satellite imagery. The algorithm consists of multiple steps including building boundary extraction and decomposition, image-based roof type classification, and initial roof parameter computation which are prior knowledge for the 3-D model fitting step. To fit 3-D models to the normalized DSM (nDSM) and to select the best one, a parameter optimization method based on exhaustive search is used sequentially in 2-D and 3-D. Finally, the neighboring building models in a building block are intersected to reconstruct the 3-D model of connecting roofs. All corresponding experiments are conducted on a dataset including four different areas of Munich city containing 208 buildings with different degrees of complexity. The results are evaluated both qualitatively and quantitatively. According to the results, the proposed approach can reliably reconstruct 3-D building models, even the complex ones with several inner yards and multiple orientations. Furthermore, the proposed approach provides a high level of automation by limiting the number of primitive roof types and by performing automatic parameter initialization

AUTOMATIC RECONSTRUCTION OF ROOF MODELS FROM BUILDING OUTLINES AND AERIAL IMAGE DATA

The knowledge of roof shapes is essential for the creation of 3D building models. Many experts and researchers use 3D building models for specialized tasks, such as creating noise maps, estimating the solar potential of roof structures, and planning new wireless infrastructures. Our aim is to introduce a technique for automating the creation of topologically correct roof building models using outlines and aerial image data. In this study, we used building footprints and vertical aerial survey photographs. Aerial survey photographs enabled us to produce an orthophoto and a digital surface model of the analysed area. The developed technique made it possible to detect roof edges from the orthophoto and to categorize the edges using spatial relationships and height information derived from the digital surface model. This method allows buildings with complicated shapes to be decomposed into simple parts that can be processed separately. In our study, a roof type and model were determined for each building part and tested with multiple datasets with different levels of quality. Excellent results were achieved for simple and medium complex roofs. Results for very complex roofs were unsatisfactory. For such structures, we propose using multitemporal images because these can lead to significant improvements and a better roof edge detection. The method used in this study was shared with the Czech national mapping agency and could be used for the creation of new 3D modelling products in the near future

MODELING OF ROOFS FROM POINT CLOUDS USING GENETIC ALGORITHMS

Building roof extraction has been studied for more than thirty years and it generates models that provide important information for many applications, especially urban planning. The present work aimed to model roofs only from point clouds using genetic algorithms (GAs) to develop a more automatized and efficient method. For this, firstly, an algorithm for edge detection was developed. Experiments were performed with simulated and real point clouds, obtained by LIDAR. In the experiments with simulated point clouds, three types of point clouds with different complexities were created, and the effects of noise and scan line spacing on the results were evaluated. For the experiments with real point clouds, five roofs were chosen as examples, each with a different characteristic. GAs were used to select, among the points identified during edge detection, the so-called ‘significant points’, those which are essential to the accurate reconstruction of the roof model. These points were then used to generate the models, which were assessed qualitatively and quantitatively. Such evaluations showed that the use of GAs proved to be efficient for the modeling of roofs, as the model geometry was satisfactory, the error was within an acceptable range, and the computational effort was clearly reduced

Assessment of Relative Accuracy of AHN-2 Laser Scanning Data Using Planar Features

AHN-2 is the second part of the Actueel Hoogtebestand Nederland project, which concerns the acquisition of high-resolution altimetry data over the entire Netherlands using airborne laser scanning. The accuracy assessment of laser altimetry data usually relies on comparing corresponding tie elements, often points or lines, in the overlapping strips. This paper proposes a new approach to strip adjustment and accuracy assessment of AHN-2 data by using planar features. In the proposed approach a transformation is estimated between two overlapping strips by minimizing the distances between points in one strip and their corresponding planes in the other. The planes and the corresponding points are extracted in an automated segmentation process. The point-to-plane distances are used as observables in an estimation model, whereby the parameters of a transformation between the two strips and their associated quality measures are estimated. We demonstrate the performance of the method for the accuracy assessment of the AHN-2 dataset over Zeeland province of The Netherlands. The results show vertical offsets of up to 4 cm between the overlapping strips, and horizontal offsets ranging from 2 cm to 34 cm

Training a Fully Convolutional Neural Network with Imbalanced, Imperfect and Incomplete Data for Roof Type Segmentation

Nowadays, satellites constantly supply world-wide coverage of large-scale, Very High-Resolution (VHR) satellite imagery. The interpretation of such imagery is very expensive if done by a human. However, modern deep learning methods automatically extract semantically meaningful features for image interpretation if trained on a set of input-output pairs of high quality. In 3D reconstruction, the automatic prediction of the roof-type is an open problem. Even though some research has been done to predict the roof-type, either the number of classes was limited to flat and non-flat, or the acquisition of the ground truth was done by manually labeling many buildings. But roof type information is publicly available through the internet, such as contained in the CityGML [3] dataset of Berlin, Germany. On the other hand, such datasets have only very few samples of some classes, contain mislabeling and are incomplete. But there are methods for dealing with class-imbalance, such as the focal loss [4] and inverse frequency weights and recently, an adaption of the loss function in deep learning has been proposed, which makes the training of an Fully Convolutional Neural Network (FCN) more robust to errors in the ground truth [5]. Furthermore, Semi-Supervised Learning (SSL) was extended from classification to semantic segmentation. For example, Virtual Adversarial Training (VAT) was evaluated for dense, pixel-wise classification on a benchmark dataset [6]. In this thesis, these solutions are assembled into a combined loss LCOM to train a DeepLabv3+ [7] for roof-type segmentation on an imbalanced, imperfect and incomplete training dataset. The proposed method achieves considerable improvements and successfully predicts the roof-type in many cases. But it also fails in some cases, which are visualized and discussed

Detection and height estimation of buildings from SAR and optical images using conditional random fields

[no abstract