Automated mapping of building facades by machine learning

Facades of buildings contain various types of objects which have to be recorded for information systems. The article describes a solution for this task focussing on automated classification by means of machine learning techniques. Stereo pairs of oblique images are used to derive 3D point clouds of buildings. The planes of the buildings are automatically detected. The derived planes are supplemented with a regular grid of points for which the colour values are found in the images. For each grid point of the façade additional attributes are derived from image and object data. This "intelligent" point cloud is analysed by a decision tree, which is derived from a small training set. The derived decision tree is then used to classify the complete point cloud. To each point of the regular façade grid a class is assigned and a façade plan is mapped by a colour palette representing the different objects. Some image processing methods are applied to improve the appearance of the interpreted façade plot and to extract additional information. The proposed method is tested on facades of a church. Accuracy measures were derived from 140 independent checkpoints, which were randomly selected. When selecting four classes ("window", "stone work", "painted wall", and "vegetation") the overall accuracy is assessed with 80 % (95 % Confidence Interval: 71 %–88 %). The user accuracy of class “stonework” was assessed with 90 % (95 % CI: 80 %–97 %). The proposed methodology has a high potential for automation and fast processing

An Approach Of Automatic Reconstruction Of Building Models For Virtual Cities From Open Resources

Along with the ever-increasing popularity of virtual reality technology in recent years, 3D city models have been used in different applications, such as urban planning, disaster management, tourism, entertainment, and video games. Currently, those models are mainly reconstructed from access-restricted data sources such as LiDAR point clouds, airborne images, satellite images, and UAV (uncrewed air vehicle) images with a focus on structural illustration of buildings’ contours and layouts. To help make 3D models closer to their real-life counterparts, this thesis research proposes a new approach for the automatic reconstruction of building models from open resources. In this approach, first, building shapes are reconstructed by using the structural and geographic information retrievable from the open repository of OpenStreetMap (OSM). Later, images available from the street view of Google maps are used to extract information of the exterior appearance of buildings for texture mapping onto their boundaries. The constructed 3D environment is used as prior knowledge for the navigation purposes in a self-driving car. The static objects from the 3D model are compared with the real-time images of static objects to reduce the computation time by eliminating them from the detection proces

Building Façade Separation in Vertical Aerial Images

Three-dimensional models of urban environments have great appeal and offer promises of interesting applications. While initially it was of interest to just have such 3D data, it increasingly becomes evident that one really would like to have interpreted urban objects. To be able to interpret buildings we have to split a visible whole building block into its different single buildings. Usually this is done using cadastral information to divide the single land parcels. The problem in this case is that sometimes the building boundaries derived from the cadastre are insufficiently accurate due to several reasons like old databases with lower accuracies or inaccuracies due to transformation between two coordinate systems. For this reason it can happen that a cadastral boundary coming from an old map is displaced by up to several meters and therefore divides two buildings incorrectly. To overcome such problems we incorporate the information from vertical aerial images. We introduce a façade separation method that is able to find individual building façades using multi view stereo. The purpose is to identify the individual façades and separate them from one another before on proceeds with the analysis of a façade’s details. The source was a set of overlapping, thus “redundant ” vertical aerial images taken by an UltraCam digital aerial camera. Therefore in a first step we determine the building block outlines using the building classification and use the height values from the Digital Surface Model (DSM) to determine approximate “façade quadrilaterals”. We also incorporate height discontinuities using the height profiles along the building outlines to enhance our façade separation. In a next step we detect repeated pattern in these “façade images ” and use them to separate the façades respectively building blocks from one another

Close range mini Uavs photogrammetry for architecture survey

The survey of historical façades contains several bottlenecks, mainly related to the geometrical structure, the decorative framework, the presence of natural or artificial obstacles, the environment limitations. Urban context presents additional restrictions, binding by ground acquisition activity and leading to building data loss. The integration of TLS and close-range photogrammetry allows to go over such stuff, not overcoming the shadows effect due to the ground point of view. In the last year the massive use of UAVs in survey activity has permitted to enlarge survey capabilities, reaching a deeper knowledge in the architecture analysis. In the meanwhile, several behaviour rules have been introduced in different countries, regulating the UAVs use in different field, strongly restricting their application in urban areas. Recently very small and light platforms have been presented, which can partially overcome these rules restrictions, opening to very interesting future scenarios. This article presents the application of one of these very small RPAS (less than 300 g), equipped with a low-cost camera, in a close range photogrammetric survey of an historical building façade in Bologna (Italy). The suggested analysis tries to point out the system accuracy and details acquisition capacity. The final aim of the paper is to validate the application of this new platform in an architectonic survey pipeline, widening the future application of close-range photogrammetry in the architecture acquisition process

UAV PHOTOGRAMMETRY WITH OBLIQUE IMAGES: FIRST ANALYSIS ON DATA ACQUISITION AND PROCESSING

In recent years, many studies revealed the advantages of using airborne oblique images for obtaining improved 3D city models (e.g. including façades and building footprints). Expensive airborne cameras, installed on traditional aerial platforms, usually acquired the data. The purpose of this paper is to evaluate the possibility of acquire and use oblique images for the 3D reconstruction of a historical building, obtained by UAV (Unmanned Aerial Vehicle) and traditional COTS (Commercial Off-the-Shelf) digital cameras (more compact and lighter than generally used devices), for the realization of high-level-of-detail architectural survey. The critical issues of the acquisitions from a common UAV (flight planning strategies, ground control points, check points distribution and measurement, etc.) are described. Another important considered aspect was the evaluation of the possibility to use such systems as low cost methods for obtaining complete information from an aerial point of view in case of emergency problems or, as in the present paper, in the cultural heritage application field. The data processing was realized using SfM-based approach for point cloud generation: different dense image-matching algorithms implemented in some commercial and open source software were tested. The achieved results are analysed and the discrepancies from some reference LiDAR data are computed for a final evaluation. The system was tested on the S. Maria Chapel, a part of the Novalesa Abbey (Italy)

A Low Cost UWB Based Solution for Direct Georeferencing UAV Photogrammetry

Thanks to their flexibility and availability at reduced costs, Unmanned Aerial Vehicles (UAVs) have been recently used on a wide range of applications and conditions. Among these, they can play an important role in monitoring critical events (e.g., disaster monitoring) when the presence of humans close to the scene shall be avoided for safety reasons, in precision farming and surveying. Despite the very large number of possible applications, their usage is mainly limited by the availability of the Global Navigation Satellite System (GNSS) in the considered environment: indeed, GNSS is of fundamental importance in order to reduce positioning error derived by the drift of (low-cost) Micro-Electro-Mechanical Systems (MEMS) internal sensors. In order to make the usage of UAVs possible even in critical environments (when GNSS is not available or not reliable, e.g., close to mountains or in city centers, close to high buildings), this paper considers the use of a low cost Ultra Wide-Band (UWB) system as the positioning method. Furthermore, assuming the use of a calibrated camera, UWB positioning is exploited to achieve metric reconstruction on a local coordinate system. Once the georeferenced position of at least three points (e.g., positions of three UWB devices) is known, then georeferencing can be obtained, as well. The proposed approach is validated on a specific case study, the reconstruction of the façade of a university building. Average error on 90 check points distributed over the building façade, obtained by georeferencing by means of the georeferenced positions of four UWB devices at fixed positions, is 0.29 m. For comparison, the average error obtained by using four ground control points is 0.18 m

Comparison of building thermography approaches using terrestrial and aerial thermographic images

Thermography is commonly used for auditing buildings. Classical manual terrestrial thermography records images of individual buildings at a short distance. When auditing a large number of buildings (e.g. whole city districts) this approach reaches its limits. Using drones with thermographic cameras allows images to be recorded automatically from different angles, with faster speed and without violating property rights. However, an airborne camera has a significantly greater distance and more varied angles to a building compared to terrestrial thermography. To investigate the influence of these factors for building auditing, we perform a study evaluating seven different drone settings of varying flight speed, angle, and altitude. A comparison is drawn to manually recorded terrestrial thermographic images. While we find that a flight speed between 1m/s and 3m/s does not influence the thermographic quality, high flight altitudes and steep viewing angles lead to a significant reduction of visible details, contrast, and to falsified temperatures. A flight altitude of 12m over buildings is found to be the most suitable for the qualitative and quantitative analysis of rooftops and a qualitative analysis of façades. A flight altitude of 42m over buildings can only be used for qualitative audits with little detail

Development of inventory datasets through remote sensing and direct observation data for earthquake loss estimation

This report summarizes the lessons learnt in extracting exposure information for the three study sites, Thessaloniki, Vienna and Messina that were addressed in SYNER-G. Fine scale information on exposed elements that for SYNER-G include buildings, civil engineering works and population, is one of the variables used to quantify risk. Collecting data and creating exposure inventories is a very time-demanding job and all possible data-gathering techniques should be used to address the data shortcoming problem. This report focuses on combining direct observation and remote sensing data for the development of exposure models for seismic risk assessment. In this report a summary of the methods for collecting, processing and archiving inventory datasets is provided in Chapter 2. Chapter 3 deals with the integration of different data sources for optimum inventory datasets, whilst Chapters 4, 5 and 6 provide some case studies where combinations between direct observation and remote sensing have been used. The cities of Vienna (Austria), Thessaloniki (Greece) and Messina (Italy) have been chosen to test the proposed approaches.JRC.G.5-European laboratory for structural assessmen

OBLIQUE MULTI-CAMERA SYSTEMS - ORIENTATION AND DENSE MATCHING ISSUES

International audience3D Optical Metrology (3DOM) unit, Bruno Kessler Foundation (FBK), Trento, Italy <rupnik, franex, remondino>@fbk.eu, http://3dom.fbk.eu Commission III-WG4 ABS TRACT: The use of oblique imagery has become a standard for many civil and mapping applications, thanks to the development of airborne digital multi-camera systems, as proposed by many companies (Blomoblique, IGI, Leica, M idas, Pictometry, Vexcel/M icrosoft, VisionM ap, etc.). The indisputable virtue of oblique photography lies in its simplicity of interpretation and understanding for inexperienced users allowing their use of oblique images in very different applications, such as building detection and reconstruction, building structural damage classification, road land updating and administration services, etc. The paper reports an overview of the actual oblique commercial systems and presents a workflow for the automated orientation and dense matching of large image blocks. Perspectives, potentialities, pitfalls and suggestions for achieving satisfactory results are given. Tests performed on two datasets acquired with two multi-camera systems over urban areas are also reported. Figure 1: Large urban area pictured with an oblique multi-camera system. Once advanced image triangulation methods have retrieved interior and exterior parameters of the cameras, dense point clouds can be deriv ed for 3D city modelling, feature extraction and mapping purposes