Semi-Supervised Learning from Street-View Images and OpenStreetMap for Automatic Building Height Estimation

Accurate building height estimation is key to the automatic derivation of 3D city models from emerging big geospatial data, including Volunteered Geographical Information (VGI). However, an automatic solution for large-scale building height estimation based on low-cost VGI data is currently missing. The fast development of VGI data platforms, especially OpenStreetMap (OSM) and crowdsourced street-view images (SVI), offers a stimulating opportunity to fill this research gap. In this work, we propose a semi-supervised learning (SSL) method of automatically estimating building height from Mapillary SVI and OSM data to generate low-cost and open-source 3D city modeling in LoD1. The proposed method consists of three parts: first, we propose an SSL schema with the option of setting a different ratio of "pseudo label" during the supervised regression; second, we extract multi-level morphometric features from OSM data (i.e., buildings and streets) for the purposed of inferring building height; last, we design a building floor estimation workflow with a pre-trained facade object detection network to generate "pseudo label" from SVI and assign it to the corresponding OSM building footprint. In a case study, we validate the proposed SSL method in the city of Heidelberg, Germany and evaluate the model performance against the reference data of building heights. Based on three different regression models, namely Random Forest (RF), Support Vector Machine (SVM), and Convolutional Neural Network (CNN), the SSL method leads to a clear performance boosting in estimating building heights with a Mean Absolute Error (MAE) around 2.1 meters, which is competitive to state-of-the-art approaches. The preliminary result is promising and motivates our future work in scaling up the proposed method based on low-cost VGI data, with possibilities in even regions and areas with diverse data quality and availability

PROSPECTIVE UPON MULTI-SOURCE URBAN SCALE DATA FOR 3D DOCUMENTATION AND MONITORING OF URBAN LEGACIES

Abstract. The investigation on the built urban heritage and its current transformations can progressively benefit from the use of geospatial data related to urban environment. This is even more interesting when urban design studies of historical and stratified cities meet the contribution of 4D geospatial data within the urban morphology researches, aiming at quickly and accurately identifying and then measuring with a spatial relationship, both localized transformation (volumes demolitions, addition, etc…) and wide-scale substantial modification resulting from urban zones of diversification spaces that incorporates urban legacies. In this domain, the comparison and analysis of multi-source and multi-scale information belonging to Spatial Data Infrastructures (SDI) organized by Municipality and Region Administration (mainly, orthoimages and DSM and digital mapping) are a crucial support for multi-temporal spatial analysis, especially if compared with new DSMs related to past urban situations. The latter can be generated by new solution of digital image-matching techniques applicable to the available historical aerial images. The goal is to investigate the amount of available data and their effectiveness, to later test different experimental tools and methods for quick detection, localization and quantification of morphological macro-transformation at urban scale. At the same time, it has been examined the opportunity to made available, with up-and-coming Mobile Mapping Systems (MMS) based on image- and range-based techniques, a rapid and effective approach of data gathering, updating and sharing at validated urban scales. The presented research, carried out in the framework of the FULL@Polito research lab, applies to urban legacies and their regeneration, and is conducted on a key redevelopment area in northern Torino, the Parco Dora, that was occupied by steel industries actively working up to 1992. The long-standing steel structures of the Ferriere FIAT lot have been refurbished and incorporated in the new urban park, generating a contemporary space with a new evolving urban fabric, and being integrated in the new updated geo-spatial databases as well.</p

IM2ELEVATION: Building Height Estimation from Single-View Aerial Imagery

Estimation of the Digital Surface Model (DSM) and building heights from single-view aerial imagery is a challenging inherently ill-posed problem that we address in this paper by resorting to machine learning. We propose an end-to-end trainable convolutional-deconvolutional deep neural network architecture that enables learning mapping from a single aerial imagery to a DSM for analysis of urban scenes. We perform multisensor fusion of aerial optical and aerial light detection and ranging (Lidar) data to prepare the training data for our pipeline. The dataset quality is key to successful estimation performance. Typically, a substantial amount of misregistration artifacts are present due to georeferencing/projection errors, sensor calibration inaccuracies, and scene changes between acquisitions. To overcome these issues, we propose a registration procedure to improve Lidar and optical data alignment that relies on Mutual Information, followed by Hough transform-based validation step to adjust misregistered image patches. We validate our building height estimation model on a high-resolution dataset captured over central Dublin, Ireland: Lidar point cloud of 2015 and optical aerial images from 2017. These data allow us to validate the proposed registration procedure and perform 3D model reconstruction from single-view aerial imagery. We also report state-of-the-art performance of our proposed architecture on several popular DSM estimation datasets

3D Spatial Data Infrastructures for web-based Visualization

In this thesis, concepts for developing Spatial Data Infrastructures with an emphasis on visualizing 3D landscape and city models in distributed environments are discussed. Spatial Data Infrastructures are important for public authorities in order to perform tasks on a daily basis, and serve as research topic in geo-informatics. Joint initiatives at national and international level exist for harmonizing procedures and technologies. Interoperability is an important aspect in this context - as enabling technology for sharing, distributing, and connecting geospatial data and services. The Open Geospatial Consortium is the main driver for developing international standards in this sector and includes government agencies, universities and private companies in a consensus process. 3D city models are becoming increasingly popular not only in desktop Virtual Reality applications but also for being used in professional purposes by public authorities. Spatial Data Infrastructures focus so far on the storage and exchange of 3D building and elevation data. For efficient streaming and visualization of spatial 3D data in distributed network environments such as the internet, concepts from the area of real time 3D Computer Graphics must be applied and combined with Geographic Information Systems (GIS). For example, scene graph data structures are commonly used for creating complex and dynamic 3D environments for computer games and Virtual Reality applications, but have not been introduced in GIS so far. In this thesis, several aspects of how to create interoperable and service-based environments for 3D spatial data are addressed. These aspects are covered by publications in journals and conference proceedings. The introductory chapter provides a logic succession from geometrical operations for processing raw data, to data integration patterns, to system designs of single components, to service interface descriptions and workflows, and finally to an architecture of a complete distributed service network. Digital Elevation Models are very important in 3D geo-visualization systems. Data structures, methods and processes are described for making them available in service based infrastructures. A specific mesh reduction method is used for generating lower levels of detail from very large point data sets. An integration technique is presented that allows the combination with 2D GIS data such as roads and land use areas. This approach allows using another optimization technique that greatly improves the usability for immersive 3D applications such as pedestrian navigation: flattening road and water surfaces. It is a geometric operation, which uses data structures and algorithms found in numerical simulation software implementing Finite Element Methods. 3D Routing is presented as a typical application scenario for detailed 3D city models. Specific problems such as bridges, overpasses and multilevel networks are addressed and possible solutions described. The integration of routing capabilities in service infrastructures can be accomplished with standards of the Open Geospatial Consortium. An additional service is described for creating 3D networks and for generating 3D routes on the fly. Visualization of indoor routes requires different representation techniques. As server interface for providing access to all 3D data, the Web 3D Service has been used and further developed. Integrating and handling scene graph data is described in order to create rich virtual environments. Coordinate transformations of scene graphs are described in detail, which is an important aspect for ensuring interoperability between systems using different spatial reference systems. The Web 3D Service plays a central part in nearly all experiments that have been carried out. It does not only provide the means for interactive web-visualizations, but also for performing further analyses, accessing detailed feature information, and for automatic content discovery. OpenStreetMap and other worldwide available datasets are used for developing a complete architecture demonstrating the scalability of 3D Spatial Data Infrastructures. Its suitability for creating 3D city models is analyzed, according to requirements set by international standards. A full virtual globe system has been developed based on OpenStreetMap including data processing, database storage, web streaming and a visualization client. Results are discussed and compared to similar approaches within geo-informatics research, clarifying in which application scenarios and under which requirements the approaches in this thesis can be applied

PROSPECTIVE UPON MULTI-SOURCE URBAN SCALE DATA FOR 3D DOCUMENTATION AND MONITORING OF URBAN LEGACIES

The investigation on the built urban heritage and its current transformations can progressively benefit from the use of geospatial data related to urban environment. This is even more interesting when urban design studies of historical and stratified cities meet the contribution of 4D geospatial data within the urban morphology researches, aiming at quickly and accurately identifying and then measuring with a spatial relationship, both localized transformation (volumes demolitions, addition, etc…) and wide-scale substantial modification resulting from urban zones of diversification spaces that incorporates urban legacies. In this domain, the comparison and analysis of multi-source and multi-scale information belonging to Spatial Data Infrastructures (SDI) organized by Municipality and Region Administration (mainly, orthoimages and DSM and digital mapping) are a crucial support for multi-temporal spatial analysis, especially if compared with new DSMs related to past urban situations. The latter can be generated by new solution of digital image-matching techniques applicable to the available historical aerial images. The goal is to investigate the amount of available data and their effectiveness, to later test different experimental tools and methods for quick detection, localization and quantification of morphological macro-transformation at urban scale. At the same time, it has been examined the opportunity to made available, with up-and-coming Mobile Mapping Systems (MMS) based on image- and range-based techniques, a rapid and effective approach of data gathering, updating and sharing at validated urban scales. The presented research, carried out in the framework of the FULL@Polito research lab, applies to urban legacies and their regeneration, and is conducted on a key redevelopment area in northern Torino, the Parco Dora, that was occupied by steel industries actively working up to 1992. The long-standing steel structures of the Ferriere FIAT lot have been refurbished and incorporated in the new urban park, generating a contemporary space with a new evolving urban fabric, and being integrated in the new updated geo-spatial databases as well

Creating 3D city models from satellite imagery for integrated assessment and forecasting of solar energy

Buildings are the most prominent component in the urban environment. The geometric identification of urban buildings plays an important role in a range of urban applications, including 3D representations of buildings, energy consumption analysis, sustainable development, urban planning, risk assessment, and change detection. In particular, 3D building models can provide a comprehensive assessment of surfaces exposed to solar radiation. However, the identification of the available surfaces on urban structures and the actual locations which receive a sufficient amount of sunlight to increase installed power capacity (e.g. Photovoltaic systems) are crucial considerations for solar energy supply efficiency. Although considerable research has been devoted to detecting the rooftops of buildings, less attention has been paid to creating and completing 3D models of urban buildings. Therefore, there is a need to increase our understanding of the solar energy potential of the surfaces of building envelopes so we can formulate future adaptive energy policies for improving the sustainability of cities. The goal of this thesis was to develop a new approach to automatically model existing buildings for the exploitation of solar energy potential within an urban environment. By investigating building footprints and heights based on shadow information derived from satellite images, 3D city models were generated. Footprints were detected using a two level segmentation process: (1) the iterative graph cuts approach for determining building regions and (2) the active contour method and the adjusted-geometry parameters method for modifying the edges and shapes of the extracted building footprints. Building heights were estimated based on the simulation of artificial shadow regions using identified building footprints and solar information in the image metadata at pre-defined height increments. The difference between the actual and simulated shadow regions at every height increment was computed using the Jaccard similarity coefficient. The 3D models at the first level of detail were then obtained by extruding the building footprints based on their heights by creating image voxels and using the marching cube approach. In conclusion, 3D models of buildings can be generated solely from 2D data of the buildings’attributes in any selected urban area. The approach outperforms the past attempts, and mean error is reduced by at least 21%. Qualitative evaluations of the study illustrate that it is possible to achieve 3D building models based on satellite images with a mean error of less than 5 m. This comprehensive study allows for 3D city models to be generated in the absence of elevation attributes and additional data. Experiments revealed that this novel, automated method can be useful in a number of spatial analyses and urban sustainability applications