Monitoring spatial sustainable development: Semi-automated analysis of satellite and aerial images for energy transition and sustainability indicators

Solar panels are installed by a large and growing number of households due to the convenience of having cheap and renewable energy to power house appliances. In contrast to other energy sources solar installations are distributed very decentralized and spread over hundred-thousands of locations. On a global level more than 25% of solar photovoltaic (PV) installations were decentralized. The effect of the quick energy transition from a carbon based economy to a green economy is though still very difficult to quantify. As a matter of fact the quick adoption of solar panels by households is difficult to track, with local registries that miss a large number of the newly built solar panels. This makes the task of assessing the impact of renewable energies an impossible task. Although models of the output of a region exist, they are often black box estimations. This project's aim is twofold: First automate the process to extract the location of solar panels from aerial or satellite images and second, produce a map of solar panels along with statistics on the number of solar panels. Further, this project takes place in a wider framework which investigates how official statistics can benefit from new digital data sources. At project completion, a method for detecting solar panels from aerial images via machine learning will be developed and the methodology initially developed for BE, DE and NL will be standardized for application to other EU countries. In practice, machine learning techniques are used to identify solar panels in satellite and aerial images for the province of Limburg (NL), Flanders (BE) and North Rhine-Westphalia (DE).Comment: This document provides the reader with an overview of the various datasets which will be used throughout the project. The collection of satellite and aerial images as well as auxiliary information such as the location of buildings and roofs which is required to train, test and validate the machine learning algorithm that is being develope

On the Importance of 3D Surface Information for Remote Sensing Classification Tasks

There has been a surge in remote sensing machine learning applications that operate on data from active or passive sensors as well as multi-sensor combinations (Ma et al. (2019)). Despite this surge, however, there has been relatively little study on the comparative value of 3D surface information for machine learning classification tasks. Adding 3D surface information to RGB imagery can provide crucial geometric information for semantic classes such as buildings, and can thus improve out-of-sample predictive performance. In this paper, we examine in-sample and out-of-sample classification performance of Fully Convolutional Neural Networks (FCNNs) and Support Vector Machines (SVMs) trained with and without 3D normalized digital surface model (nDSM) information. We assess classification performance using multispectral imagery from the International Society for Photogrammetry and Remote Sensing (ISPRS) 2D Semantic Labeling contest and the United States Special Operations Command (USSOCOM) Urban 3D Challenge. We find that providing RGB classifiers with additional 3D nDSM information results in little increase in in-sample classification performance, suggesting that spectral information alone may be sufficient for the given classification tasks. However, we observe that providing these RGB classifiers with additional nDSM information leads to significant gains in out-of-sample predictive performance. Specifically, we observe an average improvement in out-of-sample all-class accuracy of 14.4% on the ISPRS dataset and an average improvement in out-of-sample F1 score of 8.6% on the USSOCOM dataset. In addition, the experiments establish that nDSM information is critical in machine learning and classification settings that face training sample scarcity

Detecting demolished buildings after a natural hazard using high resolution RGB satellite imagery and modified U-Net convolutional neural networks

Collapsed buildings are usually linked with the highest number of human casualties reported after a natural disaster; therefore, quickly finding collapsed buildings can expedite rescue operations and save human lives. Recently, many researchers and agencies have tried to integrate satellite imagery into rapid response. The U.S. Defense Innovation Unit Experimental (DIUx) and National Geospatial Intelligence Agency (NGA) have recently released a ready-to-use dataset known as xView that contains thousands of labeled VHR RGB satellite imagery scenes with 30-cm spatial and 8-bit radiometric resolutions, respectively. Two of the labeled classes represent demolished buildings with 1067 instances and intact buildings with more than 300,000 instances, and both classes are associated with building footprints. In this study, we are using the xView imagery, with building labels (demolished and intact) to create a deep learning framework for classifying buildings as demolished or intact after a natural hazard event. We have used a modified U-Net style fully convolutional neural network (CNN). The results show that the proposed framework has 78% and 95% sensitivity in detecting the demolished and intact buildings, respectively, within the xView dataset. We have also tested the transferability and performance of the trained network on an independent dataset from the 19 September 2017 M 7.1 Pueblo earthquake in central Mexico using Google Earth imagery. To this end, we tested the network on 97 buildings including 10 demolished ones by feeding imagery and building footprints into the trained algorithm. The sensitivity for intact and demolished buildings was 89% and 60%, respectively.Published versio

Cybergis-enabled remote sensing data analytics for deep learning of landscape patterns and dynamics

Mapping landscape patterns and dynamics is essential to various scientific domains and many practical applications. The availability of large-scale and high-resolution light detection and ranging (LiDAR) remote sensing data provides tremendous opportunities to unveil complex landscape patterns and better understand landscape dynamics from a 3D perspective. LiDAR data have been applied to diverse remote sensing applications where large-scale landscape mapping is among the most important topics. While researchers have used LiDAR for understanding landscape patterns and dynamics in many fields, to fully reap the benefits and potential of LiDAR is increasingly dependent on advanced cyberGIS and deep learning approaches. In this context, the central goal of this dissertation is to develop a suite of innovative cyberGIS-enabled deep-learning frameworks for combining LiDAR and optical remote sensing data to analyze landscape patterns and dynamics with four interrelated studies. The first study demonstrates a high-accuracy land-cover mapping method by integrating 3D information from LiDAR with multi-temporal remote sensing data using a 3D deep-learning model. The second study combines a point-based classification algorithm and an object-oriented change detection strategy for urban building change detection using deep learning. The third study develops a deep learning model for accurate hydrological streamline detection using LiDAR, which has paved a new way of harnessing LiDAR data to map landscape patterns and dynamics at unprecedented computational and spatiotemporal scales. The fourth study resolves computational challenges in handling remote sensing big data and deep learning of landscape feature extraction and classification through a cutting-edge cyberGIS approach

3D detection of roof sections from a single satellite image and application to LOD2-building reconstruction

Reconstructing urban areas in 3D out of satellite raster images has been a long-standing and challenging goal of both academical and industrial research. The rare methods today achieving this objective at a Level Of Details $2$ rely on procedural approaches based on geometry, and need stereo images and/or LIDAR data as input. We here propose a method for urban 3D reconstruction named KIBS(\textit{Keypoints Inference By Segmentation}), which comprises two novel features: i) a full deep learning approach for the 3D detection of the roof sections, and ii) only one single (non-orthogonal) satellite raster image as model input. This is achieved in two steps: i) by a Mask R-CNN model performing a 2D segmentation of the buildings' roof sections, and after blending these latter segmented pixels within the RGB satellite raster image, ii) by another identical Mask R-CNN model inferring the heights-to-ground of the roof sections' corners via panoptic segmentation, unto full 3D reconstruction of the buildings and city. We demonstrate the potential of the KIBS method by reconstructing different urban areas in a few minutes, with a Jaccard index for the 2D segmentation of individual roof sections of $88.55\%$ and $75.21\%$ on our two data sets resp., and a height's mean error of such correctly segmented pixels for the 3D reconstruction of $1.60$ m and $2.06$ m on our two data sets resp., hence within the LOD2 precision range

Satellite based synthetic aperture radar and optical spatial-temporal information as aid for operational and environmental mine monitoring

A sustainable society is a society that satisfies its resource requirements without endangering the sustainability of these resources. The mineral endowment on the African continent is estimated to be the first or second largest of world reserves. Therefore, it is recognised that the African continent still heavily depends on mineral exports as a key contributor to the gross domestic product (GDP) of various countries. These mining activities, however, do introduce primary and secondary environmental degradation factors. They attract communities to these mining areas, light and heavy industrial establishments occur, giving rise to artisanal activities. This study focussed on satellite RS products as an aid to a mine’s operations and the monitoring of its environment. Effective operational mine management and control ensures a more sustainable and profitable lifecycle for mines. Satellite based RS holds the potential to observe the mine and its surrounding areas at high temporal intervals, different spectral wavelengths and spatial resolutions. The combination of SAR and optical information creates a spatial platform to observe and measure the mine’s operations and the behaviour of specific land cover and land use classes over time and contributes to a better understanding of the mining activities and their influence on the environment within a specific geographical area. This study will introduce an integrated methodology to collect, process and analyse spatial information over a specific targeted mine. This methodology utilises a medium resolution land cover base map, derived from Landsat 8, to understand the predominant land cover types of the surrounding area. Using very high resolution mono- and stereoscopic satellite imagery provides a finer scale analysis and identifies changes in features at a smaller scale. Combining these technologies with the synthetic aperture radar (SAR) applications for precise measurement of surface subsidence or upliftment becomes a spatial toolbox for mine management. This study examines a combination of satellite remote sensing products guided by a systematic workflow methodology to integrate spatial results as an aid for mining operations and environmental monitoring. Some of the results that can be highlighted is the successful land cover classification using the Landsat 8 satellite. The land cover that dominated the Kolomela mine area was the “SHRUBLAND/GRASS” class with a 94% coverage and “MINE” class of 2.6%. Sishen mine had a similar dominated land cover characteristic with a “SHRUBLAND/GRASS” class of 90% and “MINE” class of 4.8%. The Pléiades time-series classification analysis was done using three scenes each acquired at a different time interval. The Sishen and Kolomela mine showed especially changes from the bare soil class to the asphalt or mine class. The Pléiades stereoscopic analysis provided volumetric change detection over small, medium, large and recessed areas. Both the Sishen and Kolomela mines demonstrated height profile changes in each selected category. The last category of results focused on the SAR technology to measure within millimetre accuracy the subsidence and upliftment behaviour of surface areas over time. The Royal Bafokeng Platinum tailings pond area was measured using 74 TerraSAR-X scenes. The tailings wall area was confirmed as stable with natural subsidence that occurred in its surrounding area due to seasonal changes of the soil during rainy and dry periods. The Chuquicamata mine as a large open pit copper mine area was analysed using 52 TerraSAR-X scenes. The analysis demonstrated significant vertical surface movement over some of the dumping sites. It is the wish of the researcher that this dissertation and future research scholars will continue to contribute in this scientific field. These contributions can only assist the mining sector to continuously improve its mining operations as well as its monitoring of the primary as well as the secondary environmental impacts to ensure improved sustainability for the next generation.Environmental SciencesM. Sc. (Environmental Science