A Comparative Evaluation of Deep Learning Techniques for Photovoltaic Panel Detection From Aerial Images

Solar energy production has significantly increased in recent years in the European Union (EU), accounting for 12% of the total in 2022. The growth in solar energy production can be attributed to the increasing adoption of solar photovoltaic (PV) panels, which have become cost-effective and efficient means of energy production, supported by government policies and incentives. The maturity of solar technologies has also led to a decrease in the cost of solar energy, making it more competitive with other energy sources. As a result, there is a growing need for efficient methods for detecting and mapping the locations of PV panels. Automated detection can in fact save time and resources compared to manual inspection. Moreover, the resulting information can also be used by governments, environmental agencies and other companies to track the adoption of renewable sources or to optimize energy distribution across the grid. However, building effective models to support the automated detection and mapping of solar photovoltaic (PV) panels presents several challenges, including the availability of high-resolution aerial imagery and high-quality, manually-verified labels and annotations. In this study, we address these challenges by first constructing a dataset of PV panels using very-high-resolution (VHR) aerial imagery, specifically focusing on the region of Piedmont in Italy. The dataset comprises 105 large-scale images, providing more than 9,000 accurate and detailed manual annotations, including additional attributes such as the PV panel category. We first conduct a comprehensive evaluation benchmark on the newly constructed dataset, adopting various well-established deep-learning techniques. Specifically, we experiment with instance and semantic segmentation approaches, such as Rotated Faster RCNN and Unet, comparing strengths and weaknesses on the task at hand. Second, we apply ad-hoc modifications to address the specific issues of this task, such as the wide range of scales of the installations and the sparsity of the annotations, considerably improving upon the baseline results. Last, we introduce a robust and efficient post-processing polygonization algorithm that is tailored to PV panels. This algorithm converts the rough raster predictions into cleaner and more precise polygons for practical use. Our benchmark evaluation shows that both semantic and instance segmentation techniques can be effective for detecting and mapping PV panels. Instance segmentation techniques are well-suited for estimating the localization of panels, while semantic solutions excel at surface delineation. We also demonstrate the effectiveness of our ad-hoc solutions and post-processing algorithm, which can provide an improvement up to +10% on the final scores, and can accurately convert coarse raster predictions into usable polygons

Operational Pipeline for Large-scale 3D Reconstruction of Buildings from Satellite Images

International audienceAutomatic 3D reconstruction of urban scenes from stereo pairs of satellite images remains a popular yet challenging research topic, driven by numerous applications such as telecommunications and defense. The quality of reconstruction results depends particularly on the quality of the available stereo pair. In this paper, we propose an operational pipeline for large-scale 3D reconstruction of buildings from stereo satellite images. The proposed chain uses U-net to extract contour polygons of buildings, and the combination of optimization and computational geometry techniques to reconstruct a digital terrain model and a digital height model, and to correctly estimate the position of building footprints. The pipeline has proven to be efficient for 3D building reconstruction , even if the close-to-nadir image is not available

Elevation Estimation-Driven Building 3D Reconstruction from Single-View Remote Sensing Imagery

Building 3D reconstruction from remote sensing images has a wide range of applications in smart cities, photogrammetry and other fields. Methods for automatic 3D urban building modeling typically employ multi-view images as input to algorithms to recover point clouds and 3D models of buildings. However, such models rely heavily on multi-view images of buildings, which are time-intensive and limit the applicability and practicality of the models. To solve these issues, we focus on designing an efficient DSM estimation-driven reconstruction framework (Building3D), which aims to reconstruct 3D building models from the input single-view remote sensing image. First, we propose a Semantic Flow Field-guided DSM Estimation (SFFDE) network, which utilizes the proposed concept of elevation semantic flow to achieve the registration of local and global features. Specifically, in order to make the network semantics globally aware, we propose an Elevation Semantic Globalization (ESG) module to realize the semantic globalization of instances. Further, in order to alleviate the semantic span of global features and original local features, we propose a Local-to-Global Elevation Semantic Registration (L2G-ESR) module based on elevation semantic flow. Our Building3D is rooted in the SFFDE network for building elevation prediction, synchronized with a building extraction network for building masks, and then sequentially performs point cloud reconstruction, surface reconstruction (or CityGML model reconstruction). On this basis, our Building3D can optionally generate CityGML models or surface mesh models of the buildings. Extensive experiments on ISPRS Vaihingen and DFC2019 datasets on the DSM estimation task show that our SFFDE significantly improves upon state-of-the-arts. Furthermore, our Building3D achieves impressive results in the 3D point cloud and 3D model reconstruction process

AN END-TO-END DEEP LEARNING WORKFLOW FOR BUILDING SEGMENTATION, BOUNDARY REGULARIZATION AND VECTORIZATION OF BUILDING FOOTPRINTS

Automatic building footprint extraction from remote sensing imagery is a widely used method, with deep learning techniques being particularly effective. However, deep learning approaches still require additional post-processing steps due to pixel-wise predictions, that contribute to occluded and geometrically incorrectly segmented buildings. To address this issue, we propose an end-to-end workflow that utilizes binary semantic segmentation, regularization, and vectorization. We implement and assess the performance of four convolutional neural network architectures including U-Net, U-NetFormer, FT-UnetFormer, and DCSwin on the MapAI Precision in Building Segmentation competition. To additionally improve the shape of the predicted buildings we apply regularization on the predictions to assess whether regularization further improves the geometrical shape and improve the prediction accuracy. We aim to produce accurate predictions with regularized boundaries that can prove useful in many cartographic and engineering applications. The regularization and vectorization workflow is further developed into a working QGIS-plugin that can be used to extend the functionality of QGIS. Our aim is to provide an end-to-end workflow for building segmentation, regularization and vectorization

Analysis and Interpretation of Graphical Documents

International audienceThis chapter is dedicated to the analysis and the interpretation of graphical documents, and as such, builds upon many of the topics covered in other parts of this handbook. It will therefore not focus on any of the technical issues related to graphical documents, such as low level filtering and binarization, primitive extraction and vectorization as developed in Chapters 2.1 and 5.1 or symbol recognition, for instance, as developed in Chapter 5.2. These tools are put in a broader framework and threaded together in complex pipelines to solve interpretation questions. This chapter provides an overview of how analysis strategies have contributed to constructing these pipelines, how specific domain knowledge is integrated in these analyses, and which interpretation contexts have been contributed to successful approaches

Integrating openstreetmap data and sentinel-2 Imagery for classifying and monitoring informal settlements

Dissertation submitted in partial fulfilment of the requirements for the degree of Master of Science in Geospatial TechnologiesThe identification and monitoring of informal settlements in urban areas is an important step in developing and implementing pro-poor urban policies. Understanding when, where and who lives inside informal settlements is critical to efforts to improve their resilience. This study aims at integrating OSM data and sentinel-2 imagery for classifying and monitoring the growth of informal settlements methods to map informal areas in Kampala (Uganda) and Dar es Salaam (Tanzania) and to monitor their growth in Kampala. Three building feature characteristics of size, shape and Distance to nearest Neighbour were derived and used to cluster and classify informal areas using Hotspot Cluster analysis and ML approach on OSM buildings data. The resultant informal regions in Kampala were used with Sentinel-2 image tiles to investigate the spatiotemporal changes in informal areas using Convolutional Neural Networks (CNNs). Results from Optimized Hot Spot Analysis and Random Forest Classification show that Informal regions can be mapped based on building outline characteristics. An accuracy of 90.3% was achieved when an optimally trained CNN was executed on a test set of 2019 satellite image tiles. Predictions of informality from new datasets for the years 2016 and 2017 provided promising results on combining different open source geospatial datasets to identify, classify and monitor informal settlements

Fine Art Pattern Extraction and Recognition

This is a reprint of articles from the Special Issue published online in the open access journal Journal of Imaging (ISSN 2313-433X) (available at: https://www.mdpi.com/journal/jimaging/special issues/faper2020)

Deep Learning Based Exposure Analysis of LandslideProne Areas in Medellín, Colombia

In the last century, Medellín grew into one of Colombia's largest cities. Today, the city continues to grow primarily due to the influx of internally displaced people (IDP’s), who have been forced to leave their homes at the country side due to natural disasters or drug-related violence. Since the internally displaced are mostly lowincome farmers and peasants, they are migrating to the larger cities in search of greater security and jobs. In Medellín, the new residents mostly settle informally on the steep slopes to the east and west of the city. Due to limited space and steep topography, such settlements are often built in areas with medium and high probability of landslides. However, not only free land area within the municipal boundaries are exploited by the build-up of new settlements, but also free land beyond the border of the municipality, which causes the city to grow into the rural area. The study therefore seeks to find out how many residents are prone to potential landslide activity in the context of the pattern of migration. To analyze the exposure, the population is disaggregated down to the individual building block level. Such an approach requires precise building footprints to locate the population in relation to landslide-prone areas. Although the city has a cadaster including building footprints, it is more imprecise and incomplete towards the outskirts of the city, where landslide susceptibility is most pronounced. The incompleteness is due to the high population dynamics, which makes it quite difficult to maintain an up-to-date cadaster. But since Medellín's geospatial data service provides an orthophoto from 2019, a deep learning-based building extraction is applied to generate a more comprehensive building footprint dataset. This will be the main data source for the exposure analysis. The respective deep learning architecture is a U-Net has been refined with the EfficientNetB2 as a backbone and eventually fine-tuned. It could show very accurate results, while still facing some challenges, like the field-of.view of the image tiles, that is sometimes too small for the vast rooftop landscapes, which leads to misclassifications. The exposure analysis of population exposed to landslide hazard could prove the importance of having a more up-to-date data basis. While the number of residents living in formal settlements is not to different from the cadaster and the deeplearning derived building footprints, those numbers of residents of the informal settlements are much higher in the more actual deep learning derived dataset. A strong increase could also be found in the population exposed to medium and high landslide hazard. Further analyses facilitate the impression, that the poorer and the more vulnerable population has distinctively higher exposure to considerable landslide hazard, when using the deep-learning derived dataset. These findings show the strength of remote sensing techniques in order to retrieve actual building footprint data, that is clearly important for the estimation of potential consequences of landslide-prone areas