Exploring semantic relationships for hierarchical land use classification based on convolutional neural networks

Land use (LU) is an important information source commonly stored in geospatial databases. Most current work on automatic LU classification for updating topographic databases considers only one category level (e.g. residential or agricultural) consisting of a small number of classes. However, LU databases frequently contain very detailed information, using a hierarchical object catalogue where the number of categories differs depending on the hierarchy level. This paper presents a method for the classification of LU on the basis of aerial images that differentiates a fine-grained class structure, exploiting the hierarchical relationship between categories at different levels of the class catalogue. Starting from a convolutional neural network (CNN) for classifying the categories of all levels, we propose a strategy to simultaneously learn the semantic dependencies between different category levels explicitly. The input to the CNN consists of aerial images and derived data as well as land cover information derived from semantic segmentation. Its output is the class scores at three different semantic levels, based on which predictions that are consistent with the class hierarchy are made. We evaluate our method using two test sites and show how the classification accuracy depends on the semantic category level. While at the coarsest level, an overall accuracy in the order of 90% can be achieved, at the finest level, this accuracy is reduced to around 65%. Our experiments also show which classes are particularly hard to differentiate. © 2020 Copernicus GmbH. All rights reserved

Towards better classification of land cover and land use based on convolutional neural networks

Land use and land cover are two important variables in remote sensing. Commonly, the information of land use is stored in geospatial databases. In order to update such databases, we present a new approach to determine the land cover and to classify land use objects using convolutional neural networks (CNN). High-resolution aerial images and derived data such as digital surface models serve as input. An encoder-decoder based CNN is used for land cover classification. We found a composite including the infrared band and height data to outperform RGB images in land cover classification. We also propose a CNN-based methodology for the prediction of land use label from the geospatial databases, where we use masks representing object shape, the RGB images and the pixel-wise class scores of land cover as input. For this task, we developed a two-branch network where the first branch considers the whole area of an image, while the second branch focuses on a smaller relevant area. We evaluated our methods using two sites and achieved an overall accuracy of up to 89.6% and 81.7% for land cover and land use, respectively. We also tested our methods for land cover classification using the Vaihingen dataset of the ISPRS 2D semantic labelling challenge and achieved an overall accuracy of 90.7%. © Authors 2019

Mapping Local Climate Zones for a Worldwide Database of the Form and Function of Cities

Progress in urban climate science is severely restricted by the lack of useful information that describes aspects of the form and function of cities at a detailed spatial resolution. To overcome this shortcoming we are initiating an international effort to develop the World Urban Database and Access Portal Tools (WUDAPT) to gather and disseminate this information in a consistent manner for urban areas worldwide. The first step in developing WUDAPT is a description of cities based on the Local Climate Zone (LCZ) scheme, which classifies natural and urban landscapes into categories based on climate-relevant surface properties. This methodology provides a culturally-neutral framework for collecting information about the internal physical structure of cities. Moreover, studies have shown that remote sensing data can be used for supervised LCZ mapping. Mapping of LCZs is complicated because similar LCZs in different regions have dissimilar spectral properties due to differences in vegetation, building materials and other variations in cultural and physical environmental factors. The WUDAPT protocol developed here provides an easy to understand workflow; uses freely available data and software; and can be applied by someone without specialist knowledge in spatial analysis or urban climate science. The paper also provides an example use of the WUDAPT project results

MAPPING OF HIGHLY HETEROGENEOUS URBAN STRUCTURE TYPE FOR FLOOD VULNERABILITY ASSESSMENT

Vulnerability plays an important role in risk assessment. For flood vulnerability assessment, the map and characteristics of elements-at-risk at different scales are strongly required depending on the risk and vulnerability assessment requirements. This study proposes a methodology to classify urban structure type by combining object-based image classification and different high resolution remote sensing data. In this study, a high resolution satellite image and LiDAR have been acquired over Kota Bharu, Kelantan which consists of highly heterogeneous urban structure type (UST) classes. The first stage is data pre-processing that includes orthorectification and pansharpening of Geoeye satellite image, image resampling for normalised Digital Surface Model (nDSM) and followed by image segmentation for creating meaningful objects. The second stage comprises of derivation of image features, generation of training and testing datasets, and classification of UST. The classification was based on three types of machine learning classifiers, i.e. Random Forest (RF), Support Vector Machine (SVM) and Classification and Regression Tree (CART). The results obtained from the classification processes were compared using individual omission and commission error, overcall accuracy and Kappa coefficient. The results show that Random Forest classifier with all image features achieved the highest overall accuracy (93.5%) and Kappa coefficient (0.94). This is followed by CART classifier with overall accuracy of 93.7% and Kappa coefficient of 0.92. Finally, SVM classifier produced the lowest overall accuracy and Kappa coefficient with 88.6% and 0.86, respectively. The UST classification result can be further used to assist detailed building characterisation for large scale flood vulnerability assessment

A hybrid OSVM-OCNN Method for Crop Classification from Fine Spatial Resolution Remotely Sensed Imagery

Accurate information on crop distribution is of great importance for a range of applications including crop yield estimation, greenhouse gas emission measurement and management policy formulation. Fine spatial resolution (FSR) remotely sensed imagery provides new opportunities for crop mapping at a detailed level. However, crop classification from FSR imagery is known to be challenging due to the great intra-class variability and low inter-class disparity in the data. In this research, a novel hybrid method (OSVM-OCNN) was proposed for crop classification from FSR imagery, which combines a shallow-structured object-based support vector machine (OSVM) with a deep-structured object-based convolutional neural network (OCNN). Unlike pixel-wise classification methods, the OSVM-OCNN method operates on objects as the basic units of analysis and, thus, classifies remotely sensed images at the object level. The proposed OSVM-OCNN harvests the complementary characteristics of the two sub-models, the OSVM with effective extraction of low-level within-object features and the OCNN with capture and utilization of high-level between-object information. By using a rule-based fusion strategy based primarily on the OCNN’s prediction probability, the two sub-models were fused in a concise and effective manner. We investigated the effectiveness of the proposed method over two test sites (i.e., S1 and S2) that have distinctive and heterogeneous patterns of different crops in the Sacramento Valley, California, using FSR Synthetic Aperture Radar (SAR) and FSR multispectral data, respectively. Experimental results illustrated that the new proposed OSVM-OCNN approach increased markedly the classification accuracy for most of crop types in S1 and all crop types in S2, and it consistently achieved the most accurate accuracy in comparison with its two object-based sub-models (OSVM and OCNN) as well as the pixel-wise SVM (PSVM) and CNN (PCNN) methods. Our findings, thus, suggest that the proposed method is as an effective and efficient approach to solve the challenging problem of crop classification using FSR imagery (including from different remotely sensed platforms). More importantly, the OSVM-OCNN method is readily generalisable to other landscape classes and, thus, should provide a general solution to solve the complex FSR image classification problem

DATA DRIVEN DEVELOPMENT: ANALYZING URBAN CHANGE THROUGH CONSTRUCTION, DEMOLITION, AND RENOVATION

DATA DRIVEN DEVELOPMENT: ANALYZING URBAN CHANGE THROUGH CONSTRUCTION, DEMOLITION, AND RENOVATIO

Fusion of high spatial resolution multispectral & object height data for urban environmental monitoring: methods & applications

High spatial resolution (HSR) multispectral and object height data are becoming increasingly available in the urbanized regions of the world. The synergistic utilization of these data sources holds a large potential for the fine-scale characterization of a city because they are of high descriptive power and non-redundant. However, despite this promising development, detailed and area-wide maps of important settlement parameters, like land cover (LC), urban site characteristics (USCs), and urban structure types (USTs), are still lacking in many municipalities. One reason for this observation is the methodological challenge of turning the wealth of geospatial data into reliable thematic information. Accordingly, there is a strong need for accurate and transferable software solutions being able to produce some of the key data sets for human settlement monitoring from HSR multispectral and object height data. The present work aims at addressing this need. The overall goal of the dissertation was to develop methods for the fusion of HSR multispectral and object height data as well as to showcase their utility in the context of different urban environmental mapping and monitoring applications. It therefore intended to make both a technical and an applied contribution to the field of urban remote sensing. Particular emphasis was put on mapping urban LC, USCs, and USTs, as well as the usage of USCs to study urban land surface temperature (LST) and the surface urban heat island (UHI) effect. These settlement parameters were chosen because they are thematically connected, difficult to obtain from other data sources, and of high relevance for urban planning. To meet the above goal, a comprehensive literature review was conducted in advance. The review helped identifying current deficits within the chosen research fields and led to the formulation of specific thesis objectives. The latter determined the practical agenda of this work, comprising an overall number of four studies.Die Verfügbarkeit räumlich hochaufgelöster Multispektral- und Objekthöhendaten nimmt für die urbanen Gebiete der Erde stetig zu. Die synergetische Verknüpfung solcher Daten birgt ein großes Potential zur genauen Beschreibung von Städten, da diese Daten einen hohen Informationsgehalt aufweisen und redundanzfrei sind. Trotz dieser positiven Entwicklung fehlt es in vielen Städten an detaillierten Karten, welche Aufschluss über planungsrelevante Siedlungsparameter geben. Ein Grund für diese Beobachtung ist die methodische Herausforderung, die Fülle an zugänglichen Geodaten in verlässliche thematische Informationen zu überführen. Demzufolge besteht ein großer Bedarf an akkuraten und übertragbaren Auswertungsverfahren, welche sich das Synergiepotential räumlich hochaufgelöster Multispektral- und Objekthöhendaten für ein verbessertes Stadtmonitoring zunutze machen. Die vorliegende Arbeit zielt darauf ab, diesen Bedarf zu decken. Das übergeordnete Ziel der Dissertation war, Methoden zur Fusion räumlich hochaufgelöster Multispektral- und Objekthöhendaten zu entwickeln und deren Nutzen im Rahmen stadtumweltbezogener Fragestellungen zu demonstrieren. Folglich sollte die Arbeit einen technischen und einen angewandten Beitrag auf dem Gebiet der urbanen Fernerkundung leisten. Das Hauptaugenmerk lag auf der genauen und robusten Kartierung der Landbedeckung und Stadtstruktur. Darüber hinaus wurden verschiedene urbane Bewertungsindikatoren extrahiert und zu einem neuen Dichtemaß verknüpft. Die abgeleiteten Karten und Indikatoren kamen im Zuge einer abschließenden Analyse zum Einsatz, welche sich mit den Ursprüngen städtischer Wärmeinseln befasste. Um das obige Ziel zu erreichen, wurde im Vorfeld eine umfangreiche Literaturrecherche vorgenommen. Diese ermöglichte die Identifikation derzeitiger Forschungsdefizite und führte zur Formulierung spezifischer Arbeitsziele. Nach den Zielen richtete sich der praktische Teil der kumulativen Dissertation, welcher insgesamt vier Studien umfasste

Deep learning for land cover and land use classification

Recent advances in sensor technologies have witnessed a vast amount of very fine spatial resolution (VFSR) remotely sensed imagery being collected on a daily basis. These VFSR images present fine spatial details that are spectrally and spatially complicated, thus posing huge challenges in automatic land cover (LC) and land use (LU) classification. Deep learning reignited the pursuit of artificial intelligence towards a general purpose machine to be able to perform any human-related tasks in an automated fashion. This is largely driven by the wave of excitement in deep machine learning to model the high-level abstractions through hierarchical feature representations without human-designed features or rules, which demonstrates great potential in identifying and characterising LC and LU patterns from VFSR imagery. In this thesis, a set of novel deep learning methods are developed for LC and LU image classification based on the deep convolutional neural networks (CNN) as an example. Several difficulties, however, are encountered when trying to apply the standard pixel-wise CNN for LC and LU classification using VFSR images, including geometric distortions, boundary uncertainties and huge computational redundancy. These technical challenges for LC classification were solved either using rule-based decision fusion or through uncertainty modelling using rough set theory. For land use, an object-based CNN method was proposed, in which each segmented object (a group of homogeneous pixels) was sampled and predicted by CNN with both within-object and between-object information. LU was, thus, classified with high accuracy and efficiency. Both LC and LU formulate a hierarchical ontology at the same geographical space, and such representations are modelled by their joint distribution, in which LC and LU are classified simultaneously through iteration. These developed deep learning techniques achieved by far the highest classification accuracy for both LC and LU, up to around 90% accuracy, about 5% higher than the existing deep learning methods, and 10% greater than traditional pixel-based and object-based approaches. This research made a significant contribution in LC and LU classification through deep learning based innovations, and has great potential utility in a wide range of geospatial applications

Develop Urban Configurations to Mitigate the Urban Heat Island Effect in Sydney

The Urban Heat Island (UHI) phenomenon, which is the excessive warmth of parts of cities relative to other regions and rural regions, has been caused by the rapid growth of metropolitan areas concurrent with climate change. The second factor causing increased urban heat in the following years is urban expansion. In Australia, where the annual mean temperature is increasing at a pace of 0.9°C per year. In this instance, the peculiarities of the urban structure and how it contributes to the establishment of the heat island are crucial. However, due to the insufficient understanding of urban energy balance at the precinct level, the effectiveness of urban planning and implementation for UHI mitigation is substantially limited. Most past studies have concentrated on the single building, region and metro area, but relatively little study has been done at the precinct level. In addition, the structural parameters of the built environment have a significant impact on the urban energy balance. However, few researchers are examining how morphological features at the precinct level can minimize UHIs and enhance outdoor thermal comfort. This study establishes a structure for categorising the urban context into clusters representative of Open and Compact arrangements with the seven subdivisions ranging from the low-rise to high-rise that assist in urban climate research and evaluate precinct urban energy budget and its effects on UHIs, wind patterns, and outdoor thermal comfort to address these gaps. The framework discusses the correlations between precinct morphological traits, environmental climatic conditions, UHIs, and outdoor thermal comfort. This study investigates the correlation between, precinct morphological traits and the heat island effect at the urban canopy layer. Besides, to discover the most appropriate scenarios under Sydney's climate setting, appropriate mitigation techniques are also investigated in the context of suggested urban categories. Empirical investigations were carried out throughout this dissertation in a region near the Bondi precinct that has a diversity of urban settlements. Envi-met, a realistic tool for modelling the distribution of the primary climatic elements in urban environments, is a three-dimensional Computational Fluid Dynamics (CFD) model that implements a thorough numerical simulation. Over three summer days, we evaluated urban designs concerning ambient air temperature, wind characteristics, heat intensity, and outdoor thermal comfort. In all 9 configurations, we connected the results to density and built-up ratio and discovered that the greatest configurational influence on the heat island was 2.33 °C. The average wind speed figure exhibits a significant decreasing trend in configurations with built-up ratios between 0.37 and 0.5, while a built-up ratio of 0.63 indicates the minimum. In high-rise compact layouts, the average temperature dropped by 1.12 °C per hour. In terms of mitigation strategies, when the albedo of the streets and pavements is increased between 0.12-0.36 and that of the pavements between 0.3 and 0.9, the scenario predicts to lower peak ambient temperature by 0.56 °C to 1.18 °C. However, in canyons surrounded by low-to medium-height structures that are rarely covered by other buildings, this approach raises the outdoor thermal comfort. The maximum ambient temperature is lowered by 0.47 to 1.35 °C when full mitigation techniques, including surface albedo adjustments and an increase in the proportion of outdoor vegetation, are used. The peak ambient temperature at ground level is not significantly lowered by mitigation strategies employing additional roof vegetation. Furthermore, a strong connection between building layout and mitigation strategies was defined. As that in terms of suitable mitigation strategies on the urban canopy layer results shows that a maximum cooling potential of 2.15°C in OT1 and a minimum of 0.07°C in CT7 relative to the reference scenario were anticipated by full mitigation scenarios. This study focuses on urban heat island impacts, urban layout cooling potential, and related mitigation techniques to solve the major problem confronting the built environment. The intricate interconnections between precinct geometry, precinct thermodynamic efficiency, urban heat island effects, and outdoor thermal comfort are addressed using a multidisciplinary approach. The need for UHI mitigation techniques for ecologically sustainable urban growth in Sydney and other Australian cities with similar climatic conditions should be made clear by these thorough findings. The findings of this study contribute to our understanding of how selective mitigation strategies affect UHI on the urban canopy layer and can be used by decision-makers to develop effective urban planning and development control plans to improve urban resilience, such as expanding urban density and growing with a poly-centric spatial form