Multi-level Feature Fusion-based CNN for Local Climate Zone Classification from Sentinel-2 Images: Benchmark Results on the So2Sat LCZ42 Dataset

As a unique classification scheme for urban forms and functions, the local climate zone (LCZ) system provides essential general information for any studies related to urban environments, especially on a large scale. Remote sensing data-based classification approaches are the key to large-scale mapping and monitoring of LCZs. The potential of deep learning-based approaches is not yet fully explored, even though advanced convolutional neural networks (CNNs) continue to push the frontiers for various computer vision tasks. One reason is that published studies are based on different datasets, usually at a regional scale, which makes it impossible to fairly and consistently compare the potential of different CNNs for real-world scenarios. This study is based on the big So2Sat LCZ42 benchmark dataset dedicated to LCZ classification. Using this dataset, we studied a range of CNNs of varying sizes. In addition, we proposed a CNN to classify LCZs from Sentinel-2 images, Sen2LCZ-Net. Using this base network, we propose fusing multi-level features using the extended Sen2LCZ-Net-MF. With this proposed simple network architecture and the highly competitive benchmark dataset, we obtain results that are better than those obtained by the state-of-the-art CNNs, while requiring less computation with fewer layers and parameters. Large-scale LCZ classification examples of completely unseen areas are presented, demonstrating the potential of our proposed Sen2LCZ-Net-MF as well as the So2Sat LCZ42 dataset. We also intensively investigated the influence of network depth and width and the effectiveness of the design choices made for Sen2LCZ-Net-MF. Our work will provide important baselines for future CNN-based algorithm developments for both LCZ classification and other urban land cover land use classification

Unlocking the capabilities of explainable fewshot learning in remote sensing

Recent advancements have significantly improved the efficiency and effectiveness of deep learning methods for imagebased remote sensing tasks. However, the requirement for large amounts of labeled data can limit the applicability of deep neural networks to existing remote sensing datasets. To overcome this challenge, fewshot learning has emerged as a valuable approach for enabling learning with limited data. While previous research has evaluated the effectiveness of fewshot learning methods on satellite based datasets, little attention has been paid to exploring the applications of these methods to datasets obtained from UAVs, which are increasingly used in remote sensing studies. In this review, we provide an up to date overview of both existing and newly proposed fewshot classification techniques, along with appropriate datasets that are used for both satellite based and UAV based data. Our systematic approach demonstrates that fewshot learning can effectively adapt to the broader and more diverse perspectives that UAVbased platforms can provide. We also evaluate some SOTA fewshot approaches on a UAV disaster scene classification dataset, yielding promising results. We emphasize the importance of integrating XAI techniques like attention maps and prototype analysis to increase the transparency, accountability, and trustworthiness of fewshot models for remote sensing. Key challenges and future research directions are identified, including tailored fewshot methods for UAVs, extending to unseen tasks like segmentation, and developing optimized XAI techniques suited for fewshot remote sensing problems. This review aims to provide researchers and practitioners with an improved understanding of fewshot learnings capabilities and limitations in remote sensing, while highlighting open problems to guide future progress in efficient, reliable, and interpretable fewshot methods.Comment: Under review, once the paper is accepted, the copyright will be transferred to the corresponding journa

Consistent metropolitan boundaries for the remote sensing of urban land

This paper introduces an internationally consistent definition of metropolitan areas to the literature regarding the remote sensing of urban land use or land cover. In the cross-comparison of land use or land cover for explicitly bounded urban areas, the observed ‘economic’ definition is argued to hold distinct potential merits over administrative or agglomeration-based boundaries, which typically underpin other studies. To illustrate the proposed merits as well as their implications for the remote sensing literature, the empirical analysis considers the case of 687 European metropolitan areas. Across these metropolitan areas, whose boundaries are defined jointly by the OECD and the European Commission, land cover and land use are segmented in a fusion of imagery from radar and optical sensors in Sentinel satellites. Segmentation is achieved using deep learning in a well-established model architecture. The analytical focus is on built-up areas that are in a residential use or in a commercial or industrial use. Map classifications and accuracy measures are obtained for cities as well as their respective commuting zones as these together embody metropolitan areas. The results underline that not only land use area estimates but also map classification accuracy vary widely across individual metropolitan areas. Whereas classification accuracy to some degree varies for metropolitan areas within as well as between countries, classification accuracy is positively associated with population size and built-up area density as regression analysis confirms. Additionally, the extent of built-up areas in distinct uses is shown to vary across different types of metropolitan (sub-)areas. This study's findings highlight the typically unobserved role that study area definition and selection may play in affecting outcomes in remote sensing studies in urban settings, as relevant to both studies of single as well as multiple urban areas. The consistent comparison of remote sensing outcomes across metropolitan areas may further promote generalization in a growing and global field and potentially supports better-informed policy making processes.</p

Interferometric Synthetic Aperture RADAR and Radargrammetry towards the Categorization of Building Changes

The purpose of this work is the investigation of SAR techniques relying on multi image acquisition for fully automatic and rapid change detection analysis at building level. In particular, the benefits and limitations of a complementary use of two specific SAR techniques, InSAR and radargrammetry, in an emergency context are examined in term of quickness, globality and accuracy. The analysis is performed using spaceborne SAR data

Classification of Polarimetric SAR Images Using Compact Convolutional Neural Networks

Classification of polarimetric synthetic aperture radar (PolSAR) images is an active research area with a major role in environmental applications. The traditional Machine Learning (ML) methods proposed in this domain generally focus on utilizing highly discriminative features to improve the classification performance, but this task is complicated by the well-known "curse of dimensionality" phenomena. Other approaches based on deep Convolutional Neural Networks (CNNs) have certain limitations and drawbacks, such as high computational complexity, an unfeasibly large training set with ground-truth labels, and special hardware requirements. In this work, to address the limitations of traditional ML and deep CNN based methods, a novel and systematic classification framework is proposed for the classification of PolSAR images, based on a compact and adaptive implementation of CNNs using a sliding-window classification approach. The proposed approach has three advantages. First, there is no requirement for an extensive feature extraction process. Second, it is computationally efficient due to utilized compact configurations. In particular, the proposed compact and adaptive CNN model is designed to achieve the maximum classification accuracy with minimum training and computational complexity. This is of considerable importance considering the high costs involved in labelling in PolSAR classification. Finally, the proposed approach can perform classification using smaller window sizes than deep CNNs. Experimental evaluations have been performed over the most commonly-used four benchmark PolSAR images: AIRSAR L-Band and RADARSAT-2 C-Band data of San Francisco Bay and Flevoland areas. Accordingly, the best obtained overall accuracies range between 92.33 - 99.39% for these benchmark study sites

Continental-scale land cover mapping at 10 m resolution over Europe (ELC10)

Widely used European land cover maps such as CORINE are produced at medium spatial resolutions (100 m) and rely on diverse data with complex workflows requiring significant institutional capacity. We present a high resolution (10 m) land cover map (ELC10) of Europe based on a satellite-driven machine learning workflow that is annually updatable. A Random Forest classification model was trained on 70K ground-truth points from the LUCAS (Land Use/Cover Area frame Survey) dataset. Within the Google Earth Engine cloud computing environment, the ELC10 map can be generated from approx. 700 TB of Sentinel imagery within approx. 4 days from a single research user account. The map achieved an overall accuracy of 90% across 8 land cover classes and could account for statistical unit land cover proportions within 3.9% (R2 = 0.83) of the actual value. These accuracies are higher than that of CORINE (100 m) and other 10-m land cover maps including S2GLC and FROM-GLC10. We found that atmospheric correction of Sentinel-2 and speckle filtering of Sentinel-1 imagery had minimal effect on enhancing classification accuracy (< 1%). However, combining optical and radar imagery increased accuracy by 3% compared to Sentinel-2 alone and by 10% compared to Sentinel-1 alone. The conversion of LUCAS points into homogenous polygons under the Copernicus module increased accuracy by <1%, revealing that Random Forests are robust against contaminated training data. Furthermore, the model requires very little training data to achieve moderate accuracies - the difference between 5K and 50K LUCAS points is only 3% (86 vs 89%). At 10-m resolution, the ELC10 map can distinguish detailed landscape features like hedgerows and gardens, and therefore holds potential for aerial statistics at the city borough level and monitoring property-level environmental interventions (e.g. tree planting)

Estimating aboveground woody biomass change in Kalahari woodland: combining field, radar, and optical data sets

Maps that accurately quantify aboveground vegetation biomass (AGB) are essential for ecosystem monitoring and conservation. Throughout Namibia, four vegetation change processes are widespread, namely, deforestation, woodland degradation, the encroachment of the herbaceous and grassy layers by woody strata (woody thickening), and woodland regrowth. All of these vegetation change processes affect a range of key ecosystem services, yet their spatial and temporal dynamics and contributions to AGB change remain poorly understood. This study quantifies AGB associated with the different vegetation change processes over an 8-year period, for a region of Kalahari woodland savannah in northern Namibia. Using data from 101 forest inventory plots collected during two field campaigns (2014–2015), we model AGB as a function of the Advanced Land Observing Satellite Phased Array L-band synthetic aperture radar (PALSAR and PALSAR-2) and dry season Landsat vegetation index composites, for two periods (2007 and 2015). Differences in AGB between 2007 and 2015 were assessed and validated using independent data, and changes in AGB for the main vegetation processes are quantified for the whole study area (75,501 km2). We find that woodland degradation and woody thickening contributed a change in AGB of −14.3 and 2.5 Tg over 14% and 3.5% of the study area, respectively. Deforestation and regrowth contributed a smaller portion of AGB change, i.e. −1.9 and 0.2 Tg over 1.3% and 0.2% of the study area, respectively

Historical forest biomass dynamics modelled with Landsat spectral trajectories

Acknowledgements National Forest Inventory data are available online, provided by Ministerio de Agricultura, Alimentación y Medio Ambiente (España). Landsat images are available online, provided by the USGS.Peer reviewedPostprin

Mapping intra- and inter-annual dynamics in wetlands with multispectral, thermal and SAR time series

Kartierung der intra- und interannuellen Dynamik von Feuchtgebieten mit multispektralen, thermischen und SAR-Zeitreihen Die Analyse der aktuellen räumlichen Verbreitung und der zeitlichen Entwicklung von Feuchtgebieten stellt eine äußerst komplexe Aufgabe dar, welche durch die Saisonalität, die schwierige Zugänglichkeit und die besonderen Eigenschaften als Ökoton bedingt ist. Erdbeobachtungssysteme sind somit das am besten geeignete Werkzeug, um zeitliche und räumliche Muster von Feuchtgebieten auf globaler Ebene zu beobachten (saisonale Veränderungen und Langzeit-Trends) und um den Einfluss der menschlichen Aktivitäten auf ihre physischen und biologischen Eigenschaften zu untersuchen. Zur Kartierung von raum-zeitlichen Mustern wurden Zeitreihen von Radar- (Sentinel-1), Multispektral- (Sentinel-2) und Thermal-Satellitendaten (MODIS) in fünf Untersuchungsgebieten, mit für Feuchtgebiete unterschiedlichen typischen Charakteristika, untersucht. In Kapitel 1 werden die Problematik in Bezug auf die Definition von Feuchtgebieten erläutert und allgemeine Degradations-Trends beschrieben. Die Kapitel 2 und 3 behandeln einen Algorithmus, der Veränderungen mithilfe von SAR-Zeitreihen feststellt, sowie die Vorteile des Cloud-Computings für das operationelle Monitoring saisonaler Muster und die Erkennung kurzfristig auftretender Veränderungen. In den Kapiteln 4 und 5 werden die zwei Hauptursachen für den Verlust von Feuchtgebieten betrachtet: der Staudammbau und die Ausdehnung landwirtschaftlicher Flächen. In Kapitel 4 werden dichte Zeitreihen multispektraler (Sentinel-2) und SAR-Daten (Sentinel-1) verwendet, um die Feuchtgebiete Albaniens – eines Landes in dem konträre Pläne zum Ausbau seines Wasserkraftpotentials und dem Schutz intakter Flussökosysteme zu Spannungen führen – landesweit zu kartieren. Die synergetischen Vorteile, die sich durch die Fusionierung von multispektralen und SAR-Daten für die Klassifikation ergeben, werden dabei herausgestellt. Kapitel 5 veranschaulicht, dass die Kilombero-Überschwemmungsebene in Tansania ein großes und bedeutendes Feuchtgebiet ist, das in den vergangenen Jahren infolge der weitgehend unkontrollierten Ausbreitung landwirtschaftlicher Flächen in seiner Ausdehnung und seiner Ökologie stark beeinträchtigt wurde. Um die Auswirkungen der Landnutzungsänderungen des Feuchtgebietes während der vergangenen 18 Jahre zu analysieren, wurden eine Zeitreihe (2000 bis 2017) thermaler Daten (MODIS) analysiert. Die drei für die Zeitreihenanalyse angewandten Modelle zeigen, wie landwirtschaftliche Praktiken die Landoberflächentemperatur in den landwirtschaftlich genutzten Gebieten sowie in den angrenzenden natürlichen Feuchtgebieten erhöht haben.Due to wetlands’ seasonality, their difficult access and ecotone character, determining their actual extension and trends over time is a complex task. Earth Observation systems are the most appropriate tool to monitor their spatio-temporal patterns (seasonal changes and long term trends) at global scales, and to study the effects that human activities have in their physical and biological properties. In this work I use time series of radar (Sentinel-1), multispectral (Sentinel-2) and thermal (MODIS) imagery to map the spatio-temporal patterns in 5 wetlands of different characteristics. First, I introduce in chapter 1 the problematic of wetlands’ definitions and their degradation trends. I continue with a brief introduction on remote sensing, time series analysis, and their applications on wetlands’ research and management. In chapters 2 and 3 I implement an algorithm for change detection of time series of Sentinel-1 images and demonstrate the advantages of cloud computation for operational monitoring. In chapters 4 and 5 I address two of the main causes of wetland degradation: dam building and agricultural expansion. In chapter 4 I use dense time series of Sentinel-1 and Sentinel-2 images map all the wetlands of Albania; a country struggling between developing its large hydropower potential or preserving its intact and valuable river ecosystems. I evaluate the synergic advantages of fusing multispectral and radar imagery in combination with knowledge-based rules to produce classification of higher thematic and spatial resolutions. In chapter 5 I present how the Kilombero Floodplain, in Tanzania, has been degraded during the last years due to uncontrolled farmland expansion. I use a time series of thermal imagery (MODIS) from 2000 until 2017 to analyze the effect of land use changes on the wetland. I compare three models for time series analysis and reveal how farming practices have increased the surface temperature of the farmed area, as well as in adjacent natural wetlands.Mapeo de las dinámicas inter- e intra-anuales en humedales con series temporales de imágenes multiespectrales, termales y de radar Debido a la estacionalidad de los humedales, su difícil acceso y sus características de ecotono, determinar su actual extensión y sus tendencias a lo largo del tiempo es una tarea compleja. Los sistemas de observación terrestres son la herramienta más apropiada para monitorear sus patrones espacio-temporales (estacionalidad y tendencias a largo plazo) a escalas globales, y para estudiar los efectos que las actividades humanas causan en sus propiedades físicas y biológicas. En esta tesis uso series temporales de imágenes radar (Sentinel-1), multiespectrales (Sentinel-2) y termales (MODIS) para mapear los patrones espacio-temporales de 5 humedales de diferentes características. En el capítulo 1 describo los retos que derivan de las diferentes definiciones que existen de los humedales. También presento las tendencias globales de degradación que la mayoría de los humedales continúan experimentando en los últimos años. Continúo con una breve introducción de los sistemas de teledetección remota, análisis de series temporales, y sus aplicaciones a la investigación y gestión de los humedales. En los capítulos 2 y 3 implemento un algoritmo de detección de cambios para series temporales de imágenes radar, y muestro las ventajas de usar sistemas de computación en la nube para monitorear cambios en la cobertura del suelo a corto plazo. En los capítulos 4 y 5 trato con dos de las causas más comunes de degradación de humedales: la construcción de presas y la expansión de la agricultura. En el capítulo 4 uso series temporales de imágenes multiespectrales (Sentinel-2) y radar (Sentinel-1) para mapear todos los humedales Albania; un país que se debate entre desarrollar su potencial hidroenergético o preservar sus valiosos e intactos ecosistemas de rivera. Mediante la fusión de imágenes radar y multiespectrales y el uso de reglas de decisión genero un mapa de suficiente resolución espacial y temática para que pueda ser usado por sectores interesados y gestores. En el capítulo 5 presento como las llanuras inundables de Kilombero, en Tanzania, han sido degradadas durante los últimos años debido a la expansión incontrolada de la agricultura. Usando series temporales de imágenes termales (MODIS) desde 2000 hasta 2017 y mapas de cambios de usos del suelo, determino los efectos que estos cambios han tenido en el humedal. Comparo 3 modelos diferentes de análisis de series temporales y muestro cómo la expansión de la agricultura ha incrementado la temperatura superficial terrestre, no solo de la zona cultivada, sino también de zonas adyacentes aún naturales

Remote sensing of geomorphodiversity linked to biodiversity — part III: traits, processes and remote sensing characteristics

Remote sensing (RS) enables a cost-effective, extensive, continuous and standardized monitoring of traits and trait variations of geomorphology and its processes, from the local to the continental scale. To implement and better understand RS techniques and the spectral indicators derived from them in the monitoring of geomorphology, this paper presents a new perspective for the definition and recording of five characteristics of geomorphodiversity with RS, namely: geomorphic genesis diversity, geomorphic trait diversity, geomorphic structural diversity, geomorphic taxonomic diversity, and geomorphic functional diversity. In this respect, geomorphic trait diversity is the cornerstone and is essential for recording the other four characteristics using RS technologies. All five characteristics are discussed in detail in this paper and reinforced with numerous examples from various RS technologies. Methods for classifying the five characteristics of geomorphodiversity using RS, as well as the constraints of monitoring the diversity of geomorphology using RS, are discussed. RS-aided techniques that can be used for monitoring geomorphodiversity in regimes with changing land-use intensity are presented. Further, new approaches of geomorphic traits that enable the monitoring of geomorphodiversity through the valorisation of RS data from multiple missions are discussed as well as the ecosystem integrity approach. Likewise, the approach of monitoring the five characteristics of geomorphodiversity recording with RS is discussed, as are existing approaches for recording spectral geomorhic traits/ trait variation approach and indicators, along with approaches for assessing geomorphodiversity. It is shown that there is no comparable approach with which to define and record the five characteristics of geomorphodiversity using only RS data in the literature. Finally, the importance of the digitization process and the use of data science for research in the field of geomorphology in the 21st century is elucidated and discussed