Feature enhancement network for cloud removal in optical images by fusing with SAR images

Presence of cloud-covered pixels is inevitable in optical remote-sensing images. Therefore, the reconstruction of the cloud-covered details is important to improve the usage of these images for subsequent image analysis tasks. Aiming to tackle the issue of high computational resource requirements that hinder the application at scale, this paper proposes a Feature Enhancement Network(FENet) for removing clouds in satellite images by fusing Synthetic Aperture Radar (SAR) and optical images. The proposed network consists of designed Feature Aggregation Residual Block (FAResblock) and Feature Enhancement Block (FEBlock). FENet is evaluated on the publicly available SEN12MS-CR dataset and it achieves promising results compared to the benchmark and the state-of-the-art methods in terms of both visual quality and quantitative evaluation metrics. It proved that the proposed feature enhancement network is an effective solution for satellite image cloud removal using less computational and time consumption. The proposed network has the potential for practical applications in the field of remote sensing due to its effectiveness and efficiency. The developed code and trained model will be available at https://github.com/chenxiduan/FENet.</p

Weakly Supervised Learning for Multi-Image Synthesis

Machine learning-based approaches have been achieving state-of-the-art results on many computer vision tasks. While deep learning and convolutional networks have been incredibly popular, these approaches come at the expense of huge amounts of labeled data required for training. Manually annotating large amounts of data, often millions of images in a single dataset, is costly and time consuming. To deal with the problem of data annotation, the research community has been exploring approaches that require less amount of labelled data. The central problem that we consider in this research is image synthesis without any manual labeling. Image synthesis is a classic computer vision task that requires understanding of image contents and their semantic and geometric properties. We propose that we can train image synthesis models by relying on sequences of videos and using weakly supervised learning. Large amounts of unlabeled data are freely available on the internet. We propose to set up the training in a multi-image setting so that we can use one of the images as the target - this allows us to rely only on images for training and removes the need for manual annotations. We demonstrate three main contributions in this work. First, we present a method of fusing multiple noisy overhead images to make a single, artifact-free image. We present a weakly supervised method that relies on crowd-sourced labels from online maps and a completely unsupervised variant that only requires a series of satellite images as inputs. Second, we propose a single-image novel view synthesis method for complex, outdoor scenes. We propose a learning-based method that uses pairs of nearby images captured on urban roads and their respective GPS coordinates as supervision. We show that a model trained with this automatically captured data can render a new view of a scene that can be as far as 10 meters from the input image. Third, we consider the problem of synthesizing new images of a scene under different conditions, such as time of day and season, based on a single input image. As opposed to existing methods, we do not need manual annotations for transient attributes, such as fog or snow, for training. We train our model by using streams of images captured from outdoor webcams and time-lapse videos. Through these applications, we show several settings where we can train state-of-the-art deep learning methods without manual annotations. This work focuses on three image synthesis tasks. We propose weakly supervised learning and remove requirements for manual annotations by relying on sequences of images. Our approach is in line with the research efforts that aim to minimize the labels required for training machine learning methods

New Approaches to Mapping Forest Conditions and Landscape Change from Moderate Resolution Remote Sensing Data across the Species-Rich and Structurally Diverse Atlantic Northern Forest of Northeastern North America

The sustainable management of forest landscapes requires an understanding of the functional relationships between management practices, changes in landscape conditions, and ecological response. This presents a substantial need of spatial information in support of both applied research and adaptive management. Satellite remote sensing has the potential to address much of this need, but forest conditions and patterns of change remain difficult to synthesize over large areas and long time periods. Compounding this problem is error in forest attribute maps and consequent uncertainty in subsequent analyses. The research described in this document is directed at these long-standing problems. Chapter 1 demonstrates a generalizable approach to the characterization of predominant patterns of forest landscape change. Within a ~1.5 Mha northwest Maine study area, a time series of satellite-derived forest harvest maps (1973-2010) served as the basis grouping landscape units according to time series of cumulative harvest area. Different groups reflected different harvest histories, which were linked to changes in landscape composition and configuration through time series of selected landscape metrics. Time series data resolved differences in landscape change attributable to passage of the Maine Forest Practices Act, a major change in forest policy. Our approach should be of value in supporting empirical landscape research. Perhaps the single most important source of uncertainty in the characterization of landscape conditions is over- or under-representation of class prevalence caused by prediction bias. Systematic error is similarly impactful in maps of continuous forest attributes, where regression dilution or attenuation bias causes the overestimation of low values and underestimation of high values. In both cases, patterns of error tend to produce more homogeneous characterizations of landscape conditions. Chapters 2 and 3 present a machine learning method designed to simultaneously reduce systematic and total error in continuous and categorical maps, respectively. By training support vector machines with a multi-objective genetic algorithm, attenuation bias was substantially reduced in regression models of tree species relative abundance (chapter 2), and prediction bias was effectively removed from classification models predicting tree species occurrence and forest disturbance (chapter 3). This approach is generalizable to other prediction problems, other regions, or other geospatial disciplines

Landscape functional connectivity and animal movement: application of remote sensing for increasing efficiency of road mitigation measures

Roads are a major threat to wildlife due to induced mortality and restrictions to animal movement. A central issue in conservation biology is the accurate site identification for the implementation of multispecies mitigation measures, on roads. Those measures entail high costs and methodological challenges and their efficiency highly depend on the right location. The aim of this PhD is to inform, through remote sensing and connectivity modelling, how to increase the efficiency of planning mitigation measures to reduce roadkill and promote connectivity; and demonstrate the usefulness of remote sensing in defining suitable areas for the conservation of an endangered species that often occurs in the vicinity of roads. To do so, we first assessed whether occurrence-based strategies were able to infer functional connectivity, compared to those more complex and financially demanding based on telemetry, with respect to daily and dispersal movements. Secondly, we assessed whether remote sensing data were sufficiently informative to identify key habitats for a threatened species around road verges. Thirdly, we assessed the predictive and prioritisation ability of road mitigation units intercepting multispecies corridors to prevent vulnerability to roadkill. Findings revealed that simple models are suitable as complex ones for both daily and dispersal movements, allowing for costly-effective connectivity assessments. Results demonstrated the ability of free remote sensing data to identify microhabitat conditions in verges and surrounding landscape, for a threatened rodent, allowing for the delimitation of refugee areas and definition of monitoring strategies for the species. Undemanding data (occurrence and remote sensing) were able to describe species-specific ecological requirements for birds, bats and non-flying mammals as well as roadkill patterns, possibly due to similar overlapping corridors and habitats, despite some mismatches that occurred for highly mobile species. This framework ensured high efficiency in prioritisation of multispecies roadkill mitigation planning, resilient to long-term landscape dynamics; Conectividade funcional da paisagem e movimento animal: aplicação da detecção remota para aumentar a eficiência de medidas de mitigação em estradas. Resumo: As estradas constituem uma enorme ameaça para a vida selvagem devido à mortalidade. Uma questão central é a identificação dos locais para implementar medidas de mitigação multiespécies, em estradas. Essas medidas envolvem custos elevados e desafios metodológicos e sua eficiência depende muito da localização correcta. O objetivo deste doutoramento é informar, através de detecção remota e conectividade, como aumentar a eficiência do planeamento de medidas de mitigação para reduzir atropelamentos e promover a conectividade; e demonstrar a utilidade da detecção remota na definição de áreas adequadas para a conservação de espécies ameaçadas que podem ocorrer nas proximidades de estradas. Portanto, primeiro avaliamos se os dados resultantes de amostragens simples eram capazes de inferir conectividade funcional, em comparação com estratégias complexas, respeito aos movimentos diários e de dispersão. Segundo, avaliamos se os dados de detecção remota eram suficientemente informativos para identificar habitats-chave para uma espécie ameaçada em torno das margens das estradas. Terceiro, avaliamos a capacidade preditiva e de prioritização das unidades de mitigação de estradas que cruzam corredores multi-espécies para reduzir o risco de atropelamentos. Os resultados revelaram que os modelos simples são adequados quanto os complexos para os movimentos diários e de dispersão. Os resultados demonstraram a capacidade dos dados de detecção remota gratuitos em identificar condições de microhabitats nos habitats de berma e na paisagem circundante, para um roedor ameaçado, permitindo a delimitação de áreas de refúgio. Dados pouco exigentes (ocorrência e detecção remota) foram capazes de descrever os requisitos ecológicos específicos de aves, morcegos e mamíferos não voadores, bem como padrões de atropelamentos, possivelmente devido a corredores e habitats semelhantes, apesar de haver algumas incompatibilidades para espécies de maior mobilidade. Essa estrutura foi capaz de garantir uma elevada eficiência na prioritização de planeamento de mitigação de atropelamentos para multi-espécies, resiliente à dinâmica da paisagem de longo prazo

Geoscience-aware deep learning:A new paradigm for remote sensing

Information extraction is a key activity for remote sensing images. A common distinction exists between knowledge-driven and data-driven methods. Knowledge-driven methods have advanced reasoning ability and interpretability, but have difficulty in handling complicated tasks since prior knowledge is usually limited when facing the highly complex spatial patterns and geoscience phenomena found in reality. Data-driven models, especially those emerging in machine learning (ML) and deep learning (DL), have achieved substantial progress in geoscience and remote sensing applications. Although DL models have powerful feature learning and representation capabilities, traditional DL has inherent problems including working as a black box and generally requiring a large number of labeled training data. The focus of this paper is on methods that integrate domain knowledge, such as geoscience knowledge and geoscience features (GK/GFs), into the design of DL models. The paper introduces the new paradigm of geoscience-aware deep learning (GADL), in which GK/GFs and DL models are combined deeply to extract information from remote sensing data. It first provides a comprehensive summary of GK/GFs used in GADL, which forms the basis for subsequent integration of GK/GFs with DL models. This is followed by a taxonomy of approaches for integrating GK/GFs with DL models. Several approaches are detailed using illustrative examples. Challenges and research prospects in GADL are then discussed. Developing more novel and advanced methods in GADL is expected to become the prevailing trend in advancing remotely sensed information extraction in the future.</p

The recent developments in cloud removal approaches of MODIS snow cover product

The snow cover products of optical remote sensing systems play an important role in research into global climate change, the hydrological cycle, and the energy balance. Moderate Resolution Imaging Spectroradiometer (MODIS) snow cover products are the most popular datasets used in the community. However, for MODIS, cloud cover results in spatial and temporal discontinuity for long-term snow monitoring. In the last few decades, a large number of cloud removal methods for MODIS snow cover products have been proposed. In this paper, our goal is to make a comprehensive summarization of the existing algorithms for generating cloud-free MODIS snow cover products and to expose the development trends. The methods of generating cloud-free MODIS snow cover products are classified into spatial methods, temporal methods, spatio-temporal methods, and multi-source fusion methods. The spatial methods and temporal methods remove the cloud cover of the snow product based on the spatial patterns and temporal changing correlation of the snowpack, respectively. The spatio-temporal methods utilize the spatial and temporal features of snow jointly. The multi-source fusion methods utilize the complementary information among different sources among optical observations, microwave observations, and station observations.</p

Deep Learning based data-fusion methods for remote sensing applications

In the last years, an increasing number of remote sensing sensors have been launched to orbit around the Earth, with a continuously growing production of massive data, that are useful for a large number of monitoring applications, especially for the monitoring task. Despite modern optical sensors provide rich spectral information about Earth's surface, at very high resolution, they are weather-sensitive. On the other hand, SAR images are always available also in presence of clouds and are almost weather-insensitive, as well as daynight available, but they do not provide a rich spectral information and are severely affected by speckle "noise" that make difficult the information extraction. For the above reasons it is worth and challenging to fuse data provided by different sources and/or acquired at different times, in order to leverage on their diversity and complementarity to retrieve the target information. Motivated by the success of the employment of Deep Learning methods in many image processing tasks, in this thesis it has been faced different typical remote sensing data-fusion problems by means of suitably designed Convolutional Neural Networks

A review of technical factors to consider when designing neural networks for semantic segmentation of Earth Observation imagery

Semantic segmentation (classification) of Earth Observation imagery is a crucial task in remote sensing. This paper presents a comprehensive review of technical factors to consider when designing neural networks for this purpose. The review focuses on Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Generative Adversarial Networks (GANs), and transformer models, discussing prominent design patterns for these ANN families and their implications for semantic segmentation. Common pre-processing techniques for ensuring optimal data preparation are also covered. These include methods for image normalization and chipping, as well as strategies for addressing data imbalance in training samples, and techniques for overcoming limited data, including augmentation techniques, transfer learning, and domain adaptation. By encompassing both the technical aspects of neural network design and the data-related considerations, this review provides researchers and practitioners with a comprehensive and up-to-date understanding of the factors involved in designing effective neural networks for semantic segmentation of Earth Observation imagery.Comment: 145 pages with 32 figure

Using New and Long-Term Multi-Scale Remotely Sensed Data to Detect Recurrent Fires and Quantify Their Relationship to Land Cover/Use in Indonesian Peatlands

Indonesia has committed to reducing its greenhouse gases emissions by 29% (potentially up to 41% with international assistance) by 2030. Achieving those targets requires many efforts but, in particular, controlling the fire problem in Indonesia’s peatlands is paramount, since it is unlikely to diminish on its own in the coming decades. This study was conducted in Sumatra and Kalimantan peatlands in Indonesia. Four MODIS-derived products (MCD45A1 collection 5.1, MCD64A1 (collection 5.1 and 6), FireCCI51) were initially assessed to explore long-term fire frequency and land use/cover change relationships. The results indicated the product(s) could only detect half of the fires accurately. A further study was conducted using additional moderate spatial resolution data to compare two years of different severity (2014 and 2015) (Landsat, Sentinel 2, Sentinel 1, VIIRS 375 m). The results showed that MODIS BA products poorly discriminated small fires and failed to detect many burned areas due to persistent interference from clouds and smoke that often worsens as fire seasons progress. Although there are unique fire detection capabilities associated with each sensor (MODIS, VIIRS, Landsat, Sentinel 2, Sentinel 1), no single sensor was ideal for accurate detection of peatland fires under all conditions. Multisensor approaches could advance biomass-burning detection in peatlands, improving the accuracy and comprehensive coverage of burned area maps, thereby enabling better estimation of associated fire emissions. Despite missing many burned areas, MODIS BA (MCD64A1 C6) provides the best available data for evaluating longer term (2001-2018) associations between the frequency of fire occurrence and land use/cover change across large areas. Results showed that Sumatra and Kalimantan have both experienced frequent fires since 2001. Although extensive burning was present across the entire landscape, burning in peatlands was ~5- times more frequent and strongly associated with changes of forest to other land use/cover classes. If fire frequencies since 2001 remain unchanged, remnant peat swamp forests of Sumatra and Kalimantan will likely disappear over the next few decades. The findings reported in this dissertation provide critical insights for Indonesian stakeholders that can help them to minimize impacts of environmental change, manage ecological restoration efforts, and improve fire monitoring systems within Indonesia