Multisource and Multitemporal Data Fusion in Remote Sensing

The sharp and recent increase in the availability of data captured by different sensors combined with their considerably heterogeneous natures poses a serious challenge for the effective and efficient processing of remotely sensed data. Such an increase in remote sensing and ancillary datasets, however, opens up the possibility of utilizing multimodal datasets in a joint manner to further improve the performance of the processing approaches with respect to the application at hand. Multisource data fusion has, therefore, received enormous attention from researchers worldwide for a wide variety of applications. Moreover, thanks to the revisit capability of several spaceborne sensors, the integration of the temporal information with the spatial and/or spectral/backscattering information of the remotely sensed data is possible and helps to move from a representation of 2D/3D data to 4D data structures, where the time variable adds new information as well as challenges for the information extraction algorithms. There are a huge number of research works dedicated to multisource and multitemporal data fusion, but the methods for the fusion of different modalities have expanded in different paths according to each research community. This paper brings together the advances of multisource and multitemporal data fusion approaches with respect to different research communities and provides a thorough and discipline-specific starting point for researchers at different levels (i.e., students, researchers, and senior researchers) willing to conduct novel investigations on this challenging topic by supplying sufficient detail and references

Multisource and multitemporal data fusion in remote sensing:A comprehensive review of the state of the art

The recent, sharp increase in the availability of data captured by different sensors, combined with their considerable heterogeneity, poses a serious challenge for the effective and efficient processing of remotely sensed data. Such an increase in remote sensing and ancillary data sets, however, opens up the possibility of utilizing multimodal data sets in a joint manner to further improve the performance of the processing approaches with respect to applications at hand. Multisource data fusion has, therefore, received enormous attention from researchers worldwide for a wide variety of applications. Moreover, thanks to the revisit capability of several

Spatiotemporal subpixel mapping of time-series images

Land cover/land use (LCLU) information extraction from multitemporal sequences of remote sensing imagery is becoming increasingly important. Mixed pixels are a common problem in Landsat and MODIS images that are used widely for LCLU monitoring. Recently developed subpixel mapping (SPM) techniques can extract LCLU information at the subpixel level by dividing mixed pixels into subpixels to which hard classes are then allocated. However, SPM has rarely been studied for time-series images (TSIs). In this paper, a spatiotemporal SPM approach was proposed for SPM of TSIs. In contrast to conventional spatial dependence-based SPM methods, the proposed approach considers simultaneously spatial and temporal dependences, with the former considering the correlation of subpixel classes within each image and the latter considering the correlation of subpixel classes between images in a temporal sequence. The proposed approach was developed assuming the availability of one fine spatial resolution map which exists among the TSIs. The SPM of TSIs is formulated as a constrained optimization problem. Under the coherence constraint imposed by the coarse LCLU proportions, the objective is to maximize the spatiotemporal dependence, which is defined by blending both spatial and temporal dependences. Experiments on three data sets showed that the proposed approach can provide more accurate subpixel resolution TSIs than conventional SPM methods. The SPM results obtained from the TSIs provide an excellent opportunity for LCLU dynamic monitoring and change detection at a finer spatial resolution than the available coarse spatial resolution TSIs

A Hybrid Particle Swarm Optimization with Affine Transformation Approach for Cloud Free Multi-Temporal Image Registration

An image registration is the major part of the image categorization and cluster formation in multi temporal image processing. The images are affected by the different factors such as cloud shadow, water level, building shadows etc. In this paper, an enhanced registration process and the cloud removal technique is proposed for image enhancement. The Daemons, Combined Registration and Segmentation (CRS) approach, Markov Random Field (MRF) approach and Mutual Information (MI) based approaches results in more computational complexity, minimum edge preservation measure (QAB/F) and Mutual Information in image registration. In order to maximize the quality of edge preservation measure and MI with minimum computational time, this paper proposes Particle Swarm Optimization (PSO) based affine transformation technique. The proposed techniques measure and compare the computation time against the number of pixels of an image with the existing methods of CRS and MRF for the number of images. The comparative analysis of QAB/F and MI with the traditional methods of Clock Point -Least Square (CP-LS) and the Multi-Focus Image Fusion (MFIF) and Discrete Wavelet Transform (DWT) is presented to confirm the effective performance. The simulation results of the proposed transformation for registration process confirms the effective image registration in the multi-temporal image processing

Hierarchical Markov random fields for high resolution land cover classification of multisensor and multiresolution image time series

International audienc

A review of technical factors to consider when designing neural networks for semantic segmentation of Earth Observation imagery

Semantic segmentation (classification) of Earth Observation imagery is a crucial task in remote sensing. This paper presents a comprehensive review of technical factors to consider when designing neural networks for this purpose. The review focuses on Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Generative Adversarial Networks (GANs), and transformer models, discussing prominent design patterns for these ANN families and their implications for semantic segmentation. Common pre-processing techniques for ensuring optimal data preparation are also covered. These include methods for image normalization and chipping, as well as strategies for addressing data imbalance in training samples, and techniques for overcoming limited data, including augmentation techniques, transfer learning, and domain adaptation. By encompassing both the technical aspects of neural network design and the data-related considerations, this review provides researchers and practitioners with a comprehensive and up-to-date understanding of the factors involved in designing effective neural networks for semantic segmentation of Earth Observation imagery.Comment: 145 pages with 32 figure

Active Learning: Any Value for Classification of Remotely Sensed Data?

Active learning, which has a strong impact on processing data prior to the classification phase, is an active research area within the machine learning community, and is now being extended for remote sensing applications. To be effective, classification must rely on the most informative pixels, while the training set should be as compact as possible. Active learning heuristics provide capability to select unlabeled data that are the “most informative” and to obtain the respective labels, contributing to both goals. Characteristics of remotely sensed image data provide both challenges and opportunities to exploit the potential advantages of active learning. We present an overview of active learning methods, then review the latest techniques proposed to cope with the problem of interactive sampling of training pixels for classification of remotely sensed data with support vector machines (SVMs). We discuss remote sensing specific approaches dealing with multisource and spatially and time-varying data, and provide examples for high-dimensional hyperspectral imagery

Monitoring wetlands and water bodies in semi-arid Sub-Saharan regions

Surface water in wetlands is a critical resource in semi-arid West-African regions that are frequently exposed to droughts. Wetlands are of utmost importance for the population as well as the environment, and are subject to rapidly changing seasonal fluctuations. Dynamics of wetlands in the study area are still poorly understood, and the potential of remote sensing-derived information as a large-scale, multi-temporal, comparable and independent measurement source is not exploited. This work shows successful wetland monitoring with remote sensing in savannah and Sahel regions in Burkina Faso, focusing on the main study site Lac Bam (Lake Bam). Long-term optical time series from MODIS with medium spatial resolution (MR), and short-term synthetic aperture radar (SAR) time series from TerraSAR-X and RADARSAT-2 with high spatial resolution (HR) successfully demonstrate the classification and dynamic monitoring of relevant wetland features, e.g. open water, flooded vegetation and irrigated cultivation. Methodological highlights are time series analysis, e.g. spatio-temporal dynamics or multitemporal-classification, as well as polarimetric SAR (polSAR) processing, i.e. the Kennaugh elements, enabling physical interpretation of SAR scattering mechanisms for dual-polarized data. A multi-sensor and multi-frequency SAR data combination provides added value, and reveals that dual-co-pol SAR data is most recommended for monitoring wetlands of this type. The interpretation of environmental or man-made processes such as water areas spreading out further but retreating or evaporating faster, co-occurrence of droughts with surface water and vegetation anomalies, expansion of irrigated agriculture or new dam building, can be detected with MR optical and HR SAR time series. To capture long-term impacts of water extraction, sedimentation and climate change on wetlands, remote sensing solutions are available, and would have great potential to contribute to water management in Africa

Utilizing the Landsat spectral-temporal domain for improved mapping and monitoring of ecosystem state and dynamics

Just as the carbon dioxide observations that form the Keeling curve revolutionized the study of the global carbon cycle, free and open access to all available Landsat imagery is fundamentally changing how the Landsat record is being used to study ecosystems and ecological dynamics. This dissertation advances the use of Landsat time series for visualization, classification, and detection of changes in terrestrial ecological processes. More specifically, it includes new examples of how complex ecological patterns manifest in time series of Landsat observations, as well as novel approaches for detecting and quantifying these patterns. Exploration of the complexity of spectral-temporal patterns in the Landsat record reveals both seasonal variability and longer-term trajectories difficult to characterize using conventional bi-temporal or even annual observations. These examples provide empirical evidence of hypothetical ecosystem response functions proposed by Kennedy et al. (2014). Quantifying observed seasonal and phenological differences in the spectral reflectance of Massachusetts’ forest communities by combining existing harmonic curve fitting and phenology detection algorithms produces stable feature sets that consistently out-performed more traditional approaches for detailed forest type classification. This study addresses the current lack of species-level forest data at Landsat resolutions, demonstrating the advantages of spectral-temporal features as classification inputs. Development of a targeted change detection method using transformations of time series data improves spatial and temporal information on the occurrence of flood events in landscapes actively modified by recovering North American beaver (Castor canadensis) populations. These results indicate the utility of the Landsat record for the study of species-habitat relationships, even in complex wetland environments. Overall, this dissertation confirms the value of the Landsat archive as a continuous record of terrestrial ecosystem state and dynamics. Given the global coverage of remote sensing datasets, the time series visualization and analysis approaches presented here can be extended to other areas. These approaches will also be improved by more frequent collection of moderate resolution imagery, as planned by the Landsat and Sentinel-2 programs. In the modern era of global environmental change, use of the Landsat spectral-temporal domain presents new and exciting opportunities for the long-term large-scale study of ecosystem extent, composition, condition, and change