The effect of the point spread function on sub-pixel mapping

Abstract Sub-pixel mapping (SPM) is a process for predicting spatially the land cover classes within mixed pixels. In existing SPM methods, the effect of point spread function (PSF) has seldom been considered. In this paper, a generic SPM method is developed to consider the PSF effect in SPM and, thereby, to increase prediction accuracy. We first demonstrate that the spectral unmixing predictions (i.e., coarse land cover proportions used as input for SPM) are a convolution of not only sub-pixels within the coarse pixel, but also sub-pixels from neighboring coarse pixels. Based on this finding, a new SPM method based on optimization is developed which recognizes the optimal solution as the one that when convolved with the PSF, is the same as the input coarse land cover proportion. Experimental results on three separate datasets show that the SPM accuracy can be increased by considering the PSF effect

The effect of the point spread function on sub-pixel mapping

Abstract Sub-pixel mapping (SPM) is a process for predicting spatially the land cover classes within mixed pixels. In existing SPM methods, the effect of point spread function (PSF) has seldom been considered. In this paper, a generic SPM method is developed to consider the PSF effect in SPM and, thereby, to increase prediction accuracy. We first demonstrate that the spectral unmixing predictions (i.e., coarse land cover proportions used as input for SPM) are a convolution of not only sub-pixels within the coarse pixel, but also sub-pixels from neighboring coarse pixels. Based on this finding, a new SPM method based on optimization is developed which recognizes the optimal solution as the one that when convolved with the PSF, is the same as the input coarse land cover proportion. Experimental results on three separate datasets show that the SPM accuracy can be increased by considering the PSF effect

Reducing the impacts of intra-class spectral variability on the accuracy of soft classification and super-resolution mapping of shoreline

The main objective of this research is to assess the impact of intra-class spectral variation on the accuracy of soft classification and super-resolution mapping. The accuracy of both analyses was negatively related to the degree of intra-class spectral variation, but the effect could be reduced through use of spectral sub-classes. The latter is illustrated in mapping the shoreline at a sub-pixel scale from Landsat ETM+ data. Reducing the degree of intra-class spectral variation increased the accuracy of soft classification, with the correlation between predicted and actual class coverage rising from 0.87 to 0.94, and super-resolution mapping, with the RMSE in shoreline location decreasing from 41.13 m to 35.22 m

Deep Image Prior for Disentangling Mixed Pixels

A mixed pixel in remotely sensed images measures the reflectance and emission from multiple target types (e.g., tree, grass, and building) from a certain area. Mixed pixels exist commonly in spaceborne hyper-/multi-spectral images due to sensor limitations, causing the signature ambiguity problem and impeding high-resolution remote sensing mapping. Disentangling mixed pixels into the underlying constituent components is a challenging ill-posed inverse problem, which requires efficient modeling of spatial prior information and other application-dependent prior knowledge concerning the mixed pixel generation process. The recent deep image prior (DIP) approach and other application-dependent prior information are integrated into a Bayesian framework in the research, which allows comprehensive usage of different prior knowledge. The research improves mixed pixel disentangling using the Bayesian DIP in three key applications: spectral unmixing (SU), subpixel mapping (SPM), and soil moisture product downscaling (SMD). The main contributions are summarized as follows. First, to improve the decomposition of mixed pixels into pure material spectra (i.e., endmembers) and their constituting fractions (i.e., abundances) in SU, a designed deep fully convolutional neural network (DCNN) and a new spectral mixture model (SMM) with heterogeneous noise are integrated into a Bayesian framework that is efficiently solved by a new iterative optimization algorithm. Second, to improve the decomposition of mixed pixels into class labels of subpixels in SPM, a dedicated DCNN architecture and a new discrete SMM are integrated into the Bayesian framework to allow the use of both spatial prior and the forward model. Third, to improve the decomposition of mixed pixels into soil moisture concentrations of subpixels in SMD, a new DIP architecture and a forward degradation model are integrated into the Bayesian framework that is solved by the stochastic gradient descent approach. These new Bayesian approaches improve the state-of-the-art in their respective applications (i.e., SU, SPM, and SMD), which can be potentially utilized for solving other ill-posed inverse problems where simultaneously modeling of the spatial prior and other prior knowledge is needed

Updating Landsat-based forest cover maps with MODIS images using multiscale spectral-spatial-temporal superresolution mapping

Abstract With the high deforestation rates of global forest covers during the past decades, there is an ever-increasing need to monitor forest covers at both fine spatial and temporal resolutions. Moderate Resolution Imaging Spectroradiometer (MODIS) and Landsat series images have been used commonly for satellite-derived forest cover mapping. However, the spatial resolution of MODIS images and the temporal resolution of Landsat images are too coarse to observe forest cover at both fine spatial and temporal resolutions. In this paper, a novel multiscale spectral-spatial-temporal superresolution mapping (MSSTSRM) approach is proposed to update Landsat-based forest maps by integrating current MODIS images with the previous forest maps generated from Landsat image. Both the 240 m MODIS bands and 480 m MODIS bands were used as inputs of the spectral energy function of the MSSTSRM model. The principle of maximal spatial dependence was used as the spatial energy function to make the updated forest map spatially smooth. The temporal energy function was based on a multiscale spatial-temporal dependence model, and considers the land cover changes between the previous and current time. The novel MSSTSRM model was able to update Landsat-based forest maps more accurately, in terms of both visual and quantitative evaluation, than traditional pixel-based classification and the latest sub-pixel based super-resolution mapping methods The results demonstrate the great efficiency and potential of MSSTSRM for updating fine temporal resolution Landsat-based forest maps using MODIS images

Mapping annual forest cover by fusing PALSAR/PALSAR-2 and MODIS NDVI during 2007–2016

Advanced Land Observing Satellite (ALOS) Phased Arrayed L-band Synthetic Aperture Radar (PALSAR) HH and HV polarization data were used previously to produce annual, global 25 m forest maps between 2007 and 2010, and the latest global forest maps of 2015 and 2016 were produced by using the ALOS-2 PALSAR-2 data. However, annual 25 m spatial resolution forest maps during 2011–2014 are missing because of the gap in operation between ALOS and ALOS-2, preventing the construction of a continuous, fine resolution time-series dataset on the world's forests. In contrast, the MODerate Resolution Imaging Spectroradiometer (MODIS) NDVI images were available globally since 2000. This research developed a novel method to produce annual 25 m forest maps during 2007–2016 by fusing the fine spatial resolution, but asynchronous PALSAR/PALSAR-2 with coarse spatial resolution, but synchronous MODIS NDVI data, thus, filling the four-year gap in the ALOS and ALOS-2 time-series, as well as enhancing the existing mapping activity. The method was developed concentrating on two key objectives: 1) producing more accurate 25 m forest maps by integrating PALSAR/PALSAR-2 and MODIS NDVI data during 2007–2010 and 2015–2016; 2) reconstructing annual 25 m forest maps from time-series MODIS NDVI images during 2011–2014. Specifically, a decision tree classification was developed for forest mapping based on both the PALSAR/PALSAR-2 and MODIS NDVI data, and a new spatial-temporal super-resolution mapping was proposed to reconstruct the 25 m forest maps from time-series MODIS NDVI images. Three study sites including Paraguay, the USA and Russia were chosen, as they represent the world's three main forest types: tropical forest, temperate broadleaf and mixed forest, and boreal conifer forest, respectively. Compared with traditional methods, the proposed approach produced the most accurate continuous time-series of fine spatial resolution forest maps both visually and quantitatively. For the forest maps during 2007–2010 and 2015–2016, the results had greater overall accuracy values (>98%) than those of the original JAXA forest product. For the reconstructed 25 m forest maps during 2011–2014, the increases in classifications accuracy relative to three benchmark methods were statistically significant, and the overall accuracy values of the three study sites were almost universally >92%. The proposed approach, therefore, has great potential to support the production of annual 25 m forest maps by fusing PALSAR/PALSAR-2 and MODIS NDVI during 2007–2016

Super-resolution mapping

Super-resolution mapping is becoming an increasing important technique in remote sensing for land cover mapping at a sub-pixel scale from coarse spatial resolution imagery. The potential of this technique could increase the value of the low cost coarse spatial resolution imagery. Among many types of land cover patches that can be represented by the super-resolution mapping, the prediction of patches smaller than an image pixel is one of the most difficult. This is because of the lack of information on the existence and spatial extend of the small land cover patches. Another difficult problem is to represent the location of small patches accurately. This thesis focuses on the potential of super-resolution mapping for accurate land cover mapping, with particular emphasis on the mapping of small patches. Popular super-resolution mapping techniques such as pixel swapping and the Hopfield neural network are used as well as a new method proposed. Using a Hopfield neural network (HNN) for super-resolution mapping, the best parameters and configuration to represent land cover patches of different sizes, shapes and mosaics are investigated. In addition, it also shown how a fusion of time series coarse spatial resolution imagery, such as daily MODIS 250 m images, can aid the determination of small land cover patch locations, thus reducing the spatial variability of the representation of such patches. Results of the improved HNN using a time series images are evaluated in a series of assessments, and demonstrated to be superior in terms of mapping accuracy than that of the standard techniques. A novel super-resolution mapping technique based on halftoning concept is presented as an alternative solution for the super-resolution mapping. This new technique is able to represent more land cover patches than the standard techniques

Super-resolution mapping

Super-resolution mapping is becoming an increasing important technique in remote sensing for land cover mapping at a sub-pixel scale from coarse spatial resolution imagery. The potential of this technique could increase the value of the low cost coarse spatial resolution imagery. Among many types of land cover patches that can be represented by the super-resolution mapping, the prediction of patches smaller than an image pixel is one of the most difficult. This is because of the lack of information on the existence and spatial extend of the small land cover patches. Another difficult problem is to represent the location of small patches accurately. This thesis focuses on the potential of super-resolution mapping for accurate land cover mapping, with particular emphasis on the mapping of small patches. Popular super-resolution mapping techniques such as pixel swapping and the Hopfield neural network are used as well as a new method proposed. Using a Hopfield neural network (HNN) for super-resolution mapping, the best parameters and configuration to represent land cover patches of different sizes, shapes and mosaics are investigated. In addition, it also shown how a fusion of time series coarse spatial resolution imagery, such as daily MODIS 250 m images, can aid the determination of small land cover patch locations, thus reducing the spatial variability of the representation of such patches. Results of the improved HNN using a time series images are evaluated in a series of assessments, and demonstrated to be superior in terms of mapping accuracy than that of the standard techniques. A novel super-resolution mapping technique based on halftoning concept is presented as an alternative solution for the super-resolution mapping. This new technique is able to represent more land cover patches than the standard techniques

A review of technical factors to consider when designing neural networks for semantic segmentation of Earth Observation imagery

Semantic segmentation (classification) of Earth Observation imagery is a crucial task in remote sensing. This paper presents a comprehensive review of technical factors to consider when designing neural networks for this purpose. The review focuses on Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Generative Adversarial Networks (GANs), and transformer models, discussing prominent design patterns for these ANN families and their implications for semantic segmentation. Common pre-processing techniques for ensuring optimal data preparation are also covered. These include methods for image normalization and chipping, as well as strategies for addressing data imbalance in training samples, and techniques for overcoming limited data, including augmentation techniques, transfer learning, and domain adaptation. By encompassing both the technical aspects of neural network design and the data-related considerations, this review provides researchers and practitioners with a comprehensive and up-to-date understanding of the factors involved in designing effective neural networks for semantic segmentation of Earth Observation imagery.Comment: 145 pages with 32 figure

Image Registration Workshop Proceedings

Automatic image registration has often been considered as a preliminary step for higher-level processing, such as object recognition or data fusion. But with the unprecedented amounts of data which are being and will continue to be generated by newly developed sensors, the very topic of automatic image registration has become and important research topic. This workshop presents a collection of very high quality work which has been grouped in four main areas: (1) theoretical aspects of image registration; (2) applications to satellite imagery; (3) applications to medical imagery; and (4) image registration for computer vision research