Fast and Slow Changes Constrained Spatio-temporal Subpixel Mapping

Subpixel mapping (SPM) is a technique to tackle the mixed pixel problem and produce land cover and land use (LCLU) maps at a finer spatial resolution than the original coarse data. However, uncertainty exists unavoidably in SPM, which is an ill-posed downscaling problem. Spatio-temporal SPM methods have been proposed to deal with this uncertainty, but current methods fail to explore fully the information in the time-series images, especially more rapid changes over a short-time interval. In this paper, a fast and slow changes constrained spatio-temporal subpixel mapping (FSSTSPM) method is proposed to account for fast LCLU changes over a short-time interval and slow changes over a long-time interval. Namely, both fast and slow change-based temporal constraints are proposed and incorporated simultaneously into the FSSTSPM to increase the accuracy of SPM. The proposed FSSTSPM method was validated using two synthetic datasets with various proportion errors. It was also applied to oil-spill mapping using a real PlanetScope-Sentinel-2 dataset and Amazon deforestation mapping using a real Landsat-MODIS dataset. The results demonstrate the superiority of FSSTSPM. Moreover, the advantage of FSSTSPM is more obvious with an increase in proportion errors. The concepts of the fast and slow changes, together with the derived temporal constraints, provide a new insight to enhance SPM by taking fuller advantage of the temporal information in the available time-series images

Fundamental remote sensing science research program. Part 1: Status report of the mathematical pattern recognition and image analysis project

The Mathematical Pattern Recognition and Image Analysis (MPRIA) Project is concerned with basic research problems related to the study of the Earth from remotely sensed measurement of its surface characteristics. The program goal is to better understand how to analyze the digital image that represents the spatial, spectral, and temporal arrangement of these measurements for purposing of making selected inference about the Earth

Spatiotemporal subpixel mapping of time-series images

Land cover/land use (LCLU) information extraction from multitemporal sequences of remote sensing imagery is becoming increasingly important. Mixed pixels are a common problem in Landsat and MODIS images that are used widely for LCLU monitoring. Recently developed subpixel mapping (SPM) techniques can extract LCLU information at the subpixel level by dividing mixed pixels into subpixels to which hard classes are then allocated. However, SPM has rarely been studied for time-series images (TSIs). In this paper, a spatiotemporal SPM approach was proposed for SPM of TSIs. In contrast to conventional spatial dependence-based SPM methods, the proposed approach considers simultaneously spatial and temporal dependences, with the former considering the correlation of subpixel classes within each image and the latter considering the correlation of subpixel classes between images in a temporal sequence. The proposed approach was developed assuming the availability of one fine spatial resolution map which exists among the TSIs. The SPM of TSIs is formulated as a constrained optimization problem. Under the coherence constraint imposed by the coarse LCLU proportions, the objective is to maximize the spatiotemporal dependence, which is defined by blending both spatial and temporal dependences. Experiments on three data sets showed that the proposed approach can provide more accurate subpixel resolution TSIs than conventional SPM methods. The SPM results obtained from the TSIs provide an excellent opportunity for LCLU dynamic monitoring and change detection at a finer spatial resolution than the available coarse spatial resolution TSIs

General solution to reduce the point spread function effect in subpixel mapping

The point spread function (PSF) effect is ubiquitous in remote sensing images and imposes a fundamental uncertainty on subpixel mapping (SPM). The crucial PSF effect has been neglected in existing SPM methods. This paper proposes a general model to reduce the PSF effect in SPM. The model is applicable to any SPM methods treating spectral unmixing as pre-processing. To demonstrate the advantages of the new technique it was necessary to develop a new approach for accuracy assessment of SPM. To-date, accuracy assessment for SPM has been limited to subpixel classification accuracy, ignoring the performance of reproducing spatial structure in downscaling. In this paper, a new accuracy index is proposed which considers SPM performances in classification and restoration of spatial structure simultaneously. Experimental results show that by considering the PSF effect, more accurate SPM results were produced and small-sized patches and elongated features were restored more satisfactorily. Moreover, using the novel accuracy index, the quantitative evaluation was found to be more consistent with visual evaluation. This paper, thus, addresses directly two of the longest standing challenges in SPM (i.e., the limitations of the PSF effect and accuracy assessment undertaken only on a subpixel-by-subpixel basis). © 2020 Elsevier Inc

Reducing the impacts of intra-class spectral variability on the accuracy of soft classification and super-resolution mapping of shoreline

The main objective of this research is to assess the impact of intra-class spectral variation on the accuracy of soft classification and super-resolution mapping. The accuracy of both analyses was negatively related to the degree of intra-class spectral variation, but the effect could be reduced through use of spectral sub-classes. The latter is illustrated in mapping the shoreline at a sub-pixel scale from Landsat ETM+ data. Reducing the degree of intra-class spectral variation increased the accuracy of soft classification, with the correlation between predicted and actual class coverage rising from 0.87 to 0.94, and super-resolution mapping, with the RMSE in shoreline location decreasing from 41.13 m to 35.22 m

Super Resolution of Wavelet-Encoded Images and Videos

In this dissertation, we address the multiframe super resolution reconstruction problem for wavelet-encoded images and videos. The goal of multiframe super resolution is to obtain one or more high resolution images by fusing a sequence of degraded or aliased low resolution images of the same scene. Since the low resolution images may be unaligned, a registration step is required before super resolution reconstruction. Therefore, we first explore in-band (i.e. in the wavelet-domain) image registration; then, investigate super resolution. Our motivation for analyzing the image registration and super resolution problems in the wavelet domain is the growing trend in wavelet-encoded imaging, and wavelet-encoding for image/video compression. Due to drawbacks of widely used discrete cosine transform in image and video compression, a considerable amount of literature is devoted to wavelet-based methods. However, since wavelets are shift-variant, existing methods cannot utilize wavelet subbands efficiently. In order to overcome this drawback, we establish and explore the direct relationship between the subbands under a translational shift, for image registration and super resolution. We then employ our devised in-band methodology, in a motion compensated video compression framework, to demonstrate the effective usage of wavelet subbands. Super resolution can also be used as a post-processing step in video compression in order to decrease the size of the video files to be compressed, with downsampling added as a pre-processing step. Therefore, we present a video compression scheme that utilizes super resolution to reconstruct the high frequency information lost during downsampling. In addition, super resolution is a crucial post-processing step for satellite imagery, due to the fact that it is hard to update imaging devices after a satellite is launched. Thus, we also demonstrate the usage of our devised methods in enhancing resolution of pansharpened multispectral images

Principles and methods of scaling geospatial Earth science data

The properties of geographical phenomena vary with changes in the scale of measurement. The information observed at one scale often cannot be directly used as information at another scale. Scaling addresses these changes in properties in relation to the scale of measurement, and plays an important role in Earth sciences by providing information at the scale of interest, which may be required for a range of applications, and may be useful for inferring geographical patterns and processes. This paper presents a review of geospatial scaling methods for Earth science data. Based on spatial properties, we propose a methodological framework for scaling addressing upscaling, downscaling and side-scaling. This framework combines scale-independent and scale-dependent properties of geographical variables. It allows treatment of the varying spatial heterogeneity of geographical phenomena, combines spatial autocorrelation and heterogeneity, addresses scale-independent and scale-dependent factors, explores changes in information, incorporates geospatial Earth surface processes and uncertainties, and identifies the optimal scale(s) of models. This study shows that the classification of scaling methods according to various heterogeneities has great potential utility as an underpinning conceptual basis for advances in many Earth science research domains. © 2019 Elsevier B.V

Optimizing hopfield neural network for super-resolution mapping

Remote sensing is a potential source of information of land covers on the surface of the Earth. Different types of remote sensing images offer different spatial resolution quality. High resolution images contain rich information, but they are expensive, while low resolution image are less detail but they are cheap. Super-resolution mapping (SRM) technique is used to enhance the spatial resolution of the low resolution image in order to produce land cover mapping with high accuracy. The mapping technique is crucial to differentiate land cover classes. Hopfield neural network (HNN) is a popular approach in SRM. Currently, numerical implementation of HNN uses ordinary differential equation (ODE) calculated with traditional Euler method. Although producing satisfactory accuracy, Euler method is considered slow especially when dealing with large data like remote sensing image. Therefore, in this paper several advanced numerical methods are applied to the formulation of the ODE in SRM in order to speed up the iterative procedure of SRM. These methods are an improved Euler, Runge-Kutta, and Adams-Moulton. Four classes of land covers such as vegetation, water bodies, roads, and buildings are used in this work. Results of traditional Euler produces mapping accuracy of 85.18% computed in 1000 iterations within 220-1020 seconds. Improved Euler method produces accuracy of 86.63% computed in a range of 60-620 iterations within 20-500 seconds. Runge-Kutta method produces accuracy of 86.63% computed in a range of 70-600 iterations within 20-400 seconds. Adams-Moulton method produces accuracy of 86.64% in a range of 40-320 iterations within 10-150 seconds

The effect of the point spread function on sub-pixel mapping

Abstract Sub-pixel mapping (SPM) is a process for predicting spatially the land cover classes within mixed pixels. In existing SPM methods, the effect of point spread function (PSF) has seldom been considered. In this paper, a generic SPM method is developed to consider the PSF effect in SPM and, thereby, to increase prediction accuracy. We first demonstrate that the spectral unmixing predictions (i.e., coarse land cover proportions used as input for SPM) are a convolution of not only sub-pixels within the coarse pixel, but also sub-pixels from neighboring coarse pixels. Based on this finding, a new SPM method based on optimization is developed which recognizes the optimal solution as the one that when convolved with the PSF, is the same as the input coarse land cover proportion. Experimental results on three separate datasets show that the SPM accuracy can be increased by considering the PSF effect

An iterative interpolation deconvolution algorithm for superresolution land cover mapping

Super-resolution mapping (SRM) is a method to produce a fine spatial resolution land cover map from coarse spatial resolution remotely sensed imagery. A popular approach for SRM is a two-step algorithm, which first increases the spatial resolution of coarse fraction images by interpolation, and then determines class labels of fine resolution pixels using the maximum a posteriori (MAP) principle. By constructing a new image formation process that establishes the relationship between observed coarse resolution fraction images and the latent fine resolution land cover map, it is found that the MAP principle only matches with area-to-point interpolation algorithms, and should be replaced by de-convolution if an area-to-area interpolation algorithm is to be applied. A novel iterative interpolation de-convolution (IID) SRM algorithm is proposed. The IID algorithm first interpolates coarse resolution fraction images with an area-to-area interpolation algorithm, and produces an initial fine resolution land cover map by de-convolution. The fine spatial resolution land cover map is then updated by re-convolution, back-projection and de-convolution iteratively until the final result is produced. The IID algorithm was evaluated with simulated shapes, simulated multi-spectral images, and degraded Landsat images, including comparison against three widely used SRM algorithms: pixel swapping, bilinear interpolation, and Hopfield neural network. Results show that the IID algorithm can reduce the impact of fraction errors, and can preserve the patch continuity and the patch boundary smoothness, simultaneously. Moreover, the IID algorithm produced fine resolution land cover maps with higher accuracies than those produced by other SRM algorithms