Generating a series of fine spatial and temporal resolution land cover maps by fusing coarse spatial resolution remotely sensed images and fine spatial resolution land cover maps

Studies of land cover dynamics would benefit greatly from the generation of land cover maps at both fine spatial and temporal resolutions. Fine spatial resolution images are usually acquired relatively infrequently, whereas coarse spatial resolution images may be acquired with a high repetition rate but may not capture the spatial detail of the land cover mosaic of the region of interest. Traditional image spatial–temporal fusion methods focus on the blending of pixel spectra reflectance values and do not directly provide land cover maps or information on land cover dynamics. In this research, a novel Spatial–Temporal remotely sensed Images and land cover Maps Fusion Model (STIMFM) is proposed to produce land cover maps at both fine spatial and temporal resolutions using a series of coarse spatial resolution images together with a few fine spatial resolution land cover maps that pre- and post-date the series of coarse spatial resolution images. STIMFM integrates both the spatial and temporal dependences of fine spatial resolution pixels and outputs a series of fine spatial–temporal resolution land cover maps instead of reflectance images, which can be used directly for studies of land cover dynamics. Here, three experiments based on simulated and real remotely sensed images were undertaken to evaluate the STIMFM for studies of land cover change. These experiments included comparative assessment of methods based on single-date image such as the super-resolution approaches (e.g., pixel swapping-based super-resolution mapping) and the state-of-the-art spatial–temporal fusion approach that used the Enhanced Spatial and Temporal Adaptive Reflectance Fusion Model (ESTARFM) and the Flexible Spatiotemporal DAta Fusion model (FSDAF) to predict the fine-resolution images, in which the maximum likelihood classifier and the automated land cover updating approach based on integrated change detection and classification method were then applied to generate the fine-resolution land cover maps. Results show that the methods based on single-date image failed to predict the pixels of changed and unchanged land cover with high accuracy. The land cover maps that were obtained by classification of the reflectance images outputted from ESTARFM and FSDAF contained substantial misclassification, and the classification accuracy was lower for pixels of changed land cover than for pixels of unchanged land cover. In addition, STIMFM predicted fine spatial–temporal resolution land cover maps from a series of Landsat images and a few Google Earth images, to which ESTARFM and FSDAF that require correlation in reflectance bands in coarse and fine images cannot be applied. Notably, STIMFM generated higher accuracy for pixels of both changed and unchanged land cover in comparison with other methods

Superresolution Enhancement of Hyperspectral CHRIS/Proba Images With a Thin-Plate Spline Nonrigid Transform Model

Given the hyperspectral-oriented waveband configuration of multiangular CHRIS/Proba imagery, the scope of its application could widen if the present 18-m resolution would be improved. The multiangular images of CHRIS could be used as input for superresolution (SR) image reconstruction. A critical procedure in SR is an accurate registration of the low-resolution images. Conventional methods based on affine transformation may not be effective given the local geometric distortion in high off-nadir angular images. This paper examines the use of a non-rigid transform to improve the result of a nonuniform interpolation and deconvolution SR method. A scale-invariant feature transform is used to collect control points (CPs). To ensure the quality of CPs, a rigorous screening procedure is designed: 1) an ambiguity test; 2) the m-estimator sample consensus method; and 3) an iterative method using statistical characteristics of the distribution of random errors. A thin-plate spline (TPS) nonrigid transform is then used for the registration. The proposed registration method is examined with a Delaunay triangulation-based nonuniform interpolation and reconstruction SR method. Our results show that the TPS nonrigid transform allows accurate registration of angular images. SR results obtained from simulated LR images are evaluated using three quantitative measures, namely, relative mean-square error, structural similarity, and edge stability. Compared to the SR methods that use an affine transform, our proposed method performs better with all three evaluation measures. With a higher level of spatial detail, SR-enhanced CHRIS images might be more effective than the original data in various applications.JRC.H.7-Climate Risk Managemen

Assessing a temporal change strategy for sub-pixel land cover change mapping from multi-scale remote sensing imagery

Remotely sensed imagery is an attractive source of information for mapping and monitoring land cover. Fine spatial resolution imagery is typically acquired infrequently, but fine temporal resolution systems commonly provide coarse spatial resolution imagery. Sub-pixel land cover change mapping is a method that aims to use the advantages of these multiple spatial and temporal resolution sensing systems. This method produces fine spatial and temporal resolution land cover maps, by updating fine spatial resolution land cover maps using coarse spatial resolution remote sensing imagery. A critical issue for sub-pixel land cover change mapping is downscaling coarse spatial resolution fraction maps estimated by soft classification to a fine spatial resolution land cover map. The relationship between a historic fine spatial resolution map and a contemporary fine spatial resolution map to be estimated at a more recent date plays an important role in the downscaling procedure. A change strategy based on the assumption that the change for each land cover class in a coarse spatial resolution pixel is unidirectional was shown to be a promising means to describe this relationship. This paper aims to assess this change strategy by analyzing the factors that affect the accuracy of the change strategy, using six subsets of the National Land Cover Database (NLCD) of USA. The results show that the spatial resolution of coarse pixels, the time interval of the previous fine resolution land cover map and the current coarse spatial resolution images, and the thematic resolution of the used land cover class scheme have considerable influence on the accuracy of the change strategy. The accuracy of the change strategy decreases with the coarsening of spatial resolution, an increase of time interval, and an increase of thematic resolution. The results also indicate that, when the historic land cover map has a 30 m resolution, like the NLCD, the average accuracy of the change strategy is still as high as 92% when the coarse spatial resolution data used had a resolution of ~1000 m, confirming the effectiveness of the change strategy used in sub-pixel land cover change mapping for use with popular remote sensing systems

Spatiotemporal subpixel mapping of time-series images

Land cover/land use (LCLU) information extraction from multitemporal sequences of remote sensing imagery is becoming increasingly important. Mixed pixels are a common problem in Landsat and MODIS images that are used widely for LCLU monitoring. Recently developed subpixel mapping (SPM) techniques can extract LCLU information at the subpixel level by dividing mixed pixels into subpixels to which hard classes are then allocated. However, SPM has rarely been studied for time-series images (TSIs). In this paper, a spatiotemporal SPM approach was proposed for SPM of TSIs. In contrast to conventional spatial dependence-based SPM methods, the proposed approach considers simultaneously spatial and temporal dependences, with the former considering the correlation of subpixel classes within each image and the latter considering the correlation of subpixel classes between images in a temporal sequence. The proposed approach was developed assuming the availability of one fine spatial resolution map which exists among the TSIs. The SPM of TSIs is formulated as a constrained optimization problem. Under the coherence constraint imposed by the coarse LCLU proportions, the objective is to maximize the spatiotemporal dependence, which is defined by blending both spatial and temporal dependences. Experiments on three data sets showed that the proposed approach can provide more accurate subpixel resolution TSIs than conventional SPM methods. The SPM results obtained from the TSIs provide an excellent opportunity for LCLU dynamic monitoring and change detection at a finer spatial resolution than the available coarse spatial resolution TSIs

Recent Advances in Image Restoration with Applications to Real World Problems

In the past few decades, imaging hardware has improved tremendously in terms of resolution, making widespread usage of images in many diverse applications on Earth and planetary missions. However, practical issues associated with image acquisition are still affecting image quality. Some of these issues such as blurring, measurement noise, mosaicing artifacts, low spatial or spectral resolution, etc. can seriously affect the accuracy of the aforementioned applications. This book intends to provide the reader with a glimpse of the latest developments and recent advances in image restoration, which includes image super-resolution, image fusion to enhance spatial, spectral resolution, and temporal resolutions, and the generation of synthetic images using deep learning techniques. Some practical applications are also included

Infrared Image Super-Resolution: Systematic Review, and Future Trends

Image Super-Resolution (SR) is essential for a wide range of computer vision and image processing tasks. Investigating infrared (IR) image (or thermal images) super-resolution is a continuing concern within the development of deep learning. This survey aims to provide a comprehensive perspective of IR image super-resolution, including its applications, hardware imaging system dilemmas, and taxonomy of image processing methodologies. In addition, the datasets and evaluation metrics in IR image super-resolution tasks are also discussed. Furthermore, the deficiencies in current technologies and possible promising directions for the community to explore are highlighted. To cope with the rapid development in this field, we intend to regularly update the relevant excellent work at \url{https://github.com/yongsongH/Infrared_Image_SR_SurveyComment: Submitted to IEEE TNNL

Monitoring high spatiotemporal water dynamics by fusing MODIS, Landsat, water occurrence data and DEM

Monitoring the spatiotemporal dynamics of surface water from remote sensing imagery is essential for understanding water's impact on the global ecosystem and climate change. There is often a tradeoff between the spatial and temporal resolutions of imagery acquired from current satellite sensors and as such various spatiotemporal image fusion methods have been explored to circumvent the challenges this situation presents (e.g., STARFM). However, some challenges persist in mapping surface water at the desired fine spatial and temporal resolution. Principally, the spatiotemporal changes of water bodies are often abrupt and controlled by topographic conditions, which are usually unaddressed in current spatiotemporal image fusion methods. This paper proposes the SpatioTemporal Surface Water Mapping (STSWM) method, which aims to predict Landsat-like, 30 m, surface water maps at an 8-day time step (same as the MODIS 8-day composite product) by integrating topographic information into the analysis. In addition to MODIS imagery acquired on the date of map prediction and a pair of MODIS and Landsat images acquired temporally close to the date of prediction, STSWM also uses the surface water occurrence (SWO, which represents the frequency with which water is present in a pixel) and DEM data to provide, respectively, topographic information below and above the water surface. These data are used to translate the coarse spatial resolution water distribution representation observed by MODIS into a 30 m spatial resolution water distribution map. The STSWM was used to generate an 8-day time series surface water maps of 30 m resolution in six inundation regions globally, and was compared with several other state-of-the-art spatiotemporal methods. The stratified random sampling design was used, and unbiased estimators of the accuracies were provided. The results show that STSWM generated the most accurate surface water map in which the spatial details of surface water were well-represented

Spatial-temporal super-resolution land cover mapping with a local spatial-temporal dependence model

The mixed pixel problem is common in remote sensing. A soft classification can generate land cover class fraction images that illustrate the areal proportions of the various land cover classes within pixels. The spatial distribution of land cover classes within each mixed pixel is, however, not represented. Super-resolution land cover mapping (SRM) is a technique to predict the spatial distribution of land cover classes within the mixed pixel using fraction images as input. Spatial-temporal SRM (STSRM) extends the basic SRM to include a temporal dimension by using a finer-spatial resolution land cover map that pre-or postdates the image acquisition time as ancillary data. Traditional STSRM methods often use one land cover map as the constraint, but neglect the majority of available land cover maps acquired at different dates and of the same scene in reconstructing a full state trajectory of land cover changes when applying STSRM to time series data. In addition, the STSRM methods define the temporal dependence globally, and neglect the spatial variation of land cover temporal dependence intensity within images. A novel local STSRM (LSTSRM) is proposed in this paper. LSTSRM incorporates more than one available land cover map to constrain the solution, and develops a local temporal dependence model, in which the temporal dependence intensity may vary spatially. The results show that LSTSRM can eliminate speckle-like artifacts and reconstruct the spatial patterns of land cover patches in the resulting maps, and increase the overall accuracy compared with other STSRM methods