GETNET: A General End-to-end Two-dimensional CNN Framework for Hyperspectral Image Change Detection

Change detection (CD) is an important application of remote sensing, which provides timely change information about large-scale Earth surface. With the emergence of hyperspectral imagery, CD technology has been greatly promoted, as hyperspectral data with the highspectral resolution are capable of detecting finer changes than using the traditional multispectral imagery. Nevertheless, the high dimension of hyperspectral data makes it difficult to implement traditional CD algorithms. Besides, endmember abundance information at subpixel level is often not fully utilized. In order to better handle high dimension problem and explore abundance information, this paper presents a General End-to-end Two-dimensional CNN (GETNET) framework for hyperspectral image change detection (HSI-CD). The main contributions of this work are threefold: 1) Mixed-affinity matrix that integrates subpixel representation is introduced to mine more cross-channel gradient features and fuse multi-source information; 2) 2-D CNN is designed to learn the discriminative features effectively from multi-source data at a higher level and enhance the generalization ability of the proposed CD algorithm; 3) A new HSI-CD data set is designed for the objective comparison of different methods. Experimental results on real hyperspectral data sets demonstrate the proposed method outperforms most of the state-of-the-arts

Key Information Retrieval in Hyperspectral Imagery through Spatial-Spectral Data Fusion

Hyperspectral (HS) imaging is measuring the radiance of materials within each pixel area at a large number of contiguous spectral wavelength bands. The key spatial information such as small targets and border lines are hard to be precisely detected from HS data due to the technological constraints. Therefore, the need for image processing techniques is an important field of research in HS remote sensing. A novel semisupervised spatial-spectral data fusion method for resolution enhancement of HS images through maximizing the spatial correlation of the endmembers (signature of pure or purest materials in the scene) using a superresolution mapping (SRM) technique is proposed in this paper. The method adopts a linear mixture model and a fully constrained least squares spectral unmixing algorithm to obtain the endmember abundances (fractional images) of HS images. Then, the extracted endmember distribution maps are fused with the spatial information using a spatial-spectral correlation maximizing model and a learning-based SRM technique to exploit the subpixel level data. The obtained results validate the reliability of the technique for key information retrieval. The proposed method is very efficient and is low in terms of computational cost which makes it favorable for real-time applications

A Quantitative Assessment of Forest Cover Change in the Moulouya River Watershed (Morocco) by the Integration of a Subpixel-Based and Object-Based Analysis of Landsat Data

A quantitative assessment of forest cover change in the Moulouya River watershed (Morocco) was carried out by means of an innovative approach from atmospherically corrected reflectance Landsat images corresponding to 1984 (Landsat 5 Thematic Mapper) and 2013 (Landsat 8 Operational Land Imager). An object-based image analysis (OBIA) was undertaken to classify segmented objects as forested or non-forested within the 2013 Landsat orthomosaic. A Random Forest classifier was applied to a set of training data based on a features vector composed of different types of object features such as vegetation indices, mean spectral values and pixel-based fractional cover derived from probabilistic spectral mixture analysis). The very high spatial resolution image data of Google Earth 2013 were employed to train/validate the Random Forest classifier, ranking the NDVI vegetation index and the corresponding pixel-based percentages of photosynthetic vegetation and bare soil as the most statistically significant object features to extract forested and non-forested areas. Regarding classification accuracy, an overall accuracy of 92.34% was achieved. The previously developed classification scheme was applied to the 1984 Landsat data to extract the forest cover change between 1984 and 2013, showing a slight net increase of 5.3% (ca. 8800 ha) in forested areas for the whole region

Mapping Impervious Surface Using Phenology-Integrated and Fisher Transformed Linear Spectral Mixture Analysis

The impervious surface area (ISA) is a key indicator of urbanization, which brings out serious adverse environmental and ecological consequences. The ISA is often estimated from remotely sensed data via spectral mixture analysis (SMA). However, accurate extraction of ISA using SMA is compromised by two major factors, endmember spectral variability and plant phenology. This study developed a novel approach that incorporates phenology with Fisher transformation into a conventional linear spectral mixture analysis (PF-LSMA) to address these challenges. Four endmembers, high albedo, low albedo, evergreen vegetation, and seasonally exposed soil (H-L-EV-SS) were identified for PF-LSMA, considering the phenological characteristic of Shanghai. Our study demonstrated that the PF-LSMA effectively reduced the within-endmember spectral signature variation and accounted for the endmember phenology effects, and thus well-discriminated impervious surface from seasonally exposed soil, enhancing the accuracy of ISA extraction. The ISA fraction map produced by PF-LSMA (RMSE = 0.1112) outperforms the single-date image Fisher transformed unmixing method (F-LSMA) (RMSE = 0.1327) and the other existing major global ISA products. The PF-LSMA was implemented on the Google Earth Engine platform and thus can be easily adapted to extract ISA in other places with similar climate conditions.Peer Reviewe

Deep Image Prior for Disentangling Mixed Pixels

A mixed pixel in remotely sensed images measures the reflectance and emission from multiple target types (e.g., tree, grass, and building) from a certain area. Mixed pixels exist commonly in spaceborne hyper-/multi-spectral images due to sensor limitations, causing the signature ambiguity problem and impeding high-resolution remote sensing mapping. Disentangling mixed pixels into the underlying constituent components is a challenging ill-posed inverse problem, which requires efficient modeling of spatial prior information and other application-dependent prior knowledge concerning the mixed pixel generation process. The recent deep image prior (DIP) approach and other application-dependent prior information are integrated into a Bayesian framework in the research, which allows comprehensive usage of different prior knowledge. The research improves mixed pixel disentangling using the Bayesian DIP in three key applications: spectral unmixing (SU), subpixel mapping (SPM), and soil moisture product downscaling (SMD). The main contributions are summarized as follows. First, to improve the decomposition of mixed pixels into pure material spectra (i.e., endmembers) and their constituting fractions (i.e., abundances) in SU, a designed deep fully convolutional neural network (DCNN) and a new spectral mixture model (SMM) with heterogeneous noise are integrated into a Bayesian framework that is efficiently solved by a new iterative optimization algorithm. Second, to improve the decomposition of mixed pixels into class labels of subpixels in SPM, a dedicated DCNN architecture and a new discrete SMM are integrated into the Bayesian framework to allow the use of both spatial prior and the forward model. Third, to improve the decomposition of mixed pixels into soil moisture concentrations of subpixels in SMD, a new DIP architecture and a forward degradation model are integrated into the Bayesian framework that is solved by the stochastic gradient descent approach. These new Bayesian approaches improve the state-of-the-art in their respective applications (i.e., SU, SPM, and SMD), which can be potentially utilized for solving other ill-posed inverse problems where simultaneously modeling of the spatial prior and other prior knowledge is needed

Context dependent spectral unmixing.

A hyperspectral unmixing algorithm that finds multiple sets of endmembers is proposed. The algorithm, called Context Dependent Spectral Unmixing (CDSU), is a local approach that adapts the unmixing to different regions of the spectral space. It is based on a novel function that combines context identification and unmixing. This joint objective function models contexts as compact clusters and uses the linear mixing model as the basis for unmixing. Several variations of the CDSU, that provide additional desirable features, are also proposed. First, the Context Dependent Spectral unmixing using the Mahalanobis Distance (CDSUM) offers the advantage of identifying non-spherical clusters in the high dimensional spectral space. Second, the Cluster and Proportion Constrained Multi-Model Unmixing (CC-MMU and PC-MMU) algorithms use partial supervision information, in the form of cluster or proportion constraints, to guide the search process and narrow the space of possible solutions. The supervision information could be provided by an expert, generated by analyzing the consensus of multiple unmixing algorithms, or extracted from co-located data from a different sensor. Third, the Robust Context Dependent Spectral Unmixing (RCDSU) introduces possibilistic memberships into the objective function to reduce the effect of noise and outliers in the data. Finally, the Unsupervised Robust Context Dependent Spectral Unmixing (U-RCDSU) algorithm learns the optimal number of contexts in an unsupervised way. The performance of each algorithm is evaluated using synthetic and real data. We show that the proposed methods can identify meaningful and coherent contexts, and appropriate endmembers within each context. The second main contribution of this thesis is consensus unmixing. This approach exploits the diversity and similarity of the large number of existing unmixing algorithms to identify an accurate and consistent set of endmembers in the data. We run multiple unmixing algorithms using different parameters, and combine the resulting unmixing ensemble using consensus analysis. The extracted endmembers will be the ones that have a consensus among the multiple runs. The third main contribution consists of developing subpixel target detectors that rely on the proposed CDSU algorithms to adapt target detection algorithms to different contexts. A local detection statistic is computed for each context and then all scores are combined to yield a final detection score. The context dependent unmixing provides a better background description and limits target leakage, which are two essential properties for target detection algorithms

Uncertainty Assessment of Spectral Mixture Analysis in Remote Sensing Imagery

Spectral mixture analysis (SMA), a scheme of sub-pixel-based classifications, is one of the widely used models to map fractional land use and land cover information in remote sensing imagery. It assumes that: 1) a mixed pixel is composed by several pure land cover classes (endmembers) linearly or nonlinearly, and 2) the spectral signature of each endmember is a constant within the entire spatial extent of analysis. SMA has been commonly applied to impervious surface area extraction, vegetation fraction estimation, and land use and land cover change (LULC) mapping. Limitations of SMA, however, still exist. First, the existence of between- and within-class variability prevents the selection of accurate endmembers, which results in poor accuracy of fractional land cover estimates. Weighted spectral mixture analysis (WSMA) and transformed spectral mixture analysis (TSMA) are alternate means to address the within- and between- class variability. These methods, however, have not been analyzed systematically and comprehensively. The effectiveness of each WSMA and TSMA scheme is still unknown, in particular within different urban areas. Second, multiple endmember SMA (MESMA) is a better alternative to address spectral mixture model uncertainties. It, nonetheless, is time consuming and inefficient. Further, incorrect endmember selections may still limit model performance as the best-fit endmember model might not be the optimal model due to the existence of spectral variability. Therefore, this study aims 1) to explore endmember uncertainties by examining WSMA and TSMA modeling comprehensively, and 2) to develop an improved MESMA model in order to address the uncertainties of spectral mixture models. Results of the WSMA examination illustrated that some weighting schemes did reduce endmember uncertainties since they could improve the fractional estimates significantly. The results also indicated that spectral class variance played a key role in addressing the endmember uncertainties, as the better performing weighting schemes were constructed with spectral class variance. In addition, the results of TSMA examination demonstrated that some TSMAs, such as normalized spectral mixture analysis (NSMA), could effectively solve the endmember uncertainties because of their stable performance in different study areas. Results of Class-based MEMSA (C-MESMA) indicated that it could address spectral mixture model uncertainties by reducing a lot of the calculation burden and effectively improving accuracy. Assessment demonstrated that C-MEMSA significantly improving accuracy. Major contributions of this study can be summarized as follow. First, the effectiveness of addressing endmember uncertainties have been fully discussed by examining: 1) the effectiveness of ten weighted spectral mixture models in urban environments; and 2) the effectiveness of 26 transformed spectral mixture models in three locations. Constructive guidance regarding handling endmember uncertainties using WSMA and TSMA have been provided. Second, the uncertainties of spectral mixture model were reduced by developing an improved MESMA model, named C-MESMA. C-MESMA could restrict the distribution of endmembers and reduce the calculation burden of traditional MESMA, increasing SMA accuracy significantly

Multiscale Imaging of Evapotranspiration

Evapotranspiration (ET; evaporation + transpiration) is central to a wide range of biological, chemical, and physical processes in the Earth system. Accurate remote sensing of ET is challenging due to the interrelated and generally scale dependent nature of the physical factors which contribute to the process. The evaporation of water from porous media like sands and soils is an important subset of the complete ET problem. Chapter 1 presents a laboratory investigation into this question, examining the effects of grain size and composition on the evolution of drying sands. The effects of composition are found to be 2-5x greater than the effects of grain size, indicating that differences in heating caused by differences in reflectance may dominate hydrologic differences caused by grain size variation. In order to relate the results of Chapter 1 to the satellite image archive, however, the question of information loss between hyperspectral (measurements at 100s of wavelength intervals) laboratory measurements and multispectral (≤ 12 wavelength intervals) satellite images must be addressed. Chapter 2 focuses on this question as applied to substrate materials such as sediment, soil, rock, and non-photosynthetic vegetation. The results indicate that the continuum that is resolved by multispectral sensors is sufficient to resolve the gradient between sand-rich and clay-rich soils, and that this gradient is also a dominant feature in hyperspectral mixing spaces where the actual absorptions can be resolved. Multispectral measurements can be converted to biogeophysically relevant quantities using spectral mixture analysis (SMA). However, retrospective multitemporal analysis first requires cross-sensor calibration of the mixture model. Chapter 3 presents this calibration, allowing multispectral image data to be used interchangeably throughout the Landsat 4-8 archive. In addition, a theoretical explanation is advanced for the observed superior scaling properties of SMA-derived fraction images over spectral indices. The physical quantities estimated by the spectral mixture model are then compared to simultaneously imaged surface temperature, as well as to the derived parameters of ET Fraction and Moisture Availability. SMA-derived vegetation abundance is found to produce substantially more informative ET maps, and SMA-derived substrate fraction is found to yield a surprisingly strong linear relationship with surface temperature. These results provide context for agricultural applications. Chapter 5 investigates the question of mapping and monitoring rice agricultural using optical and thermal satellite image time series. Thermal image time series are found to produce more accurate maps of rice presence/absence, but optical image time series are found to produce more accurate maps of rice crop timing. Chapter 6 takes a more global approach, investigating the spatial structure of agricultural networks for a diverse set of landscapes. Surprisingly consistent scaling relations are found. These relations are assessed in the context of a network-based approach to land cover analysis, with potential implications for the scale dependence of ET estimates. In sum, this thesis present a novel approach to improving ET estimation based on a synthesis of complementary laboratory measurements, satellite image analysis, and field observations. Alone, each of these independent sources of information provides novel insights. Viewed together, these insights form the basis of a more accurate and complete geophysical understanding of the ET phenomenon