Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches

Imaging spectrometers measure electromagnetic energy scattered in their instantaneous field view in hundreds or thousands of spectral channels with higher spectral resolution than multispectral cameras. Imaging spectrometers are therefore often referred to as hyperspectral cameras (HSCs). Higher spectral resolution enables material identification via spectroscopic analysis, which facilitates countless applications that require identifying materials in scenarios unsuitable for classical spectroscopic analysis. Due to low spatial resolution of HSCs, microscopic material mixing, and multiple scattering, spectra measured by HSCs are mixtures of spectra of materials in a scene. Thus, accurate estimation requires unmixing. Pixels are assumed to be mixtures of a few materials, called endmembers. Unmixing involves estimating all or some of: the number of endmembers, their spectral signatures, and their abundances at each pixel. Unmixing is a challenging, ill-posed inverse problem because of model inaccuracies, observation noise, environmental conditions, endmember variability, and data set size. Researchers have devised and investigated many models searching for robust, stable, tractable, and accurate unmixing algorithms. This paper presents an overview of unmixing methods from the time of Keshava and Mustard's unmixing tutorial [1] to the present. Mixing models are first discussed. Signal-subspace, geometrical, statistical, sparsity-based, and spatial-contextual unmixing algorithms are described. Mathematical problems and potential solutions are described. Algorithm characteristics are illustrated experimentally.Comment: This work has been accepted for publication in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensin

Image Processing and Machine Learning for Hyperspectral Unmixing: An Overview and the HySUPP Python Package

Spectral pixels are often a mixture of the pure spectra of the materials, called endmembers, due to the low spatial resolution of hyperspectral sensors, double scattering, and intimate mixtures of materials in the scenes. Unmixing estimates the fractional abundances of the endmembers within the pixel. Depending on the prior knowledge of endmembers, linear unmixing can be divided into three main groups: supervised, semi-supervised, and unsupervised (blind) linear unmixing. Advances in Image processing and machine learning substantially affected unmixing. This paper provides an overview of advanced and conventional unmixing approaches. Additionally, we draw a critical comparison between advanced and conventional techniques from the three categories. We compare the performance of the unmixing techniques on three simulated and two real datasets. The experimental results reveal the advantages of different unmixing categories for different unmixing scenarios. Moreover, we provide an open-source Python-based package available at https://github.com/BehnoodRasti/HySUPP to reproduce the results

Context dependent spectral unmixing.

A hyperspectral unmixing algorithm that finds multiple sets of endmembers is proposed. The algorithm, called Context Dependent Spectral Unmixing (CDSU), is a local approach that adapts the unmixing to different regions of the spectral space. It is based on a novel function that combines context identification and unmixing. This joint objective function models contexts as compact clusters and uses the linear mixing model as the basis for unmixing. Several variations of the CDSU, that provide additional desirable features, are also proposed. First, the Context Dependent Spectral unmixing using the Mahalanobis Distance (CDSUM) offers the advantage of identifying non-spherical clusters in the high dimensional spectral space. Second, the Cluster and Proportion Constrained Multi-Model Unmixing (CC-MMU and PC-MMU) algorithms use partial supervision information, in the form of cluster or proportion constraints, to guide the search process and narrow the space of possible solutions. The supervision information could be provided by an expert, generated by analyzing the consensus of multiple unmixing algorithms, or extracted from co-located data from a different sensor. Third, the Robust Context Dependent Spectral Unmixing (RCDSU) introduces possibilistic memberships into the objective function to reduce the effect of noise and outliers in the data. Finally, the Unsupervised Robust Context Dependent Spectral Unmixing (U-RCDSU) algorithm learns the optimal number of contexts in an unsupervised way. The performance of each algorithm is evaluated using synthetic and real data. We show that the proposed methods can identify meaningful and coherent contexts, and appropriate endmembers within each context. The second main contribution of this thesis is consensus unmixing. This approach exploits the diversity and similarity of the large number of existing unmixing algorithms to identify an accurate and consistent set of endmembers in the data. We run multiple unmixing algorithms using different parameters, and combine the resulting unmixing ensemble using consensus analysis. The extracted endmembers will be the ones that have a consensus among the multiple runs. The third main contribution consists of developing subpixel target detectors that rely on the proposed CDSU algorithms to adapt target detection algorithms to different contexts. A local detection statistic is computed for each context and then all scores are combined to yield a final detection score. The context dependent unmixing provides a better background description and limits target leakage, which are two essential properties for target detection algorithms

Improving Hyperspectral Subpixel Target Detection Using Hybrid Detection Space

A Hyper-Spectral Image (HSI) has high spectral and low spatial resolution. As a result, most targets exist as subpixels, which pose challenges in target detection. Moreover, limitation of target and background samples always hinders the target detection performance. In this thesis, a hybrid method for subpixel target detection of an HSI using minimal prior knowledge is developed. The Matched Filter (MF) and Adaptive Cosine Estimator (ACE) are two popular algorithms in HSI target detection. They have different advantages in differentiating target from background. In the proposed method, the scores of MF and ACE algorithms are used to construct a hybrid detection space. First, some high abundance target spectra are randomly picked from the scene to perform initial detection to determine the target and background subsets. Then, the reference target spectrum and background covariance matrix are improved iteratively, using the hybrid detection space. As the iterations continue, the reference target spectrum gets closer and closer to the central line that connects the centers of target and background and resulting in noticeable improvement in target detection. Two synthetic datasets and two real datasets are used in the experiments. The results are evaluated based on the mean detection rate, Receiver Operating Characteristic (ROC) curve and observation of the detection results. Compared to traditional MF and ACE algorithms with Reed-Xiaoli Detector (RXD) background covariance matrix estimation, the new method shows much better performance on all four datasets. This method can be applied in environmental monitoring, mineral detection, as well as oceanography and forestry reconnaissance to search for extremely small target distribution in a large scene

Inertia-Constrained Pixel-by-Pixel Nonnegative Matrix Factorisation: a Hyperspectral Unmixing Method Dealing with Intra-class Variability

Blind source separation is a common processing tool to analyse the constitution of pixels of hyperspectral images. Such methods usually suppose that pure pixel spectra (endmembers) are the same in all the image for each class of materials. In the framework of remote sensing, such an assumption is no more valid in the presence of intra-class variabilities due to illumination conditions, weathering, slight variations of the pure materials, etc... In this paper, we first describe the results of investigations highlighting intra-class variability measured in real images. Considering these results, a new formulation of the linear mixing model is presented leading to two new methods. Unconstrained Pixel-by-pixel NMF (UP-NMF) is a new blind source separation method based on the assumption of a linear mixing model, which can deal with intra-class variability. To overcome UP-NMF limitations an extended method is proposed, named Inertia-constrained Pixel-by-pixel NMF (IP-NMF). For each sensed spectrum, these extended versions of NMF extract a corresponding set of source spectra. A constraint is set to limit the spreading of each source's estimates in IP-NMF. The methods are tested on a semi-synthetic data set built with spectra extracted from a real hyperspectral image and then numerically mixed. We thus demonstrate the interest of our methods for realistic source variabilities. Finally, IP-NMF is tested on a real data set and it is shown to yield better performance than state of the art methods

Target Decomposition of Quad-Polarimetric SAR Images as an Unmixing Problem

Classic target decomposition methods use scattering space in their approaches. However, the goal for this project is to investigate whether a different approach to retrieve accurate and reliable estimates on the earth composition is possible when using the feature space with covariance matrix-based features. The approach consists of four steps. Generating multidimensional feature space data from sea ice scenes, extracting endmembers, finding the optimal number of endmembers in the scene and finding the contribution for the endmembers to each of the polarimetric feature pixels in the scene. In order to validate the performance of the approach several validation steps where conducted. Classification of the endmembers, calculating the average reconstruction error, classification of the scene and studding the abundance coefficients were some of these steps. Also, generation of synthetic data was conducted as an additional review of the approach. The system in this approach does not take in to account the variability of the polarimetric feature values in the different classes. It also assumes that the pixels are linearly mixed, something they probably not are. As a consequence, the approach is not able to retrieve accurate and reliable estimates on the earth composition for scenes consisting of sea ice. However, the approach gave good results on the synthetic datasets. Further work and investigation on the approach would include adapting the approach to consider the variability all sea ice data suffers from. Further, the methods considering linear mixing should then be replaced with methods considering nonlinear mixing