Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches

Imaging spectrometers measure electromagnetic energy scattered in their instantaneous field view in hundreds or thousands of spectral channels with higher spectral resolution than multispectral cameras. Imaging spectrometers are therefore often referred to as hyperspectral cameras (HSCs). Higher spectral resolution enables material identification via spectroscopic analysis, which facilitates countless applications that require identifying materials in scenarios unsuitable for classical spectroscopic analysis. Due to low spatial resolution of HSCs, microscopic material mixing, and multiple scattering, spectra measured by HSCs are mixtures of spectra of materials in a scene. Thus, accurate estimation requires unmixing. Pixels are assumed to be mixtures of a few materials, called endmembers. Unmixing involves estimating all or some of: the number of endmembers, their spectral signatures, and their abundances at each pixel. Unmixing is a challenging, ill-posed inverse problem because of model inaccuracies, observation noise, environmental conditions, endmember variability, and data set size. Researchers have devised and investigated many models searching for robust, stable, tractable, and accurate unmixing algorithms. This paper presents an overview of unmixing methods from the time of Keshava and Mustard's unmixing tutorial [1] to the present. Mixing models are first discussed. Signal-subspace, geometrical, statistical, sparsity-based, and spatial-contextual unmixing algorithms are described. Mathematical problems and potential solutions are described. Algorithm characteristics are illustrated experimentally.Comment: This work has been accepted for publication in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensin

Dictionary-based Tensor Canonical Polyadic Decomposition

To ensure interpretability of extracted sources in tensor decomposition, we introduce in this paper a dictionary-based tensor canonical polyadic decomposition which enforces one factor to belong exactly to a known dictionary. A new formulation of sparse coding is proposed which enables high dimensional tensors dictionary-based canonical polyadic decomposition. The benefits of using a dictionary in tensor decomposition models are explored both in terms of parameter identifiability and estimation accuracy. Performances of the proposed algorithms are evaluated on the decomposition of simulated data and the unmixing of hyperspectral images

Bounded Simplex-Structured Matrix Factorization: Algorithms, Identifiability and Applications

In this paper, we propose a new low-rank matrix factorization model dubbed bounded simplex-structured matrix factorization (BSSMF). Given an input matrix $X$ and a factorization rank $r$, BSSMF looks for a matrix $W$ with $r$ columns and a matrix $H$ with $r$ rows such that $X \approx WH$ where the entries in each column of $W$ are bounded, that is, they belong to given intervals, and the columns of $H$ belong to the probability simplex, that is, $H$ is column stochastic. BSSMF generalizes nonnegative matrix factorization (NMF), and simplex-structured matrix factorization (SSMF). BSSMF is particularly well suited when the entries of the input matrix $X$ belong to a given interval; for example when the rows of $X$ represent images, or $X$ is a rating matrix such as in the Netflix and MovieLens datasets where the entries of $X$ belong to the interval $[1,5]$. The simplex-structured matrix $H$ not only leads to an easily understandable decomposition providing a soft clustering of the columns of $X$, but implies that the entries of each column of $WH$ belong to the same intervals as the columns of $W$. In this paper, we first propose a fast algorithm for BSSMF, even in the presence of missing data in $X$. Then we provide identifiability conditions for BSSMF, that is, we provide conditions under which BSSMF admits a unique decomposition, up to trivial ambiguities. Finally, we illustrate the effectiveness of BSSMF on two applications: extraction of features in a set of images, and the matrix completion problem for recommender systems.Comment: 14 pages, new title, new numerical experiments on synthetic data, clarifications of several parts of the paper, run times adde

Bayesian fusion of multi-band images : A powerful tool for super-resolution

Hyperspectral (HS) imaging, which consists of acquiring a same scene in several hundreds of contiguous spectral bands (a three dimensional data cube), has opened a new range of relevant applications, such as target detection [MS02], classification [C.-03] and spectral unmixing [BDPD+12]. However, while HS sensors provide abundant spectral information, their spatial resolution is generally more limited. Thus, fusing the HS image with other highly resolved images of the same scene, such as multispectral (MS) or panchromatic (PAN) images is an interesting problem. The problem of fusing a high spectral and low spatial resolution image with an auxiliary image of higher spatial but lower spectral resolution, also known as multi-resolution image fusion, has been explored for many years [AMV+11]. From an application point of view, this problem is also important as motivated by recent national programs, e.g., the Japanese next-generation space-borne hyperspectral image suite (HISUI), which fuses co-registered MS and HS images acquired over the same scene under the same conditions [YI13]. Bayesian fusion allows for an intuitive interpretation of the fusion process via the posterior distribution. Since the fusion problem is usually ill-posed, the Bayesian methodology offers a convenient way to regularize the problem by defining appropriate prior distribution for the scene of interest. The aim of this thesis is to study new multi-band image fusion algorithms to enhance the resolution of hyperspectral image. In the first chapter, a hierarchical Bayesian framework is proposed for multi-band image fusion by incorporating forward model, statistical assumptions and Gaussian prior for the target image to be restored. To derive Bayesian estimators associated with the resulting posterior distribution, two algorithms based on Monte Carlo sampling and optimization strategy have been developed. In the second chapter, a sparse regularization using dictionaries learned from the observed images is introduced as an alternative of the naive Gaussian prior proposed in Chapter 1. instead of Gaussian prior is introduced to regularize the ill-posed problem. Identifying the supports jointly with the dictionaries circumvented the difficulty inherent to sparse coding. To minimize the target function, an alternate optimization algorithm has been designed, which accelerates the fusion process magnificently comparing with the simulation-based method. In the third chapter, by exploiting intrinsic properties of the blurring and downsampling matrices, a much more efficient fusion method is proposed thanks to a closed-form solution for the Sylvester matrix equation associated with maximizing the likelihood. The proposed solution can be embedded into an alternating direction method of multipliers or a block coordinate descent method to incorporate different priors or hyper-priors for the fusion problem, allowing for Bayesian estimators. In the last chapter, a joint multi-band image fusion and unmixing scheme is proposed by combining the well admitted linear spectral mixture model and the forward model. The joint fusion and unmixing problem is solved in an alternating optimization framework, mainly consisting of solving a Sylvester equation and projecting onto a simplex resulting from the non-negativity and sum-to-one constraints. The simulation results conducted on synthetic and semi-synthetic images illustrate the advantages of the developed Bayesian estimators, both qualitatively and quantitatively

Nonlinear unmixing of Hyperspectral images

Le démélange spectral est un des sujets majeurs de l’analyse d’images hyperspectrales. Ce problème consiste à identifier les composants macroscopiques présents dans une image hyperspectrale et à quantifier les proportions (ou abondances) de ces matériaux dans tous les pixels de l’image. La plupart des algorithmes de démélange suppose un modèle de mélange linéaire qui est souvent considéré comme une approximation au premier ordre du mélange réel. Cependant, le modèle linéaire peut ne pas être adapté pour certaines images associées par exemple à des scènes engendrant des trajets multiples (forêts, zones urbaines) et des modèles non-linéaires plus complexes doivent alors être utilisés pour analyser de telles images. Le but de cette thèse est d’étudier de nouveaux modèles de mélange non-linéaires et de proposer des algorithmes associés pour l’analyse d’images hyperspectrales. Dans un premier temps, un modèle paramétrique post-non-linéaire est étudié et des algorithmes d’estimation basés sur ce modèle sont proposés. Les connaissances a priori disponibles sur les signatures spectrales des composants purs, sur les abondances et les paramètres de la non-linéarité sont exploitées à l’aide d’une approche bayesienne. Le second modèle étudié dans cette thèse est basé sur l’approximation de la variété non-linéaire contenant les données observées à l’aide de processus gaussiens. L’algorithme de démélange associé permet d’estimer la relation non-linéaire entre les abondances des matériaux et les pixels observés sans introduire explicitement les signatures spectrales des composants dans le modèle de mélange. Ces signatures spectrales sont estimées dans un second temps par prédiction à base de processus gaussiens. La prise en compte d’effets non-linéaires dans les images hyperspectrales nécessite souvent des stratégies de démélange plus complexes que celles basées sur un modèle linéaire. Comme le modèle linéaire est souvent suffisant pour approcher la plupart des mélanges réels, il est intéressant de pouvoir détecter les pixels ou les régions de l’image où ce modèle linéaire est approprié. On pourra alors, après cette détection, appliquer les algorithmes de démélange non-linéaires aux pixels nécessitant réellement l’utilisation de modèles de mélange non-linéaires. La dernière partie de ce manuscrit se concentre sur l’étude de détecteurs de non-linéarités basés sur des modèles linéaires et non-linéaires pour l’analyse d’images hyperspectrales. Les méthodes de démélange non-linéaires proposées permettent d’améliorer la caractérisation des images hyperspectrales par rapport au méthodes basées sur un modèle linéaire. Cette amélioration se traduit en particulier par une meilleure erreur de reconstruction des données. De plus, ces méthodes permettent de meilleures estimations des signatures spectrales et des abondances quand les pixels résultent de mélanges non-linéaires. Les résultats de simulations effectuées sur des données synthétiques et réelles montrent l’intérêt d’utiliser des méthodes de détection de non-linéarités pour l’analyse d’images hyperspectrales. En particulier, ces détecteurs peuvent permettre d’identifier des composants très peu représentés et de localiser des régions où les effets non-linéaires sont non-négligeables (ombres, reliefs,...). Enfin, la considération de corrélations spatiales dans les images hyperspectrales peut améliorer les performances des algorithmes de démélange non-linéaires et des détecteurs de non-linéarités. ABSTRACT : Spectral unmixing is one the major issues arising when analyzing hyperspectral images. It consists of identifying the macroscopic materials present in a hyperspectral image and quantifying the proportions of these materials in the image pixels. Most unmixing techniques rely on a linear mixing model which is often considered as a first approximation of the actual mixtures. However, the linear model can be inaccurate for some specific images (for instance images of scenes involving multiple reflections) and more complex nonlinear models must then be considered to analyze such images. The aim of this thesis is to study new nonlinear mixing models and to propose associated algorithms to analyze hyperspectral images. First, a ost-nonlinear model is investigated and efficient unmixing algorithms based on this model are proposed. The prior knowledge about the components present in the observed image, their proportions and the nonlinearity parameters is considered using Bayesian inference. The second model considered in this work is based on the approximation of the nonlinear manifold which contains the observed pixels using Gaussian processes. The proposed algorithm estimates the relation between the observations and the unknown material proportions without explicit dependency on the material spectral signatures, which are estimated subsequentially. Considering nonlinear effects in hyperspectral images usually requires more complex unmixing strategies than those assuming linear mixtures. Since the linear mixing model is often sufficient to approximate accurately most actual mixtures, it is interesting to detect pixels or regions where the linear model is accurate. This nonlinearity detection can be applied as a pre-processing step and nonlinear unmixing strategies can then be applied only to pixels requiring the use of nonlinear models. The last part of this thesis focuses on new nonlinearity detectors based on linear and nonlinear models to identify pixels or regions where nonlinear effects occur in hyperspectral images. The proposed nonlinear unmixing algorithms improve the characterization of hyperspectral images compared to methods based on a linear model. These methods allow the reconstruction errors to be reduced. Moreover, these methods provide better spectral signature and abundance estimates when the observed pixels result from nonlinear mixtures. The simulation results conducted on synthetic and real images illustrate the advantage of using nonlinearity detectors for hyperspectral image analysis. In particular, the proposed detectors can identify components which are present in few pixels (and hardly distinguishable) and locate areas where significant nonlinear effects occur (shadow, relief, ...). Moreover, it is shown that considering spatial correlation in hyperspectral images can improve the performance of nonlinear unmixing and nonlinearity detection algorithms

Robust Distributed Multi-Source Detection and Labeling in Wireless Acoustic Sensor Networks

The growing demand in complex signal processing methods associated with low-energy large scale wireless acoustic sensor networks (WASNs) urges the shift to a new information and communication technologies (ICT) paradigm. The emerging research perception aspires for an appealing wireless network communication where multiple heterogeneous devices with different interests can cooperate in various signal processing tasks (MDMT). Contributions in this doctoral thesis focus on distributed multi-source detection and labeling applied to audio enhancement scenarios pursuing an MDMT fashioned node-specific source-of-interest signal enhancement in WASNs. In fact, an accurate detection and labeling is a pre-requisite to pursue the MDMT paradigm where nodes in the WASN communicate effectively their sources-of-interest and, therefore, multiple signal processing tasks can be enhanced via cooperation. First, a novel framework based on a dominant source model in distributed WASNs for resolving the activity detection of multiple speech sources in a reverberant and noisy environment is introduced. A preliminary rank-one multiplicative non-negative independent component analysis (M-NICA) for unique dominant energy source extraction given associated node clusters is presented. Partitional algorithms that minimize the within-cluster mean absolute deviation (MAD) and weighted MAD objectives are proposed to determine the cluster membership of the unmixed energies, and thus establish a source specific voice activity recognition. In a second study, improving the energy signal separation to alleviate the multiple source activity discrimination task is targeted. Sparsity inducing penalties are enforced on iterative rank-one singular value decomposition layers to extract sparse right rotations. Then, sparse non-negative blind energy separation is realized using multiplicative updates. Hence, the multiple source detection problem is converted into a sparse non-negative source energy decorrelation. Sparsity tunes the supposedly non-active energy signatures to exactly zero-valued energies so that it is easier to identify active energies and an activity detector can be constructed in a straightforward manner. In a centralized scenario, the activity decision is controlled by a fusion center that delivers the binary source activity detection for every participating energy source. This strategy gives precise detection results for small source numbers. With a growing number of interfering sources, the distributed detection approach is more promising. Conjointly, a robust distributed energy separation algorithm for multiple competing sources is proposed. A robust and regularized $t_{\nu}M$-estimation of the covariance matrix of the mixed energies is employed. This approach yields a simple activity decision using only the robustly unmixed energy signatures of the sources in the WASN. The performance of the robust activity detector is validated with a distributed adaptive node-specific signal estimation method for speech enhancement. The latter enhances the quality and intelligibility of the signal while exploiting the accurately estimated multi-source voice decision patterns. In contrast to the original M-NICA for source separation, the extracted binary activity patterns with the robust energy separation significantly improve the node-specific signal estimation. Due to the increased computational complexity caused by the additional step of energy signal separation, a new approach to solving the detection question of multi-device multi-source networks is presented. Stability selection for iterative extraction of robust right singular vectors is considered. The sub-sampling selection technique provides transparency in properly choosing the regularization variable in the Lasso optimization problem. In this way, the strongest sparse right singular vectors using a robust $\ell_1$-norm and stability selection are the set of basis vectors that describe the input data efficiently. Active/non-active source classification is achieved based on a robust Mahalanobis classifier. For this, a robust $M$-estimator of the covariance matrix in the Mahalanobis distance is utilized. Extensive evaluation in centralized and distributed settings is performed to assess the effectiveness of the proposed approach. Thus, overcoming the computationally demanding source separation scheme is possible via exploiting robust stability selection for sparse multi-energy feature extraction. With respect to the labeling problem of various sources in a WASN, a robust approach is introduced that exploits the direction-of-arrival of the impinging source signals. A short-time Fourier transform-based subspace method estimates the angles of locally stationary wide band signals using a uniform linear array. The median of angles estimated at every frequency bin is utilized to obtain the overall angle for each participating source. The features, in this case, exploit the similarity across devices in the particular frequency bins that produce reliable direction-of-arrival estimates for each source. Reliability is defined with respect to the median across frequencies. All source-specific frequency bands that contribute to correct estimated angles are selected. A feature vector is formed for every source at each device by storing the frequency bin indices that lie within the upper and lower interval of the median absolute deviation scale of the estimated angle. Labeling is accomplished by a distributed clustering of the extracted angle-based feature vectors using consensus averaging

Méthodes de séparation aveugle de sources pour l'imagerie hyperspectrale : application à la télédétection urbaine et à l'astrophysique

Au cours de cette thèse nous avons développé des méthodes de Séparation Aveugle de Sources (SAS) pour des images hyperspectrales, dans le cadre de deux champs d'application : la télédétection urbaine et l'astrophysique. Dans la première partie de la thèse nous nous sommes intéressés au démélange hyperspectral pour des images urbaines, le but étant de retrouver d'une manière non supervisée les matériaux présents sur la scène en extrayant leurs spectres et leurs proportions. La plupart des méthodes de la littérature sont basées sur un modèle linéaire, qui n'est pas valide en milieu urbain à cause des structures 3D. Une première étape a donc été d'établir un modèle de mélange adapté aux milieux urbains, en partant d'équations physiques basées sur la théorie du transfert radiatif. Le modèle final de forme linéaire quadratique invariant spectralement, ainsi que les possibles hypothèses sur les coefficients de mélange, sont justifiés par les résultats obtenus sur des images simulées réalistes. Nous avons ensuite proposé, pour le démélange, des méthodes de SAS fondées sur la FMN (Factorisation en Matrices Non-négatives). Ces méthodes sont basées sur un calcul de gradient qui tient compte des termes quadratiques. La première méthode utilise un algorithme de gradient à pas fixe, à partir de laquelle une version de Newton a aussi été proposée. La dernière méthode est un algorithme FMN multiplicatif. Les méthodes proposées donnent de meilleures performances qu'une méthode linéaire de la littérature. En astrophysique nous avons développé des méthodes de SAS pour des images de champs denses d'étoiles du spectro-imageur MUSE. A cause de la PSF (Point Spread Function), les informations contenues dans les pixels peuvent résulter des contributions de plusieurs étoiles. C'est là que réside l'intérêt de la SAS : extraire, à partir de ces signaux qui sont des mélanges, les spectres des étoiles qui sont donc nos "sources". Le modèle de mélange est linéaire non invariant spectralement. Nous avons proposé une méthode de SAS basée sur la positivité des données. Cette approche exploite le modèle paramétrique de la FSF (Field Spread Function) de MUSE. La méthode mise en place est itérative et alterne l'estimation des spectres par moindres carrés (avec contraintes de positivité) et estimation des paramètres de la FSF par un algorithme de gradient projeté. La méthode proposée donne de bonnes performances sur des images simulées de MUSE.In this work, we developed Blind Source Separation methods (BSS) for hyperspectral images, concerning two applications : urban remote sensing and astrophysics. The first part of this work concerned spectral unmixing for urban images, with the aim of finding, by an unsupervised method, the materials present in the scene, by extracting their spectra and their proportions. Most existing methods rely on a linear model, which is not valid in urban environments because of 3D structures. Therefore, the first step was to derive a mixing model adapted to urban environments, starting from physical equations based on radiative transfer theory. The derived linear-quadratic model, and possible hypotheses on the mixing coefficients, are justified by results obtained with simulated realistic images. We then proposed, for the unmixing, BSS methods based on NMF (Non-negative Matrix Factorization). These methods are based on gradient computation taking into account the quadratic terms.The first method uses a gradient descent algorithm with a constant step, from which we then derived a Newton version. The last proposed method is a multiplicative NMF algorithm. These methods give better performance than a linear method from the literature. Concerning astrophysics, we developed BSS methods for dense field images of the MUSE instrument. Due to the PSF (Point Spread Function) effect, information contained in the pixels can result from contributions of many stars. Hence, there is a need for BSS, to extract from these signals that are mixtures, the star spectra which are our "sources". The mixing model is linear but spectrally non-invariant. We proposed a BSS method based on positivity. This approach uses the parametric model of MUSE FSF (Field Spread Function). The implemented method is iterative and alternates spectra estimation using least squares (with positivity constraint) and FSF parameter estimation by a projected gradient descent algorithm. The proposed method yields good performance with simulated MUSE images

Contributions to probabilistic non-negative matrix factorization - Maximum marginal likelihood estimation and Markovian temporal models

Non-negative matrix factorization (NMF) has become a popular dimensionality reductiontechnique, and has found applications in many different fields, such as audio signal processing,hyperspectral imaging, or recommender systems. In its simplest form, NMF aims at finding anapproximation of a non-negative data matrix (i.e., with non-negative entries) as the product of twonon-negative matrices, called the factors. One of these two matrices can be interpreted as adictionary of characteristic patterns of the data, and the other one as activation coefficients ofthese patterns. This low-rank approximation is traditionally retrieved by optimizing a measure of fitbetween the data matrix and its approximation. As it turns out, for many choices of measures of fit,the problem can be shown to be equivalent to the joint maximum likelihood estimation of thefactors under a certain statistical model describing the data. This leads us to an alternativeparadigm for NMF, where the learning task revolves around probabilistic models whoseobservation density is parametrized by the product of non-negative factors. This general framework, coined probabilistic NMF, encompasses many well-known latent variable models ofthe literature, such as models for count data. In this thesis, we consider specific probabilistic NMFmodels in which a prior distribution is assumed on the activation coefficients, but the dictionary remains a deterministic variable. The objective is then to maximize the marginal likelihood in thesesemi-Bayesian NMF models, i.e., the integrated joint likelihood over the activation coefficients.This amounts to learning the dictionary only; the activation coefficients may be inferred in asecond step if necessary. We proceed to study in greater depth the properties of this estimation process. In particular, two scenarios are considered. In the first one, we assume the independence of the activation coefficients sample-wise. Previous experimental work showed that dictionarieslearned with this approach exhibited a tendency to automatically regularize the number of components, a favorable property which was left unexplained. In the second one, we lift thisstandard assumption, and consider instead Markov structures to add statistical correlation to themodel, in order to better analyze temporal data