Nonlinear unmixing of hyperspectral images using a semiparametric model and spatial regularization

Incorporating spatial information into hyperspectral unmixing procedures has been shown to have positive effects, due to the inherent spatial-spectral duality in hyperspectral scenes. Current research works that consider spatial information are mainly focused on the linear mixing model. In this paper, we investigate a variational approach to incorporating spatial correlation into a nonlinear unmixing procedure. A nonlinear algorithm operating in reproducing kernel Hilbert spaces, associated with an $\ell_1$ local variation norm as the spatial regularizer, is derived. Experimental results, with both synthetic and real data, illustrate the effectiveness of the proposed scheme.Comment: 5 pages, 1 figure, submitted to ICASSP 201

Nonlinear unmixing of hyperspectral images: Models and algorithms

When considering the problem of unmixing hyperspectral images, most of the literature in the geoscience and image processing areas relies on the widely used linear mixing model (LMM). However, the LMM may be not valid, and other nonlinear models need to be considered, for instance, when there are multiscattering effects or intimate interactions. Consequently, over the last few years, several significant contributions have been proposed to overcome the limitations inherent in the LMM. In this article, we present an overview of recent advances in nonlinear unmixing modeling

Manifold learning based spectral unmixing of hyperspectral remote sensing data

Nonlinear mixing effects inherent in hyperspectral data are not properly represented in linear spectral unmixing models. Although direct nonlinear unmixing models provide capability to capture nonlinear phenomena, they are difficult to formulate and the results are not always generalizable. Manifold learning based spectral unmixing accommodates nonlinearity in the data in the feature extraction stage followed by linear mixing, thereby incorporating some characteristics of nonlinearity while retaining advantages of linear unmixing approaches. Since endmember selection is critical to successful spectral unmixing, it is important to select proper endmembers from the manifold space. However, excessive computational burden hinders development of manifolds for large-scale remote sensing datasets. This dissertation addresses issues related to high computational overhead requirements of manifold learning for developing representative manifolds for the spectral unmixing task. Manifold approximations using landmarks are popular for mitigating the computational complexity of manifold learning. A new computationally effective landmark selection method that exploits spatial redundancy in the imagery is proposed. A robust, less costly landmark set with low spectral and spatial redundancy is successfully incorporated with a hybrid manifold which shares properties of both global and local manifolds. While landmark methods reduce computational demand, the resulting manifolds may not represent subtle features of the manifold adequately. Active learning heuristics are introduced to increase the number of landmarks, with the goal of developing more representative manifolds for spectral unmixing. By communicating between the landmark set and the query criteria relative to spectral unmixing, more representative and stable manifolds with less spectrally and spatially redundant landmarks are developed. A new ranking method based on the pixels with locally high spectral variability within image subsets and convex-geometry finds a solution more quickly and precisely. Experiments were conducted to evaluate the proposed methods using the AVIRIS Cuprite hyperspectral reference dataset. A case study of manifold learning based spectral unmixing in agricultural areas is included in the dissertation.Remotely sensed data collected by airborne or spaceborne sensors are utilized to quantify crop residue cover over an extensive area. Although remote sensing indices are popular for characterizing residue amounts, they are not effective with noisy Hyperion data because the effect of residual striping artifacts is amplified in ratios involving band differences. In this case study, spectral unmixing techniques are investigated for estimating crop residue as an alternative approach to empirical models developed using band based indices. The spectral unmixing techniques, and especially the manifold learning approaches, provide more robust, lower RMSE estimates for crop residue cover than the hyperspectral index based method for Hyperion data

Nonparametric Detection of Nonlinearly Mixed Pixels and Endmember Estimation in Hyperspectral Images

International audienceMixing phenomena in hyperspectral images depend on a variety of factors, such as the resolution of observation devices, the properties of materials, and how these materials interact with incident light in the scene. Different parametric and nonparametric models have been considered to address hyperspectral unmixing problems. The simplest one is the linear mixing model. Nevertheless, it has been recognized that the mixing phenomena can also be nonlinear. The corresponding nonlinear analysis techniques are necessarily more challenging and complex than those employed for linear unmixing. Within this context, it makes sense to detect the nonlinearly mixed pixels in an image prior to its analysis, and then employ the simplest possible unmixing technique to analyze each pixel. In this paper, we propose a technique for detecting nonlinearly mixed pixels. The detection approach is based on the comparison of the reconstruction errors using both a Gaussian process regression model and a linear regression model. The two errors are combined into a detection statistics for which a probability density function can be reasonably approximated. We also propose an iterative endmember extraction algorithm to be employed in combination with the detection algorithm. The proposed detect-then-unmix strategy, which consists of extracting endmembers, detecting nonlinearly mixed pixels and unmixing, is tested with synthetic and real images

EndNet: Sparse AutoEncoder Network for Endmember Extraction and Hyperspectral Unmixing

Data acquired from multichannel sensors are a highly valuable asset to interpret the environment for a variety of remote sensing applications. However, low spatial resolution is a critical limitation for previous sensors, and the constituent materials of a scene can be mixed in different fractions due to their spatial interactions. Spectral unmixing is a technique that allows us to obtain the material spectral signatures and their fractions from hyperspectral data. In this paper, we propose a novel endmember extraction and hyperspectral unmixing scheme, so-called EndNet, that is based on a two-staged autoencoder network. This well-known structure is completely enhanced and restructured by introducing additional layers and a projection metric [i.e., spectral angle distance (SAD) instead of inner product] to achieve an optimum solution. Moreover, we present a novel loss function that is composed of a Kullback-Leibler divergence term with SAD similarity and additional penalty terms to improve the sparsity of the estimates. These modifications enable us to set the common properties of endmembers, such as nonlinearity and sparsity for autoencoder networks. Finally, due to the stochastic-gradient-based approach, the method is scalable for large-scale data and it can be accelerated on graphical processing units. To demonstrate the superiority of our proposed method, we conduct extensive experiments on several well-known data sets. The results confirm that the proposed method considerably improves the performance compared to the state-of-the-art techniques in the literature

Characterization and Reduction of Noise in Manifold Representations of Hyperspectral Imagery

A new workflow to produce dimensionality reduced manifold coordinates based on the improvements of landmark Isometric Mapping (ISOMAP) algorithms using local spectral models is proposed. Manifold space from nonlinear dimensionality reduction better addresses the nonlinearity of the hyperspectral data and often has better per- formance comparing to the results of linear methods such as Minimum Noise Fraction (MNF). The dissertation mainly focuses on using adaptive local spectral models to fur- ther improve the performance of ISOMAP algorithms by addressing local noise issues and perform guided landmark selection and nearest neighborhood construction in local spectral subsets. This work could benefit the performance of common hyperspectral image analysis tasks, such as classification, target detection, etc., but also keep the computational burden low. This work is based on and improves the previous ENH- ISOMAP algorithm in various ways. The workflow is based on a unified local spectral subsetting framework. Embedding spaces in local spectral subsets as local noise models are first proposed and used to perform noise estimation, MNF regression and guided landmark selection in a local sense. Passive and active methods are proposed and ver- ified to select landmarks deliberately to ensure local geometric structure coverage and local noise avoidance. Then, a novel local spectral adaptive method is used to construct the k-nearest neighbor graph. Finally, a global MNF transformation in the manifold space is also introduced to further compress the signal dimensions. The workflow is implemented using C++ with multiple implementation optimizations, including using heterogeneous computing platforms that are available in personal computers. The re- sults are presented and evaluated by Jeffries-Matsushita separability metric, as well as the classification accuracy of supervised classifiers. The proposed workflow shows sig- nificant and stable improvements over the dimensionality reduction performance from traditional MNF and ENH-ISOMAP on various hyperspectral datasets. The computa- tional speed of the proposed implementation is also improved

Nonlinear unmixing of Hyperspectral images

Le démélange spectral est un des sujets majeurs de l’analyse d’images hyperspectrales. Ce problème consiste à identifier les composants macroscopiques présents dans une image hyperspectrale et à quantifier les proportions (ou abondances) de ces matériaux dans tous les pixels de l’image. La plupart des algorithmes de démélange suppose un modèle de mélange linéaire qui est souvent considéré comme une approximation au premier ordre du mélange réel. Cependant, le modèle linéaire peut ne pas être adapté pour certaines images associées par exemple à des scènes engendrant des trajets multiples (forêts, zones urbaines) et des modèles non-linéaires plus complexes doivent alors être utilisés pour analyser de telles images. Le but de cette thèse est d’étudier de nouveaux modèles de mélange non-linéaires et de proposer des algorithmes associés pour l’analyse d’images hyperspectrales. Dans un premier temps, un modèle paramétrique post-non-linéaire est étudié et des algorithmes d’estimation basés sur ce modèle sont proposés. Les connaissances a priori disponibles sur les signatures spectrales des composants purs, sur les abondances et les paramètres de la non-linéarité sont exploitées à l’aide d’une approche bayesienne. Le second modèle étudié dans cette thèse est basé sur l’approximation de la variété non-linéaire contenant les données observées à l’aide de processus gaussiens. L’algorithme de démélange associé permet d’estimer la relation non-linéaire entre les abondances des matériaux et les pixels observés sans introduire explicitement les signatures spectrales des composants dans le modèle de mélange. Ces signatures spectrales sont estimées dans un second temps par prédiction à base de processus gaussiens. La prise en compte d’effets non-linéaires dans les images hyperspectrales nécessite souvent des stratégies de démélange plus complexes que celles basées sur un modèle linéaire. Comme le modèle linéaire est souvent suffisant pour approcher la plupart des mélanges réels, il est intéressant de pouvoir détecter les pixels ou les régions de l’image où ce modèle linéaire est approprié. On pourra alors, après cette détection, appliquer les algorithmes de démélange non-linéaires aux pixels nécessitant réellement l’utilisation de modèles de mélange non-linéaires. La dernière partie de ce manuscrit se concentre sur l’étude de détecteurs de non-linéarités basés sur des modèles linéaires et non-linéaires pour l’analyse d’images hyperspectrales. Les méthodes de démélange non-linéaires proposées permettent d’améliorer la caractérisation des images hyperspectrales par rapport au méthodes basées sur un modèle linéaire. Cette amélioration se traduit en particulier par une meilleure erreur de reconstruction des données. De plus, ces méthodes permettent de meilleures estimations des signatures spectrales et des abondances quand les pixels résultent de mélanges non-linéaires. Les résultats de simulations effectuées sur des données synthétiques et réelles montrent l’intérêt d’utiliser des méthodes de détection de non-linéarités pour l’analyse d’images hyperspectrales. En particulier, ces détecteurs peuvent permettre d’identifier des composants très peu représentés et de localiser des régions où les effets non-linéaires sont non-négligeables (ombres, reliefs,...). Enfin, la considération de corrélations spatiales dans les images hyperspectrales peut améliorer les performances des algorithmes de démélange non-linéaires et des détecteurs de non-linéarités. ABSTRACT : Spectral unmixing is one the major issues arising when analyzing hyperspectral images. It consists of identifying the macroscopic materials present in a hyperspectral image and quantifying the proportions of these materials in the image pixels. Most unmixing techniques rely on a linear mixing model which is often considered as a first approximation of the actual mixtures. However, the linear model can be inaccurate for some specific images (for instance images of scenes involving multiple reflections) and more complex nonlinear models must then be considered to analyze such images. The aim of this thesis is to study new nonlinear mixing models and to propose associated algorithms to analyze hyperspectral images. First, a ost-nonlinear model is investigated and efficient unmixing algorithms based on this model are proposed. The prior knowledge about the components present in the observed image, their proportions and the nonlinearity parameters is considered using Bayesian inference. The second model considered in this work is based on the approximation of the nonlinear manifold which contains the observed pixels using Gaussian processes. The proposed algorithm estimates the relation between the observations and the unknown material proportions without explicit dependency on the material spectral signatures, which are estimated subsequentially. Considering nonlinear effects in hyperspectral images usually requires more complex unmixing strategies than those assuming linear mixtures. Since the linear mixing model is often sufficient to approximate accurately most actual mixtures, it is interesting to detect pixels or regions where the linear model is accurate. This nonlinearity detection can be applied as a pre-processing step and nonlinear unmixing strategies can then be applied only to pixels requiring the use of nonlinear models. The last part of this thesis focuses on new nonlinearity detectors based on linear and nonlinear models to identify pixels or regions where nonlinear effects occur in hyperspectral images. The proposed nonlinear unmixing algorithms improve the characterization of hyperspectral images compared to methods based on a linear model. These methods allow the reconstruction errors to be reduced. Moreover, these methods provide better spectral signature and abundance estimates when the observed pixels result from nonlinear mixtures. The simulation results conducted on synthetic and real images illustrate the advantage of using nonlinearity detectors for hyperspectral image analysis. In particular, the proposed detectors can identify components which are present in few pixels (and hardly distinguishable) and locate areas where significant nonlinear effects occur (shadow, relief, ...). Moreover, it is shown that considering spatial correlation in hyperspectral images can improve the performance of nonlinear unmixing and nonlinearity detection algorithms

Nonlinear hyperspectral unmixing: strategies for nonlinear mixture detection, endmember estimation and band-selection

Tese (doutorado) - Universidade Federal de Santa Catarina, Centro Tecnológico, Programa de Pós-Graduação em Engenharia Elétrica, Florianópolis, 2016.Abstract : Mixing phenomena in hyperspectral images depend on a variety of factors such as the resolution of observation devices, the properties of materials, and how these materials interact with incident light in the scene. Different parametric and nonparametric models have been considered to address hyperspectral unmixing problems. The simplest one is the linear mixing model. Nevertheless, it has been recognized that mixing phenomena can also be nonlinear. Kernel-based nonlinear mixing models have been applied to unmix spectral information of hyperspectral images when the type of mixing occurring in the scene is too complex or unknown. However, the corresponding nonlinear analysis techniques are necessarily more challenging and complex than those employed for linear unmixing. Within this context, it makes sense to search for different strategies to produce simpler and/or more accurate results. In this thesis, we tackle three distinct parts of the complete spectral unmixing (SU) problem. First, we propose a technique for detecting nonlinearly mixed pixels. The detection approach is based on the comparison of the reconstruction errors using both a Gaussian process regression model and a linear regression model. The two errors are combined into a detection test statistics for which a probability density function can be reasonably approximated. Second, we propose an iterative endmember extraction algorithm to be employed in combination with the detection algorithm. The proposed detect-then-unmix strategy, which consists of extracting endmembers, detecting nonlinearly mixed pixels and unmixing, is tested with synthetic and real images. Finally, we propose two methods for band selection (BS) in the reproducing kernel Hilbert space (RKHS), which lead to a significant reduction of the processing time required by nonlinear unmixing techniques. The first method employs the kernel k-means (KKM) algorithm to find clusters in the RKHS. Each cluster centroid is then associated to the closest mapped spectral vector. The second method is centralized, and it is based upon the coherence criterion, which sets the largest value allowed for correlations between the basis kernel functions characterizing the unmixing model. We show that the proposed BS approach is equivalent to solving a maximum clique problem (MCP), that is, to searching for the largest complete subgraph in a graph. Furthermore, we devise a strategy for selecting the coherence threshold and the Gaussian kernel bandwidth using coherence bounds for linearly independent bases. Simulation results illustrate the efficiency of the proposed method.Imagem hiperespectral (HI) é uma imagem em que cada pixel contém centenas (ou até milhares) de bandas estreitas e contíguas amostradas num amplo domínio do espectro eletromagnético. Sensores hiperespectrais normalmente trocam resolução espacial por resolução espectral devido principalmente a fatores como a distância entre o instrumento e a cena alvo, e limitada capacidade de processamento, transmissão e armazenamento históricas, mas que se tornam cada vez menos problemáticas. Este tipo de imagem encontra ampla utilização em uma gama de aplicações em astronomia, agricultura, imagens biomédicas, geociências, física, vigilância e sensoriamento remoto. A usual baixa resolução espacial de sensores espectrais implica que o que se observa em cada pixel é normalmente uma mistura das assinaturas espectrais dos materiais presentes na cena correspondente (normalmente denominados de endmembers). Assim um pixel em uma imagem hiperespectral não pode mais ser determinado por um tom ou cor mas sim por uma assinatura espectral do material, ou materiais, que se encontram na região analisada. O modelo mais simples e amplamente utilizado em aplicações com imagens hiperespectrais é o modelo linear, no qual o pixel observado é modelado como uma combinação linear dos endmembers. No entanto, fortes evidências de múltiplas reflexões da radiação solar e/ou materiais intimamente misturados, i.e., misturados em nível microscópico, resultam em diversos modelos não-lineares dos quais destacam-se os modelos bilineares, modelos de pós não-linearidade, modelos de mistura íntima e modelos não-paramétricos. Define-se então o problema de desmistura espectral (ou em inglês spectral unmixing - SU), que consiste em determinar as assinaturas espectrais dos endmembers puros presentes em uma cena e suas proporções (denominadas de abundâncias) para cada pixel da imagem. SU é um problema inverso e por natureza cego uma vez que raramente estão disponíveis informações confiáveis sobre o número de endmembers, suas assinaturas espectrais e suas distribuições em uma dada cena. Este problema possui forte conexão com o problema de separação cega de fontes mas difere no fato de que no problema de SU a independência de fontes não pode ser considerada já que as abundâncias são de fato proporções e por isso dependentes (abundâncias são positivas e devem somar 1). A determinação dos endmembers é conhecida como extração de endmembers e a literatura apresenta uma gama de algoritmos com esse propósito. Esses algoritmos normalmente exploram a geometria convexa resultante do modelo linear e da restrições sobre as abundâncias. Quando os endmembers são considerados conhecidos, ou estimados em um passo anterior, o problema de SU torna-se um problema supervisionado, com pares de entrada (endmembers) e saída (pixels), reduzindo-se a uma etapa de inversão, ou regressão, para determinar as proporções dos endmembers em cada pixel. Quando modelos não-lineares são considerados, a literatura apresenta diversas técnicas que podem ser empregadas dependendo da disponibilidade de informações sobre os endmembers e sobre os modelos que regem a interação entre a luz e os materiais numa dada cena. No entanto, informações sobre o tipo de mistura presente em cenas reais são raramente disponíveis. Nesse contexto, métodos kernelizados, que assumem modelos não-paramétricos, têm sido especialmente bem sucedidos quando aplicados ao problema de SU. Dentre esses métodos destaca-se o SK-Hype, que emprega a teoria de mínimos quadrados-máquinas de vetores de suporte (LS-SVM), numa abordagem que considera um modelo linear com uma flutuação não-linear representada por uma função pertencente a um espaço de Hilbert de kernel reprodutivos (RKHS). Nesta tese de doutoramento diferentes problemas foram abordados dentro do processo de SU de imagens hiperespectrais não-lineares como um todo. Contribuições foram dadas para a detecção de misturas não-lineares, estimação de endmembers quando uma parte considerável da imagem possui misturas não-lineares, e seleção de bandas no espaço de Hilbert de kernels reprodutivos (RKHS). Todos os métodos foram testados através de simulações com dados sintéticos e reais, e considerando unmixing supervisionado e não-supervisionado. No Capítulo 4, um método semi-paramétrico de detecção de misturas não-lineares é apresentado para imagens hiperespectrais. Esse detector compara a performance de dois modelos: um linear paramétrico, usando mínimos-quadrados (LS), e um não-linear não-paramétrico usando processos Gaussianos. A idéia da utilização de modelos não-paramétricos se conecta com o fato de que na prática pouco se sabe sobre a real natureza da não-linearidade presente na cena. Os erros de ajuste desses modelos são então comparados em uma estatística de teste para a qual é possível aproximar a distribuição na hipótese de misturas lineares e, assim, estimar um limiar de detecção para uma dada probabilidade de falso-alarme. A performance do detector proposto foi estudada considerando problemas supervisionados e não-supervisionados, sendo mostrado que a melhoria obtida no desempenho SU utilizando o detector proposto é estatisticamente consistente. Além disso, um grau de não-linearidade baseado nas energias relativas das contribuições lineares e não-lineares do processo de mistura foi definido para quantificar a importância das parcelas linear e não-linear dos modelos. Tal definição é importante para uma correta avaliação dos desempenhos relativos de diferentes estratégias de detecção de misturas não-lineares. No Capítulo 5 um algoritmo iterativo foi proposto para a estimação de endmembers como uma etapa de pré-processamento para problemas SU não supervisionados. Esse algoritmo intercala etapas de detecção de misturas não-lineares e estimação de endmembers de forma iterativa, na qual uma etapa de estimação de endmembers é seguida por uma etapa de detecção, na qual uma parcela dos pixels mais não-lineares é descartada. Esse processo é repetido por um número máximo de execuções ou até um critério de parada ser atingido. Demonstra-se que o uso combinado do detector proposto com um algoritmo de estimação de endmembers leva a melhores resultados de SU quando comparado com soluções do estado da arte. Simulações utilizando diferentes cenários corroboram as conclusões. No Capítulo 6 dois métodos para SU não-linear de imagens hiperespectrais, que empregam seleção de bandas (BS) diretamente no espaço de Hilbert de kernels reprodutivos (RKHS), são apresentados. O primeiro método utiliza o algoritmo Kernel K-Means (KKM) para encontrar clusters diretamente no RKHS onde cada centroide é então associada ao vetor espectral mais próximo. O segundo método é centralizado e baseado no critério de coerência, que incorpora uma medida da qualidade do dicionário no RKHS para a SU não-linear. Essa abordagem centralizada é equivalente a resolver um problema de máximo clique (MCP). Contrariamente a outros métodos concorrentes que não incluem uma escolha eficiente dos parâmetros do modelo, o método proposto requer apenas uma estimativa inicial do número de bandas selecionadas. Os resultados das simulações empregando dados, tanto sintéticos como reais, ilustram a qualidade dos resultados de unmixing obtidos com os métodos de BS propostos. Ao utilizar o SK-Hype, para um número reduzido de bandas, são obtidas estimativas de abundância tão precisas quanto aquelas obtidas utilizando o método SK-Hype com todo o espectro disponível, mas com uma pequena fração do custo computacional