Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches

Imaging spectrometers measure electromagnetic energy scattered in their instantaneous field view in hundreds or thousands of spectral channels with higher spectral resolution than multispectral cameras. Imaging spectrometers are therefore often referred to as hyperspectral cameras (HSCs). Higher spectral resolution enables material identification via spectroscopic analysis, which facilitates countless applications that require identifying materials in scenarios unsuitable for classical spectroscopic analysis. Due to low spatial resolution of HSCs, microscopic material mixing, and multiple scattering, spectra measured by HSCs are mixtures of spectra of materials in a scene. Thus, accurate estimation requires unmixing. Pixels are assumed to be mixtures of a few materials, called endmembers. Unmixing involves estimating all or some of: the number of endmembers, their spectral signatures, and their abundances at each pixel. Unmixing is a challenging, ill-posed inverse problem because of model inaccuracies, observation noise, environmental conditions, endmember variability, and data set size. Researchers have devised and investigated many models searching for robust, stable, tractable, and accurate unmixing algorithms. This paper presents an overview of unmixing methods from the time of Keshava and Mustard's unmixing tutorial [1] to the present. Mixing models are first discussed. Signal-subspace, geometrical, statistical, sparsity-based, and spatial-contextual unmixing algorithms are described. Mathematical problems and potential solutions are described. Algorithm characteristics are illustrated experimentally.Comment: This work has been accepted for publication in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensin

A Novel Methodology for Calculating Large Numbers of Symmetrical Matrices on a Graphics Processing Unit: Towards Efficient, Real-Time Hyperspectral Image Processing

Hyperspectral imagery (HSI) is often processed to identify targets of interest. Many of the quantitative analysis techniques developed for this purpose mathematically manipulate the data to derive information about the target of interest based on local spectral covariance matrices. The calculation of a local spectral covariance matrix for every pixel in a given hyperspectral data scene is so computationally intensive that real-time processing with these algorithms is not feasible with today’s general purpose processing solutions. Specialized solutions are cost prohibitive, inflexible, inaccessible, or not feasible for on-board applications. Advances in graphics processing unit (GPU) capabilities and programmability offer an opportunity for general purpose computing with access to hundreds of processing cores in a system that is affordable and accessible. The GPU also offers flexibility, accessibility and feasibility that other specialized solutions do not offer. The architecture for the NVIDIA GPU used in this research is significantly different from the architecture of other parallel computing solutions. With such a substantial change in architecture it follows that the paradigm for programming graphics hardware is significantly different from traditional serial and parallel software development paradigms. In this research a methodology for mapping an HSI target detection algorithm to the NVIDIA GPU hardware and Compute Unified Device Architecture (CUDA) Application Programming Interface (API) is developed. The RX algorithm is chosen as a representative stochastic HSI algorithm that requires the calculation of a spectral covariance matrix. The developed methodology is designed to calculate a local covariance matrix for every pixel in the input HSI data scene. A characterization of the limitations imposed by the chosen GPU is given and a path forward toward optimization of a GPU-based method for real-time HSI data processing is defined

Diseño e implementación de una cadena completa para desmezclado de imágenes hiperespectrales en tarjetas gráficas programables (GPUs)

La principal contribución del presente trabajo de tesis doctoral viene dada por la propuesta de nuevos algoritmos paralelos para desmezclado de imágenes hiperespectrales en aplicaciones de observación remota de la superficie terrestre mediante sensores aerotransportados o de tipo satélite. Dichos algoritmos se fundamentan en el problema de la mezcla, que permite expresar los píxels de una imagen hiperespectral como una combinación lineal o no lineal de elementos espectralmente puros (“endmembers”) ponderados por sus correspondientes fracciones de abundancia. Una vez descrita la base teórica del estudio, la tesis doctoral presenta una serie de nuevos algoritmos paralelos desarrollados, los cuales integran una cadena completa de desmezclado espectral o “unmixing” con las siguientes etapas: 1) estimación automática del número de “endmembers” en una imagen hiperespectral, 2) identificación automática de dichos “endmembers” en la imagen hiperespectral, y 3) estimación de la abundancia de cada “endmember” en cada píxel de la imagen. Tras presentar los nuevos algoritmos paralelos desarrollados con motivo del presente trabajo, realizamos un detallado estudio cuantitativo y comparativo de su precisión en el proceso de desmezclado y su rendimiento computacional en un conjunto de arquitecturas basadas en tarjetas tarjeta gráficas programables de NVidia (modelos Nvidia Tesla C1060 y NVidia GeForce 580 GTX). Los resultados experimentales han sido obtenidos utilizando imágenes hiperespectrales obtenidas por los sensores Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) e Hyperion de NASA en el contexto de varias aplicaciones reales de gran relevancia social, consistentes en la detección de los incendios que se propagaron en los días posteriores al atentado terrorista del World Trade Center en Nueva York o en la identificación automática de minerales en la región de Cuprite, Nevada, Estados Unidos. En dichos escenarios, los equipos de NASA y el Instituto Geológico de Estados Unidos (USGS) que participaron en las tareas de extinción y emergencia (en el caso de la imagen World Trade Center) e identificación de minerales (en el caso de la imagen de Cuprite) reconocieron que la disponibilidad de técnicas de desmezclado espectral en tiempo real hubiese facilitado las labores de los equipos que actuaron en dichas zonas, por lo que las técnicas desarrolladas se han desarrollado con el objetivo de permitir la realización de dichas tareas en el futuro. La memoria de tesis concluye con una discusión de las técnicas desarrolladas (incluyendo una serie de recomendaciones sobre su mejor uso en diferentes circunstancias), con la descripción de las principales conclusiones y líneas futuras derivadas del estudio, y con la bibliografía relacionada, tanto en la literatura general como la generada por el candidato.The main contribution of the present thesis work is given by the proposal of several new parallel algoritms for spectral mixture analysis of remotely sensed hyperspectral images obtained from airborne or satellite Earth observation platforms. These algorithms are focused on the identification of the most spectrally pure constituents of a hyperspectral image, and on the characterization of mixed pixels as linear or nonlinear combinations of endmembers weighted by their fractional abundances on a sub-pixel basis. Once the theoretical foundations of the proposed study are described, we proceed to describe in detail the new parallel algorithms developed as the main contribution of this research work, discussing the different steps followed in their development which comprise the following stages: 1) automatic identification of the number of endmembers in the hyperspectral image; 2) automatic identification of the spectral signatures of such endmembers; and 3) estimation of the fractional abundance of endmembers on a sub-pixel basis. After describing the new parallel algorithms introduced in this work, we develop a comprehensive quantitative and comparative analysis in terms of unmixing accuracy and computational performance using a set of graphics processing unit (GPU)-based architectures, including the NVidia Tesla C1060 and the NVidia GeForce 580 GTX. The experimental results reported in this work are evaluated in the context of two real applications with great societal impact: the possibility to automatically detect the thermal hot spots of the fires which spread in the World Trade Center area during the days after the terrorist attack of September 11th, 2001, and the possibility to perform real-time mapping of minerals in the Cuprite mining district of Nevada, USA, using hyperspectral data sets collected by NASA’s Airborne Visible Infra-Red Imaging Spectrometer (AVIRS) and the Hyperion instrument onboard Earth Observing One (EO-1) spacecraft. It is acknowledged by some of the organizations that, if high performance computing infrastructure had been available at the time of these events, the hyperspectral data would have been much more useful. The design of new techniques for this purpose may help the development of such tasks in future events. The thesis document concludes with a detailed discussion on the techniques presented herein (including processing recommendations and best practice), with the drawing of the main conclusions and hints at plausible future research, and with a detailed bibliography on the research area and on the specific contributions provided by the candidate to the scientific literature devoted to this topic

Efficient multitemporal change detection techniques for hyperspectral images on GPU

Hyperspectral images contain hundreds of reflectance values for each pixel. Detecting regions of change in multiple hyperspectral images of the same scene taken at different times is of widespread interest for a large number of applications. For remote sensing, in particular, a very common application is land-cover analysis. The high dimensionality of the hyperspectral images makes the development of computationally efficient processing schemes critical. This thesis focuses on the development of change detection approaches at object level, based on supervised direct multidate classification, for hyperspectral datasets. The proposed approaches improve the accuracy of current state of the art algorithms and their projection onto Graphics Processing Units (GPUs) allows their execution in real-time scenarios

Enhancement and Edge-Preserving Denoising: An OpenCL-Based Approach for Remote Sensing Imagery

Image enhancement and edge-preserving denoising are relevant steps before classification or other postprocessing techniques for remote sensing images. However, multisensor array systems are able to simultaneously capture several low-resolution images from the same area on different wavelengths, forming a high spatial/spectral resolution image and raising a series of new challenges. In this paper, an open computing language based parallel implementation approach is presented for near real-time enhancement based on Bayesian maximum entropy (BME), as well as an edge-preserving denoising algorithm for remote sensing imagery, which uses the local linear Stein’s unbiased risk estimate (LLSURE). BME was selected for its results on synthetic aperture radar image enhancement, whereas LLSURE has shown better noise removal properties than other commonly used methods. Within this context, image processing methods are algorithmically adapted via parallel computing techniques and efficiently implemented using CPUs and commodity graphics processing units (GPUs). Experimental results demonstrate the reduction of computational load of real-world image processing for near real-time GPU adapted implementation.ITESO, A.C

A Stepwise Analytical Projected Gradient Descent Search for Hyperspectral Unmixing and Its Code Vectorization

We present, in this paper, a new methodology for spectral unmixing, where a vector of fractions, corresponding to a set of endmembers (EMs), is estimated for each pixel in the image. The process first provides an initial estimate of the fraction vector, followed by an iterative procedure that converges to an optimal solution. Specifically, projected gradient descent (PGD) optimization is applied to (a variant of) the spectral angle mapper objective function, so as to significantly reduce the estimation error due to amplitude (i.e., magnitude) variations in EM spectra, caused by the illumination change effect. To improve the computational efficiency of our method over a commonly used gradient descent technique, we have analytically derived the objective function's gradient and the optimal step size (used in each iteration). To gain further improvement, we have implemented our unmixing module via code vectorization, where the entire process is ''folded'' into a single loop, and the fractions for all of the pixels are solved simultaneously. We call this new parallel scheme vectorized code PGD unmixing (VPGDU). VPGDU has the advantage of solving (simultaneously) an independent optimization problem per image pixel, exactly as other pixelwise algorithms, but significantly faster. Its performance was compared with the commonly used fully constrained least squares unmixing (FCLSU), the generalized bilinear model (GBM) method for hyperspectral unmixng, and the fast state-of-the-art methods, sparse unmixing by variable splitting and augmented Lagrangian (SUnSAL) and collaborative SUnSAL (CLSUnSAL) based on the alternating direction method of multipliers. Considering all of the prospective EMs of a scene at each pixel (i.e., without a priori knowledge which/how many EMs are actually present in a given pixel), we demonstrate that the accuracy due to VPGDU is considerably higher than that obtained by FCLSU, GBM, SUnSAL, and CLSUnSAL under varying illumination, and is, otherwise, comparable with respect to these methods. However, while our method is significantly faster than FCLSU and GBM, it is slower than SUnSAL and CLSUnSAL by roughly an order of magnitude.Israel Science Ministry Scientific Infrastructure Research Grant Scheme, Helen Norman Asher Space Research Grant Scheme, Technion PhD Scholarship, new England fund Technion, Environmental Mapping and Monitoring of Iceland by Remote Sensing EMMIRS projectPeer Reviewe

Spectral-spatial classification of n-dimensional images in real-time based on segmentation and mathematical morphology on GPUs

The objective of this thesis is to develop efficient schemes for spectral-spatial n-dimensional image classification. By efficient schemes, we mean schemes that produce good classification results in terms of accuracy, as well as schemes that can be executed in real-time on low-cost computing infrastructures, such as the Graphics Processing Units (GPUs) shipped in personal computers. The n-dimensional images include images with two and three dimensions, such as images coming from the medical domain, and also images ranging from ten to hundreds of dimensions, such as the multiand hyperspectral images acquired in remote sensing. In image analysis, classification is a regularly used method for information retrieval in areas such as medical diagnosis, surveillance, manufacturing and remote sensing, among others. In addition, as the hyperspectral images have been widely available in recent years owing to the reduction in the size and cost of the sensors, the number of applications at lab scale, such as food quality control, art forgery detection, disease diagnosis and forensics has also increased. Although there are many spectral-spatial classification schemes, most are computationally inefficient in terms of execution time. In addition, the need for efficient computation on low-cost computing infrastructures is increasing in line with the incorporation of technology into everyday applications. In this thesis we have proposed two spectral-spatial classification schemes: one based on segmentation and other based on wavelets and mathematical morphology. These schemes were designed with the aim of producing good classification results and they perform better than other schemes found in the literature based on segmentation and mathematical morphology in terms of accuracy. Additionally, it was necessary to develop techniques and strategies for efficient GPU computing, for example, a block–asynchronous strategy, resulting in an efficient implementation on GPU of the aforementioned spectral-spatial classification schemes. The optimal GPU parameters were analyzed and different data partitioning and thread block arrangements were studied to exploit the GPU resources. The results show that the GPU is an adequate computing platform for on-board processing of hyperspectral information