42 research outputs found
Image Processing and Machine Learning for Hyperspectral Unmixing: An Overview and the HySUPP Python Package
Spectral pixels are often a mixture of the pure spectra of the materials,
called endmembers, due to the low spatial resolution of hyperspectral sensors,
double scattering, and intimate mixtures of materials in the scenes. Unmixing
estimates the fractional abundances of the endmembers within the pixel.
Depending on the prior knowledge of endmembers, linear unmixing can be divided
into three main groups: supervised, semi-supervised, and unsupervised (blind)
linear unmixing. Advances in Image processing and machine learning
substantially affected unmixing. This paper provides an overview of advanced
and conventional unmixing approaches. Additionally, we draw a critical
comparison between advanced and conventional techniques from the three
categories. We compare the performance of the unmixing techniques on three
simulated and two real datasets. The experimental results reveal the advantages
of different unmixing categories for different unmixing scenarios. Moreover, we
provide an open-source Python-based package available at
https://github.com/BehnoodRasti/HySUPP to reproduce the results
Classification of hyperspectral images by exploiting spectral-spatial information of superpixel via multiple kernels
For the classification of hyperspectral images (HSIs), this paper presents a novel framework to effectively utilize the spectral-spatial information of superpixels via multiple kernels, termed as superpixel-based classification via multiple kernels (SC-MK). In HSI, each superpixel can be regarded as a shape-adaptive region which consists of a number of spatial-neighboring pixels with very similar spectral characteristics. Firstly, the proposed SC-MK method adopts an over-segmentation algorithm to cluster the HSI into many superpixels. Then, three kernels are separately employed for the utilization of the spectral information as well as spatial information within and among superpixels. Finally, the three kernels are combined together and incorporated into a support vector machines classifier. Experimental results on three widely used real HSIs indicate that the proposed SC-MK approach outperforms several well-known classification methods
Interpretable Hyperspectral AI: When Non-Convex Modeling meets Hyperspectral Remote Sensing
Hyperspectral imaging, also known as image spectrometry, is a landmark
technique in geoscience and remote sensing (RS). In the past decade, enormous
efforts have been made to process and analyze these hyperspectral (HS) products
mainly by means of seasoned experts. However, with the ever-growing volume of
data, the bulk of costs in manpower and material resources poses new challenges
on reducing the burden of manual labor and improving efficiency. For this
reason, it is, therefore, urgent to develop more intelligent and automatic
approaches for various HS RS applications. Machine learning (ML) tools with
convex optimization have successfully undertaken the tasks of numerous
artificial intelligence (AI)-related applications. However, their ability in
handling complex practical problems remains limited, particularly for HS data,
due to the effects of various spectral variabilities in the process of HS
imaging and the complexity and redundancy of higher dimensional HS signals.
Compared to the convex models, non-convex modeling, which is capable of
characterizing more complex real scenes and providing the model
interpretability technically and theoretically, has been proven to be a
feasible solution to reduce the gap between challenging HS vision tasks and
currently advanced intelligent data processing models
Graph-based Data Modeling and Analysis for Data Fusion in Remote Sensing
Hyperspectral imaging provides the capability of increased sensitivity and discrimination over traditional imaging methods by combining standard digital imaging with spectroscopic methods. For each individual pixel in a hyperspectral image (HSI), a continuous spectrum is sampled as the spectral reflectance/radiance signature to facilitate identification of ground cover and surface material. The abundant spectrum knowledge allows all available information from the data to be mined. The superior qualities within hyperspectral imaging allow wide applications such as mineral exploration, agriculture monitoring, and ecological surveillance, etc. The processing of massive high-dimensional HSI datasets is a challenge since many data processing techniques have a computational complexity that grows exponentially with the dimension. Besides, a HSI dataset may contain a limited number of degrees of freedom due to the high correlations between data points and among the spectra. On the other hand, merely taking advantage of the sampled spectrum of individual HSI data point may produce inaccurate results due to the mixed nature of raw HSI data, such as mixed pixels, optical interferences and etc.
Fusion strategies are widely adopted in data processing to achieve better performance, especially in the field of classification and clustering. There are mainly three types of fusion strategies, namely low-level data fusion, intermediate-level feature fusion, and high-level decision fusion. Low-level data fusion combines multi-source data that is expected to be complementary or cooperative. Intermediate-level feature fusion aims at selection and combination of features to remove redundant information. Decision level fusion exploits a set of classifiers to provide more accurate results. The fusion strategies have wide applications including HSI data processing. With the fast development of multiple remote sensing modalities, e.g. Very High Resolution (VHR) optical sensors, LiDAR, etc., fusion of multi-source data can in principal produce more detailed information than each single source. On the other hand, besides the abundant spectral information contained in HSI data, features such as texture and shape may be employed to represent data points from a spatial perspective. Furthermore, feature fusion also includes the strategy of removing redundant and noisy features in the dataset.
One of the major problems in machine learning and pattern recognition is to develop appropriate representations for complex nonlinear data. In HSI processing, a particular data point is usually described as a vector with coordinates corresponding to the intensities measured in the spectral bands. This vector representation permits the application of linear and nonlinear transformations with linear algebra to find an alternative representation of the data. More generally, HSI is multi-dimensional in nature and the vector representation may lose the contextual correlations. Tensor representation provides a more sophisticated modeling technique and a higher-order generalization to linear subspace analysis.
In graph theory, data points can be generalized as nodes with connectivities measured from the proximity of a local neighborhood. The graph-based framework efficiently characterizes the relationships among the data and allows for convenient mathematical manipulation in many applications, such as data clustering, feature extraction, feature selection and data alignment. In this thesis, graph-based approaches applied in the field of multi-source feature and data fusion in remote sensing area are explored. We will mainly investigate the fusion of spatial, spectral and LiDAR information with linear and multilinear algebra under graph-based framework for data clustering and classification problems
BiGSeT: Binary Mask-Guided Separation Training for DNN-based Hyperspectral Anomaly Detection
Hyperspectral anomaly detection (HAD) aims to recognize a minority of
anomalies that are spectrally different from their surrounding background
without prior knowledge. Deep neural networks (DNNs), including autoencoders
(AEs), convolutional neural networks (CNNs) and vision transformers (ViTs),
have shown remarkable performance in this field due to their powerful ability
to model the complicated background. However, for reconstruction tasks, DNNs
tend to incorporate both background and anomalies into the estimated
background, which is referred to as the identical mapping problem (IMP) and
leads to significantly decreased performance. To address this limitation, we
propose a model-independent binary mask-guided separation training strategy for
DNNs, named BiGSeT. Our method introduces a separation training loss based on a
latent binary mask to separately constrain the background and anomalies in the
estimated image. The background is preserved, while the potential anomalies are
suppressed by using an efficient second-order Laplacian of Gaussian (LoG)
operator, generating a pure background estimate. In order to maintain
separability during training, we periodically update the mask using a robust
proportion threshold estimated before the training. In our experiments, We
adopt a vanilla AE as the network to validate our training strategy on several
real-world datasets. Our results show superior performance compared to some
state-of-the-art methods. Specifically, we achieved a 90.67% AUC score on the
HyMap Cooke City dataset. Additionally, we applied our training strategy to
other deep network structures, achieving improved detection performance
compared to their original versions, demonstrating its effective
transferability. The code of our method will be available at
https://github.com/enter-i-username/BiGSeT.Comment: 13 pages, 13 figures, submitted to IEEE TRANSACTIONS ON IMAGE
PROCESSIN
Deep Learning Meets Hyperspectral Image Analysis: A Multidisciplinary Review
Modern hyperspectral imaging systems produce huge datasets potentially conveying a great abundance of information; such a resource, however, poses many challenges in the analysis and interpretation of these data. Deep learning approaches certainly offer a great variety of opportunities for solving classical imaging tasks and also for approaching new stimulating problems in the spatial–spectral domain. This is fundamental in the driving sector of Remote Sensing where hyperspectral technology was born and has mostly developed, but it is perhaps even more true in the multitude of current and evolving application sectors that involve these imaging technologies. The present review develops on two fronts: on the one hand, it is aimed at domain professionals who want to have an updated overview on how hyperspectral acquisition techniques can combine with deep learning architectures to solve specific tasks in different application fields. On the other hand, we want to target the machine learning and computer vision experts by giving them a picture of how deep learning technologies are applied to hyperspectral data from a multidisciplinary perspective. The presence of these two viewpoints and the inclusion of application fields other than Remote Sensing are the original contributions of this review, which also highlights some potentialities and critical issues related to the observed development trends