333 research outputs found
Image Processing and Machine Learning for Hyperspectral Unmixing: An Overview and the HySUPP Python Package
Spectral pixels are often a mixture of the pure spectra of the materials,
called endmembers, due to the low spatial resolution of hyperspectral sensors,
double scattering, and intimate mixtures of materials in the scenes. Unmixing
estimates the fractional abundances of the endmembers within the pixel.
Depending on the prior knowledge of endmembers, linear unmixing can be divided
into three main groups: supervised, semi-supervised, and unsupervised (blind)
linear unmixing. Advances in Image processing and machine learning
substantially affected unmixing. This paper provides an overview of advanced
and conventional unmixing approaches. Additionally, we draw a critical
comparison between advanced and conventional techniques from the three
categories. We compare the performance of the unmixing techniques on three
simulated and two real datasets. The experimental results reveal the advantages
of different unmixing categories for different unmixing scenarios. Moreover, we
provide an open-source Python-based package available at
https://github.com/BehnoodRasti/HySUPP to reproduce the results
Bidirectional recurrent imputation and abundance estimation of LULC classes with MODIS multispectral time series and geo-topographic and climatic data
Remotely sensed data are dominated by mixed Land Use and Land Cover (LULC)
types. Spectral unmixing (SU) is a key technique that disentangles mixed pixels
into constituent LULC types and their abundance fractions. While existing
studies on Deep Learning (DL) for SU typically focus on single time-step
hyperspectral (HS) or multispectral (MS) data, our work pioneers SU using MODIS
MS time series, addressing missing data with end-to-end DL models. Our approach
enhances a Long-Short Term Memory (LSTM)-based model by incorporating
geographic, topographic (geo-topographic), and climatic ancillary information.
Notably, our method eliminates the need for explicit endmember extraction,
instead learning the input-output relationship between mixed spectra and LULC
abundances through supervised learning. Experimental results demonstrate that
integrating spectral-temporal input data with geo-topographic and climatic
information significantly improves the estimation of LULC abundances in mixed
pixels. To facilitate this study, we curated a novel labeled dataset for
Andalusia (Spain) with monthly MODIS multispectral time series at 460m
resolution for 2013. Named Andalusia MultiSpectral MultiTemporal Unmixing
(Andalusia-MSMTU), this dataset provides pixel-level annotations of LULC
abundances along with ancillary information. The dataset
(https://zenodo.org/records/7752348) and code
(https://github.com/jrodriguezortega/MSMTU) are available to the public
GAUSS: Guided Encoder-Decoder Architecture for Hyperspectral Unmixing with Spatial Smoothness
In recent hyperspectral unmixing (HU) literature, the application of deep
learning (DL) has become more prominent, especially with the autoencoder (AE)
architecture. We propose a split architecture and use a pseudo-ground truth for
abundances to guide the `unmixing network' (UN) optimization. Preceding the UN,
an `approximation network' (AN) is proposed, which will improve the association
between the centre pixel and its neighbourhood. Hence, it will accentuate
spatial correlation in the abundances as its output is the input to the UN and
the reference for the `mixing network' (MN). In the Guided Encoder-Decoder
Architecture for Hyperspectral Unmixing with Spatial Smoothness (GAUSS), we
proposed using one-hot encoded abundances as the pseudo-ground truth to guide
the UN; computed using the k-means algorithm to exclude the use of prior HU
methods. Furthermore, we release the single-layer constraint on MN by
introducing the UN generated abundances in contrast to the standard AE for HU.
Secondly, we experimented with two modifications on the pre-trained network
using the GAUSS method. In GAUSS, we have concatenated the UN
and the MN to back-propagate the reconstruction error gradients to the encoder.
Then, in the GAUSS, abundance results of a signal processing
(SP) method with reliable abundance results were used as the pseudo-ground
truth with the GAUSS architecture. According to quantitative and graphical
results for four experimental datasets, the three architectures either
transcended or equated the performance of existing HU algorithms from both DL
and SP domains.Comment: 16 pages, 6 figure
Unsupervised Hyperspectral and Multispectral Images Fusion Based on the Cycle Consistency
Hyperspectral images (HSI) with abundant spectral information reflected
materials property usually perform low spatial resolution due to the hardware
limits. Meanwhile, multispectral images (MSI), e.g., RGB images, have a high
spatial resolution but deficient spectral signatures. Hyperspectral and
multispectral image fusion can be cost-effective and efficient for acquiring
both high spatial resolution and high spectral resolution images. Many of the
conventional HSI and MSI fusion algorithms rely on known spatial degradation
parameters, i.e., point spread function, spectral degradation parameters,
spectral response function, or both of them. Another class of deep
learning-based models relies on the ground truth of high spatial resolution HSI
and needs large amounts of paired training images when working in a supervised
manner. Both of these models are limited in practical fusion scenarios. In this
paper, we propose an unsupervised HSI and MSI fusion model based on the cycle
consistency, called CycFusion. The CycFusion learns the domain transformation
between low spatial resolution HSI (LrHSI) and high spatial resolution MSI
(HrMSI), and the desired high spatial resolution HSI (HrHSI) are considered to
be intermediate feature maps in the transformation networks. The CycFusion can
be trained with the objective functions of marginal matching in single
transform and cycle consistency in double transforms. Moreover, the estimated
PSF and SRF are embedded in the model as the pre-training weights, which
further enhances the practicality of our proposed model. Experiments conducted
on several datasets show that our proposed model outperforms all compared
unsupervised fusion methods. The codes of this paper will be available at this
address: https: //github.com/shuaikaishi/CycFusion for reproducibility
- …