926 research outputs found
Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net
Hyperspectral imaging can help better understand the characteristics of
different materials, compared with traditional image systems. However, only
high-resolution multispectral (HrMS) and low-resolution hyperspectral (LrHS)
images can generally be captured at video rate in practice. In this paper, we
propose a model-based deep learning approach for merging an HrMS and LrHS
images to generate a high-resolution hyperspectral (HrHS) image. In specific,
we construct a novel MS/HS fusion model which takes the observation models of
low-resolution images and the low-rankness knowledge along the spectral mode of
HrHS image into consideration. Then we design an iterative algorithm to solve
the model by exploiting the proximal gradient method. And then, by unfolding
the designed algorithm, we construct a deep network, called MS/HS Fusion Net,
with learning the proximal operators and model parameters by convolutional
neural networks. Experimental results on simulated and real data substantiate
the superiority of our method both visually and quantitatively as compared with
state-of-the-art methods along this line of research.Comment: 10 pages, 7 figure
HyperKon: A Self-Supervised Contrastive Network for Hyperspectral Image Analysis
The exceptional spectral resolution of hyperspectral imagery enables material
insights that are not possible with RGB or multispectral images. Yet, the full
potential of this data is often underutilized by deep learning techniques due
to the scarcity of hyperspectral-native CNN backbones. To bridge this gap, we
introduce HyperKon, a self-supervised contrastive learning network designed and
trained on hyperspectral data from the EnMAP Hyperspectral
Satellite\cite{kaufmann2012environmental}. HyperKon uniquely leverages the high
spectral continuity, range, and resolution of hyperspectral data through a
spectral attention mechanism and specialized convolutional layers. We also
perform a thorough ablation study on different kinds of layers, showing their
performance in understanding hyperspectral layers. It achieves an outstanding
98% Top-1 retrieval accuracy and outperforms traditional RGB-trained backbones
in hyperspectral pan-sharpening tasks. Additionally, in hyperspectral image
classification, HyperKon surpasses state-of-the-art methods, indicating a
paradigm shift in hyperspectral image analysis and underscoring the importance
of hyperspectral-native backbones.Comment: 10 pages, 8 figure
Unsupervised Sparse Dirichlet-Net for Hyperspectral Image Super-Resolution
In many computer vision applications, obtaining images of high resolution in
both the spatial and spectral domains are equally important. However, due to
hardware limitations, one can only expect to acquire images of high resolution
in either the spatial or spectral domains. This paper focuses on hyperspectral
image super-resolution (HSI-SR), where a hyperspectral image (HSI) with low
spatial resolution (LR) but high spectral resolution is fused with a
multispectral image (MSI) with high spatial resolution (HR) but low spectral
resolution to obtain HR HSI. Existing deep learning-based solutions are all
supervised that would need a large training set and the availability of HR HSI,
which is unrealistic. Here, we make the first attempt to solving the HSI-SR
problem using an unsupervised encoder-decoder architecture that carries the
following uniquenesses. First, it is composed of two encoder-decoder networks,
coupled through a shared decoder, in order to preserve the rich spectral
information from the HSI network. Second, the network encourages the
representations from both modalities to follow a sparse Dirichlet distribution
which naturally incorporates the two physical constraints of HSI and MSI.
Third, the angular difference between representations are minimized in order to
reduce the spectral distortion. We refer to the proposed architecture as
unsupervised Sparse Dirichlet-Net, or uSDN. Extensive experimental results
demonstrate the superior performance of uSDN as compared to the
state-of-the-art.Comment: Accepted by The IEEE Conference on Computer Vision and Pattern
Recognition (CVPR 2018, Spotlight
- …