83 research outputs found
Hyperspectral Super-Resolution with Coupled Tucker Approximation: Recoverability and SVD-based algorithms
We propose a novel approach for hyperspectral super-resolution, that is based
on low-rank tensor approximation for a coupled low-rank multilinear (Tucker)
model. We show that the correct recovery holds for a wide range of multilinear
ranks. For coupled tensor approximation, we propose two SVD-based algorithms
that are simple and fast, but with a performance comparable to the
state-of-the-art methods. The approach is applicable to the case of unknown
spatial degradation and to the pansharpening problem.Comment: IEEE Transactions on Signal Processing, Institute of Electrical and
Electronics Engineers, in Pres
Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net
Hyperspectral imaging can help better understand the characteristics of
different materials, compared with traditional image systems. However, only
high-resolution multispectral (HrMS) and low-resolution hyperspectral (LrHS)
images can generally be captured at video rate in practice. In this paper, we
propose a model-based deep learning approach for merging an HrMS and LrHS
images to generate a high-resolution hyperspectral (HrHS) image. In specific,
we construct a novel MS/HS fusion model which takes the observation models of
low-resolution images and the low-rankness knowledge along the spectral mode of
HrHS image into consideration. Then we design an iterative algorithm to solve
the model by exploiting the proximal gradient method. And then, by unfolding
the designed algorithm, we construct a deep network, called MS/HS Fusion Net,
with learning the proximal operators and model parameters by convolutional
neural networks. Experimental results on simulated and real data substantiate
the superiority of our method both visually and quantitatively as compared with
state-of-the-art methods along this line of research.Comment: 10 pages, 7 figure
Hyperspectral and Multispectral Image Fusion using Optimized Twin Dictionaries
Spectral or spatial dictionary has been widely used in fusing low-spatial-resolution hyperspectral (LH) images and high-spatial-resolution multispectral (HM) images. However, only using spectral dictionary is insufficient for preserving spatial information, and vice versa. To address this problem, a new LH and HM image fusion method termed OTD using optimized twin dictionaries is proposed in this paper. The fusion problem of OTD is formulated analytically in the framework of sparse representation, as an optimization of twin spectral-spatial dictionaries and their corresponding sparse coefficients. More specifically, the spectral dictionary representing the generalized spectrums and its spectral sparse coefficients are optimized by utilizing the observed LH and HM images in the spectral domain; and the spatial dictionary representing the spatial information and its spatial sparse coefficients are optimized by modeling the rest of high-frequency information in the spatial domain. In addition, without non-negative constraints, the alternating direction methods of multipliers (ADMM) are employed to implement the above optimization process. Comparison results with the related state-of-the-art fusion methods on various datasets demonstrate that our proposed OTD method achieves a better fusion performance in both spatial and spectral domains
DDRF: Denoising Diffusion Model for Remote Sensing Image Fusion
Denosing diffusion model, as a generative model, has received a lot of
attention in the field of image generation recently, thanks to its powerful
generation capability. However, diffusion models have not yet received
sufficient research in the field of image fusion. In this article, we introduce
diffusion model to the image fusion field, treating the image fusion task as
image-to-image translation and designing two different conditional injection
modulation modules (i.e., style transfer modulation and wavelet modulation) to
inject coarse-grained style information and fine-grained high-frequency and
low-frequency information into the diffusion UNet, thereby generating fused
images. In addition, we also discussed the residual learning and the selection
of training objectives of the diffusion model in the image fusion task.
Extensive experimental results based on quantitative and qualitative
assessments compared with benchmarks demonstrates state-of-the-art results and
good generalization performance in image fusion tasks. Finally, it is hoped
that our method can inspire other works and gain insight into this field to
better apply the diffusion model to image fusion tasks. Code shall be released
for better reproducibility
Model Inspired Autoencoder for Unsupervised Hyperspectral Image Super-Resolution
This paper focuses on hyperspectral image (HSI) super-resolution that aims to
fuse a low-spatial-resolution HSI and a high-spatial-resolution multispectral
image to form a high-spatial-resolution HSI (HR-HSI). Existing deep
learning-based approaches are mostly supervised that rely on a large number of
labeled training samples, which is unrealistic. The commonly used model-based
approaches are unsupervised and flexible but rely on hand-craft priors.
Inspired by the specific properties of model, we make the first attempt to
design a model inspired deep network for HSI super-resolution in an
unsupervised manner. This approach consists of an implicit autoencoder network
built on the target HR-HSI that treats each pixel as an individual sample. The
nonnegative matrix factorization (NMF) of the target HR-HSI is integrated into
the autoencoder network, where the two NMF parts, spectral and spatial
matrices, are treated as decoder parameters and hidden outputs respectively. In
the encoding stage, we present a pixel-wise fusion model to estimate hidden
outputs directly, and then reformulate and unfold the model's algorithm to form
the encoder network. With the specific architecture, the proposed network is
similar to a manifold prior-based model, and can be trained patch by patch
rather than the entire image. Moreover, we propose an additional unsupervised
network to estimate the point spread function and spectral response function.
Experimental results conducted on both synthetic and real datasets demonstrate
the effectiveness of the proposed approach
- …