1,964 research outputs found
Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net
Hyperspectral imaging can help better understand the characteristics of
different materials, compared with traditional image systems. However, only
high-resolution multispectral (HrMS) and low-resolution hyperspectral (LrHS)
images can generally be captured at video rate in practice. In this paper, we
propose a model-based deep learning approach for merging an HrMS and LrHS
images to generate a high-resolution hyperspectral (HrHS) image. In specific,
we construct a novel MS/HS fusion model which takes the observation models of
low-resolution images and the low-rankness knowledge along the spectral mode of
HrHS image into consideration. Then we design an iterative algorithm to solve
the model by exploiting the proximal gradient method. And then, by unfolding
the designed algorithm, we construct a deep network, called MS/HS Fusion Net,
with learning the proximal operators and model parameters by convolutional
neural networks. Experimental results on simulated and real data substantiate
the superiority of our method both visually and quantitatively as compared with
state-of-the-art methods along this line of research.Comment: 10 pages, 7 figure
Unsupervised Sparse Dirichlet-Net for Hyperspectral Image Super-Resolution
In many computer vision applications, obtaining images of high resolution in
both the spatial and spectral domains are equally important. However, due to
hardware limitations, one can only expect to acquire images of high resolution
in either the spatial or spectral domains. This paper focuses on hyperspectral
image super-resolution (HSI-SR), where a hyperspectral image (HSI) with low
spatial resolution (LR) but high spectral resolution is fused with a
multispectral image (MSI) with high spatial resolution (HR) but low spectral
resolution to obtain HR HSI. Existing deep learning-based solutions are all
supervised that would need a large training set and the availability of HR HSI,
which is unrealistic. Here, we make the first attempt to solving the HSI-SR
problem using an unsupervised encoder-decoder architecture that carries the
following uniquenesses. First, it is composed of two encoder-decoder networks,
coupled through a shared decoder, in order to preserve the rich spectral
information from the HSI network. Second, the network encourages the
representations from both modalities to follow a sparse Dirichlet distribution
which naturally incorporates the two physical constraints of HSI and MSI.
Third, the angular difference between representations are minimized in order to
reduce the spectral distortion. We refer to the proposed architecture as
unsupervised Sparse Dirichlet-Net, or uSDN. Extensive experimental results
demonstrate the superior performance of uSDN as compared to the
state-of-the-art.Comment: Accepted by The IEEE Conference on Computer Vision and Pattern
Recognition (CVPR 2018, Spotlight
Random Weights Networks Work as Loss Prior Constraint for Image Restoration
In this paper, orthogonal to the existing data and model studies, we instead
resort our efforts to investigate the potential of loss function in a new
perspective and present our belief ``Random Weights Networks can Be Acted as
Loss Prior Constraint for Image Restoration''. Inspired by Functional theory,
we provide several alternative solutions to implement our belief in the strict
mathematical manifolds including Taylor's Unfolding Network, Invertible Neural
Network, Central Difference Convolution and Zero-order Filtering as ``random
weights network prototype'' with respect of the following four levels: 1) the
different random weights strategies; 2) the different network architectures,
\emph{eg,} pure convolution layer or transformer; 3) the different network
architecture depths; 4) the different numbers of random weights network
combination. Furthermore, to enlarge the capability of the randomly initialized
manifolds, we devise the manner of random weights in the following two
variants: 1) the weights are randomly initialized only once during the whole
training procedure; 2) the weights are randomly initialized at each training
iteration epoch. Our propose belief can be directly inserted into existing
networks without any training and testing computational cost. Extensive
experiments across multiple image restoration tasks, including image
de-noising, low-light image enhancement, guided image super-resolution
demonstrate the consistent performance gains obtained by introducing our
belief. To emphasize, our main focus is to spark the realms of loss function
and save their current neglected status. Code will be publicly available
- …