22 research outputs found
Unsupervised Sparse Dirichlet-Net for Hyperspectral Image Super-Resolution
In many computer vision applications, obtaining images of high resolution in
both the spatial and spectral domains are equally important. However, due to
hardware limitations, one can only expect to acquire images of high resolution
in either the spatial or spectral domains. This paper focuses on hyperspectral
image super-resolution (HSI-SR), where a hyperspectral image (HSI) with low
spatial resolution (LR) but high spectral resolution is fused with a
multispectral image (MSI) with high spatial resolution (HR) but low spectral
resolution to obtain HR HSI. Existing deep learning-based solutions are all
supervised that would need a large training set and the availability of HR HSI,
which is unrealistic. Here, we make the first attempt to solving the HSI-SR
problem using an unsupervised encoder-decoder architecture that carries the
following uniquenesses. First, it is composed of two encoder-decoder networks,
coupled through a shared decoder, in order to preserve the rich spectral
information from the HSI network. Second, the network encourages the
representations from both modalities to follow a sparse Dirichlet distribution
which naturally incorporates the two physical constraints of HSI and MSI.
Third, the angular difference between representations are minimized in order to
reduce the spectral distortion. We refer to the proposed architecture as
unsupervised Sparse Dirichlet-Net, or uSDN. Extensive experimental results
demonstrate the superior performance of uSDN as compared to the
state-of-the-art.Comment: Accepted by The IEEE Conference on Computer Vision and Pattern
Recognition (CVPR 2018, Spotlight
Application of Deep Learning Approaches for Enhancing Mastcam Images
There are two mast cameras (Mastcam) onboard the Mars rover Curiosity. Both Mastcams are multispectral imagers with nine bands in each. The right Mastcam has three times higher resolution than the left. In this chapter, we apply some recently developed deep neural network models to enhance the left Mastcam images with help from the right Mastcam images. Actual Mastcam images were used to demonstrate the performance of the proposed algorithms
Guided Deep Decoder: Unsupervised Image Pair Fusion
The fusion of input and guidance images that have a tradeoff in their
information (e.g., hyperspectral and RGB image fusion or pansharpening) can be
interpreted as one general problem. However, previous studies applied a
task-specific handcrafted prior and did not address the problems with a unified
approach. To address this limitation, in this study, we propose a guided deep
decoder network as a general prior. The proposed network is composed of an
encoder-decoder network that exploits multi-scale features of a guidance image
and a deep decoder network that generates an output image. The two networks are
connected by feature refinement units to embed the multi-scale features of the
guidance image into the deep decoder network. The proposed network allows the
network parameters to be optimized in an unsupervised way without training
data. Our results show that the proposed network can achieve state-of-the-art
performance in various image fusion problems.Comment: ECCV 202