Hyperspectral and multispectral image fusion via tensor sparsity regularization

Hyperspectral image (HSI) super-resolution scheme based on HSI and multispectral image (MSI) fusion has been a prevalent research theme in remote sensing. However, most of the existing HSI-MSI fusion (HMF) methods adopt the sparsity prior across spatial or spectral domains via vectorizing hyperspectral cubes along a certain dimension, which results in the spatial or spectral informations distortion. Moreover, the current HMF works rarely pay attention to leveraging the nonlocal similar structure over spatial domain of the HSI. In this paper, we propose a new HSI-MSI fusion approach via tensor sparsity regularization which can encode essential spatial and spectral sparsity of an HSI. Specifically, we study how to utilize reasonably the sparsity of tensor to describe the spatialspectral correlation hidden in an HSI. Then, we resort to an efficient optimization strategy based on the alternative direction multiplier method (ADMM) for solving the resulting minimization problem. Experimental results on Pavia University data verify the merits of the proposed HMF algorithm

Unsupervised Sparse Dirichlet-Net for Hyperspectral Image Super-Resolution

In many computer vision applications, obtaining images of high resolution in both the spatial and spectral domains are equally important. However, due to hardware limitations, one can only expect to acquire images of high resolution in either the spatial or spectral domains. This paper focuses on hyperspectral image super-resolution (HSI-SR), where a hyperspectral image (HSI) with low spatial resolution (LR) but high spectral resolution is fused with a multispectral image (MSI) with high spatial resolution (HR) but low spectral resolution to obtain HR HSI. Existing deep learning-based solutions are all supervised that would need a large training set and the availability of HR HSI, which is unrealistic. Here, we make the first attempt to solving the HSI-SR problem using an unsupervised encoder-decoder architecture that carries the following uniquenesses. First, it is composed of two encoder-decoder networks, coupled through a shared decoder, in order to preserve the rich spectral information from the HSI network. Second, the network encourages the representations from both modalities to follow a sparse Dirichlet distribution which naturally incorporates the two physical constraints of HSI and MSI. Third, the angular difference between representations are minimized in order to reduce the spectral distortion. We refer to the proposed architecture as unsupervised Sparse Dirichlet-Net, or uSDN. Extensive experimental results demonstrate the superior performance of uSDN as compared to the state-of-the-art.Comment: Accepted by The IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2018, Spotlight