114 research outputs found
Fusing Multiple Multiband Images
We consider the problem of fusing an arbitrary number of multiband, i.e.,
panchromatic, multispectral, or hyperspectral, images belonging to the same
scene. We use the well-known forward observation and linear mixture models with
Gaussian perturbations to formulate the maximum-likelihood estimator of the
endmember abundance matrix of the fused image. We calculate the Fisher
information matrix for this estimator and examine the conditions for the
uniqueness of the estimator. We use a vector total-variation penalty term
together with nonnegativity and sum-to-one constraints on the endmember
abundances to regularize the derived maximum-likelihood estimation problem. The
regularization facilitates exploiting the prior knowledge that natural images
are mostly composed of piecewise smooth regions with limited abrupt changes,
i.e., edges, as well as coping with potential ill-posedness of the fusion
problem. We solve the resultant convex optimization problem using the
alternating direction method of multipliers. We utilize the circular
convolution theorem in conjunction with the fast Fourier transform to alleviate
the computational complexity of the proposed algorithm. Experiments with
multiband images constructed from real hyperspectral datasets reveal the
superior performance of the proposed algorithm in comparison with the
state-of-the-art algorithms, which need to be used in tandem to fuse more than
two multiband images
A convex formulation for hyperspectral image superresolution via subspace-based regularization
Hyperspectral remote sensing images (HSIs) usually have high spectral
resolution and low spatial resolution. Conversely, multispectral images (MSIs)
usually have low spectral and high spatial resolutions. The problem of
inferring images which combine the high spectral and high spatial resolutions
of HSIs and MSIs, respectively, is a data fusion problem that has been the
focus of recent active research due to the increasing availability of HSIs and
MSIs retrieved from the same geographical area.
We formulate this problem as the minimization of a convex objective function
containing two quadratic data-fitting terms and an edge-preserving regularizer.
The data-fitting terms account for blur, different resolutions, and additive
noise. The regularizer, a form of vector Total Variation, promotes
piecewise-smooth solutions with discontinuities aligned across the
hyperspectral bands.
The downsampling operator accounting for the different spatial resolutions,
the non-quadratic and non-smooth nature of the regularizer, and the very large
size of the HSI to be estimated lead to a hard optimization problem. We deal
with these difficulties by exploiting the fact that HSIs generally "live" in a
low-dimensional subspace and by tailoring the Split Augmented Lagrangian
Shrinkage Algorithm (SALSA), which is an instance of the Alternating Direction
Method of Multipliers (ADMM), to this optimization problem, by means of a
convenient variable splitting. The spatial blur and the spectral linear
operators linked, respectively, with the HSI and MSI acquisition processes are
also estimated, and we obtain an effective algorithm that outperforms the
state-of-the-art, as illustrated in a series of experiments with simulated and
real-life data.Comment: IEEE Trans. Geosci. Remote Sens., to be publishe
Hyperspectral Image Super-Resolution Using Optimization and DCNN-Based Methods
Reconstructing a high-resolution (HR) hyperspectral (HS) image from the observed low-resolution (LR) hyperspectral image or a high-resolution multispectral (RGB) image obtained using the exiting imaging cameras is an important research topic for capturing comprehensive scene information in both spatial and spectral domains. The HR-HS hyperspectral image reconstruction mainly consists of two research strategies: optimization-based and the deep convolutional neural network-based learning methods. The optimization-based approaches estimate HR-HS image via minimizing the reconstruction errors of the available low-resolution hyperspectral and high-resolution multispectral images with different constrained prior knowledge such as representation sparsity, spectral physical properties, spatial smoothness, and so on. Recently, deep convolutional neural network (DCNN) has been applied to resolution enhancement of natural images and is proven to achieve promising performance. This chapter provides a comprehensive description of not only the conventional optimization-based methods but also the recently investigated DCNN-based learning methods for HS image super-resolution, which mainly include spectral reconstruction CNN and spatial and spectral fusion CNN. Experiment results on benchmark datasets have been shown for validating effectiveness of HS image super-resolution in both quantitative values and visual effect
Guided Deep Decoder: Unsupervised Image Pair Fusion
The fusion of input and guidance images that have a tradeoff in their
information (e.g., hyperspectral and RGB image fusion or pansharpening) can be
interpreted as one general problem. However, previous studies applied a
task-specific handcrafted prior and did not address the problems with a unified
approach. To address this limitation, in this study, we propose a guided deep
decoder network as a general prior. The proposed network is composed of an
encoder-decoder network that exploits multi-scale features of a guidance image
and a deep decoder network that generates an output image. The two networks are
connected by feature refinement units to embed the multi-scale features of the
guidance image into the deep decoder network. The proposed network allows the
network parameters to be optimized in an unsupervised way without training
data. Our results show that the proposed network can achieve state-of-the-art
performance in various image fusion problems.Comment: ECCV 202
- …