19 research outputs found
Remote sensing image fusion via compressive sensing
In this paper, we propose a compressive sensing-based method to pan-sharpen the low-resolution multispectral (LRM) data, with the help of high-resolution panchromatic (HRP) data. In order to successfully implement the compressive sensing theory in pan-sharpening, two requirements should be satisfied: (i) forming a comprehensive dictionary in which the estimated coefficient vectors are sparse; and (ii) there is no correlation between the constructed dictionary and the measurement matrix. To fulfill these, we propose two novel strategies. The first is to construct a dictionary that is trained with patches across different image scales. Patches at different scales or equivalently multiscale patches provide texture atoms without requiring any external database or any prior atoms. The redundancy of the dictionary is removed through K-singular value decomposition (K-SVD). Second, we design an iterative
l1-l2
minimization algorithm based on alternating direction method of multipliers (ADMM) to seek the sparse coefficient vectors. The proposed algorithm stacks missing high-resolution multispectral (HRM) data with the captured LRM data, so that the latter is used as a constraint for the estimation of the former during the process of seeking the representation coefficients. Three datasets are used to test the performance of the proposed method. A comparative study between the proposed method and several state-of-the-art ones shows its effectiveness in dealing with complex structures of remote sensing imagery
Pansharpening of Hyperspectral images using spatial distortion optimization
International audienc
Image Fusion in Remote Sensing and Quality Evaluation of Fused Images
In remote sensing, acquired optical images of high spectral resolution have usually a lower spatial resolution than images of lower spectral resolution. This is due to physical, cost and complexity constraints. To make the most of the available imagery, many image fusion techniques have been developed to address this problem. Image fusion is an ill-posed inverse problem where an image of low spatial resolution and high spectral resolution is enhanced in spatial-resolution by using an auxiliary image of high spatial resolution and low spectral resolution. It is assumed that both images display the same scene and are properly co-registered. Thus, the problem is essentially to transfer details from the higher spatial resolution auxiliary image to the upscaled lower resolution image in a manner that minimizes the spatial and spectral distortion of the fused image. The most common image fusion problem is pansharpening, where a multispectral (MS) image is enhanced using wide-band panchromatic (PAN) image. A similar problem is the enhancement of a hyperspectral (HS) image by either a PAN image or an MS image. As there is no reference image available, the reliable quantitative evaluation of the quality of the fused image is a difficult problem. This thesis addresses the image fusion problem in three different ways and also addresses the problem of quantitative quality evaluation.Í fjarkönnun hafa myndir með háa rófsupplausn lægri rúmupplausn en myndir með lægri rófsupplausn vegna eðlisfræðilegra og kostnaðarlegra takmarkana. Til að auka upplýsingamagn
slíkra mynda hafa verið þróaðar fjölmargar sambræðsluaðferðir á síðustu
tveimur áratugum. Myndsambræðsla er illa framsett andhverft vandmál (e. inverse
problem) þar sem rúmupplausn myndar af hárri rófsupplausn er aukin með því að
nota upplýsingar frá mynd af hárri rúmupplausn og lægri rófsupplausn. Það er gert
ráð fyrir að báðar myndir sýni nákvæmlega sama landsvæði. Þannig er vandamálið í
eðli sínu að flytja fíngerða eiginleika myndar af hærri rúmupplausn yfir á mynd af lægri
rúmupplausn sem hefur verið brúuð upp í stærð hinnar myndarinnar, án þess að skerða
gæði rófsupplýsinga upphaflegu myndarinnar. Algengasta myndbræðsluvandamálið í
fjarkönnun er svokölluð panskerpun (e. pansharpening) þar sem fjölrásamynd (e. multispectral
image) er endurbætt í rúmi með svokallaðri víðbandsmynd (e. panchromatic
image) sem hefur aðeins eina rás af hárri upplausn. Annað svipað vandamál er sambræðsla
háfjölrásamyndar (e. hyperspectral image) og annaðhvort fjölrásamyndar eða
víðbandsmyndar. Þar sem myndsambræðsla er andhverft vandmál er engin háupplausnar
samanburðarmynd tiltæk, sem gerir mat á gæðum sambræddu myndarinnar
að erfiðu vandamáli. Í þessari ritgerð eru kynntar þrjár aðferðir sem taka á myndsambræðlsu
og einnig er fjallað um mat á gæðum sambræddra mynda, þá sérstaklega
panskerptra mynda
A Spectral Diffusion Prior for Hyperspectral Image Super-Resolution
Fusion-based hyperspectral image (HSI) super-resolution aims to produce a
high-spatial-resolution HSI by fusing a low-spatial-resolution HSI and a
high-spatial-resolution multispectral image. Such a HSI super-resolution
process can be modeled as an inverse problem, where the prior knowledge is
essential for obtaining the desired solution. Motivated by the success of
diffusion models, we propose a novel spectral diffusion prior for fusion-based
HSI super-resolution. Specifically, we first investigate the spectrum
generation problem and design a spectral diffusion model to model the spectral
data distribution. Then, in the framework of maximum a posteriori, we keep the
transition information between every two neighboring states during the reverse
generative process, and thereby embed the knowledge of trained spectral
diffusion model into the fusion problem in the form of a regularization term.
At last, we treat each generation step of the final optimization problem as its
subproblem, and employ the Adam to solve these subproblems in a reverse
sequence. Experimental results conducted on both synthetic and real datasets
demonstrate the effectiveness of the proposed approach. The code of the
proposed approach will be available on https://github.com/liuofficial/SDP
A New Variational Approach Based on Proximal Deep Injection and Gradient Intensity Similarity for Spatio-Spectral Image Fusion
Pansharpening is a very debated spatio-spectral fusion problem. It refers to the fusion of a high spatial resolution panchromatic image with a lower spatial but higher spectral resolution multispectral image in order to obtain an image with high resolution in both the domains. In this article, we propose a novel variational optimization-based (VO) approach to address this issue incorporating the outcome of a deep convolutional neural network (DCNN). This solution can take advantages of both the paradigms. On one hand, higher performance can be expected introducing machine learning (ML) methods based on the training by examples philosophy into VO approaches. On other hand, the combination of VO techniques with DCNNs can aid the generalization ability of these latter. In particular, we formulate a -based proximal deep injection term to evaluate the distance between the DCNN outcome, and the desired high spatial resolution multispectral image. This represents the regularization term for our VO model. Furthermore, a new data fitting term measuring the spatial fidelity is proposed. Finally, the proposed convex VO problem is efficiently solved by exploiting the framework of the alternating direction method of multipliers (ADMM), thus guaranteeing the convergence of the algorithm. Extensive experiments both on simulated, and real datasets demonstrate that the proposed approach can outperform state-of-the-art spatio-spectral fusion methods, even showing a significant generalization ability. Please find the project page at https://liangjiandeng.github.io/Projects_Res/DMPIF_2020jstars.html
Pansharpening via Frequency-Aware Fusion Network with Explicit Similarity Constraints
The process of fusing a high spatial resolution (HR) panchromatic (PAN) image
and a low spatial resolution (LR) multispectral (MS) image to obtain an HRMS
image is known as pansharpening. With the development of convolutional neural
networks, the performance of pansharpening methods has been improved, however,
the blurry effects and the spectral distortion still exist in their fusion
results due to the insufficiency in details learning and the frequency mismatch
between MSand PAN. Therefore, the improvement of spatial details at the premise
of reducing spectral distortion is still a challenge. In this paper, we propose
a frequency-aware fusion network (FAFNet) together with a novel high-frequency
feature similarity loss to address above mentioned problems. FAFNet is mainly
composed of two kinds of blocks, where the frequency aware blocks aim to
extract features in the frequency domain with the help of discrete wavelet
transform (DWT) layers, and the frequency fusion blocks reconstruct and
transform the features from frequency domain to spatial domain with the
assistance of inverse DWT (IDWT) layers. Finally, the fusion results are
obtained through a convolutional block. In order to learn the correspondence,
we also propose a high-frequency feature similarity loss to constrain the HF
features derived from PAN and MS branches, so that HF features of PAN can
reasonably be used to supplement that of MS. Experimental results on three
datasets at both reduced- and full-resolution demonstrate the superiority of
the proposed method compared with several state-of-the-art pansharpening
models.Comment: 14 page