56 research outputs found
Target-adaptive CNN-based pansharpening
We recently proposed a convolutional neural network (CNN) for remote sensing
image pansharpening obtaining a significant performance gain over the state of
the art. In this paper, we explore a number of architectural and training
variations to this baseline, achieving further performance gains with a
lightweight network which trains very fast. Leveraging on this latter property,
we propose a target-adaptive usage modality which ensures a very good
performance also in the presence of a mismatch w.r.t. the training set, and
even across different sensors. The proposed method, published online as an
off-the-shelf software tool, allows users to perform fast and high-quality
CNN-based pansharpening of their own target images on general-purpose hardware
Model Inspired Autoencoder for Unsupervised Hyperspectral Image Super-Resolution
This paper focuses on hyperspectral image (HSI) super-resolution that aims to
fuse a low-spatial-resolution HSI and a high-spatial-resolution multispectral
image to form a high-spatial-resolution HSI (HR-HSI). Existing deep
learning-based approaches are mostly supervised that rely on a large number of
labeled training samples, which is unrealistic. The commonly used model-based
approaches are unsupervised and flexible but rely on hand-craft priors.
Inspired by the specific properties of model, we make the first attempt to
design a model inspired deep network for HSI super-resolution in an
unsupervised manner. This approach consists of an implicit autoencoder network
built on the target HR-HSI that treats each pixel as an individual sample. The
nonnegative matrix factorization (NMF) of the target HR-HSI is integrated into
the autoencoder network, where the two NMF parts, spectral and spatial
matrices, are treated as decoder parameters and hidden outputs respectively. In
the encoding stage, we present a pixel-wise fusion model to estimate hidden
outputs directly, and then reformulate and unfold the model's algorithm to form
the encoder network. With the specific architecture, the proposed network is
similar to a manifold prior-based model, and can be trained patch by patch
rather than the entire image. Moreover, we propose an additional unsupervised
network to estimate the point spread function and spectral response function.
Experimental results conducted on both synthetic and real datasets demonstrate
the effectiveness of the proposed approach
Deep Learning based data-fusion methods for remote sensing applications
In the last years, an increasing number of remote sensing sensors have been launched to orbit around the Earth, with a continuously growing production of massive data, that are useful for a large number of monitoring applications, especially for the monitoring task. Despite modern optical sensors provide rich spectral information about Earth's surface, at very high resolution, they are weather-sensitive. On the other hand, SAR images are always available also in presence of clouds and are almost weather-insensitive, as well as daynight available, but they do not provide a rich spectral information and are severely affected by speckle "noise" that make difficult the information extraction. For the above reasons it is worth and challenging to fuse data provided by different sources and/or acquired at different times, in order to leverage on their diversity and complementarity to retrieve the target information. Motivated by the success of the employment of Deep Learning methods in many image processing tasks, in this thesis it has been faced different typical remote sensing data-fusion problems by means of suitably designed Convolutional Neural Networks
Panchromatic and multispectral image fusion for remote sensing and earth observation: Concepts, taxonomy, literature review, evaluation methodologies and challenges ahead
Panchromatic and multispectral image fusion, termed pan-sharpening, is to merge the spatial and spectral information of the source images into a fused one, which has a higher spatial and spectral resolution and is more reliable for downstream tasks compared with any of the source images. It has been widely applied to image interpretation and pre-processing of various applications. A large number of methods have been proposed to achieve better fusion results by considering the spatial and spectral relationships among panchromatic and multispectral images. In recent years, the fast development of artificial intelligence (AI) and deep learning (DL) has significantly enhanced the development of pan-sharpening techniques. However, this field lacks a comprehensive overview of recent advances boosted by the rise of AI and DL. This paper provides a comprehensive review of a variety of pan-sharpening methods that adopt four different paradigms, i.e., component substitution, multiresolution analysis, degradation model, and deep neural networks. As an important aspect of pan-sharpening, the evaluation of the fused image is also outlined to present various assessment methods in terms of reduced-resolution and full-resolution quality measurement. Then, we conclude this paper by discussing the existing limitations, difficulties, and challenges of pan-sharpening techniques, datasets, and quality assessment. In addition, the survey summarizes the development trends in these areas, which provide useful methodological practices for researchers and professionals. Finally, the developments in pan-sharpening are summarized in the conclusion part. The aim of the survey is to serve as a referential starting point for newcomers and a common point of agreement around the research directions to be followed in this exciting area
DDRF: Denoising Diffusion Model for Remote Sensing Image Fusion
Denosing diffusion model, as a generative model, has received a lot of
attention in the field of image generation recently, thanks to its powerful
generation capability. However, diffusion models have not yet received
sufficient research in the field of image fusion. In this article, we introduce
diffusion model to the image fusion field, treating the image fusion task as
image-to-image translation and designing two different conditional injection
modulation modules (i.e., style transfer modulation and wavelet modulation) to
inject coarse-grained style information and fine-grained high-frequency and
low-frequency information into the diffusion UNet, thereby generating fused
images. In addition, we also discussed the residual learning and the selection
of training objectives of the diffusion model in the image fusion task.
Extensive experimental results based on quantitative and qualitative
assessments compared with benchmarks demonstrates state-of-the-art results and
good generalization performance in image fusion tasks. Finally, it is hoped
that our method can inspire other works and gain insight into this field to
better apply the diffusion model to image fusion tasks. Code shall be released
for better reproducibility
Computationally Inexpensive Landsat 8 Operational Land Imager (OLI) Pansharpening
Pansharpening algorithms fuse higher spatial resolution panchromatic with lower spatial resolution multispectral imagery to create higher spatial resolution multispectral images. The free-availability and systematic global acquisition of Landsat 8 data indicate an expected need for global coverage and so computationally efficient Landsat 8 pansharpening. This study adapts and evaluates the established, and relatively computationally inexpensive, Brovey and context adaptive Gram Schmidt component substitution (CS) pansharpening methods for application to the Landsat 8 15 m panchromatic and 30 m red, green, blue, and near-infrared bands. The intensity images used by these CS pansharpening methods are derived as a weighted linear combination of the multispectral bands in three different ways using band spectral weights set (i) equally as the reciprocal of the number of bands; (ii) using fixed Landsat 8 spectral response function based (SRFB) weights derived considering laboratory spectra; and (iii) using image specific spectral weights derived by regression between the multispectral and the degraded panchromatic bands. The spatial and spectral distortion and computational cost of the different methods are assessed using Landsat 8 test images acquired over agricultural scenes in South Dakota, China, and India. The results of this study indicate that, for global Landsat 8 application, the context adaptive Gram Schmidt pansharpening with an intensity image defined using the SRFB spectral weights is appropriate. The context adaptive Gram Schmidt pansharpened results had lower distortion than the Brovey results and the least distortion was found using intensity images derived using the SRFB and image specific spectral weights but the computational cost using the image specific weights was greater than the using the SRFB weights. Recommendations for large area Landsat 8 pansharpening application are described briefly and the SRFB spectral weights are provided so users may implement computationally inexpensive Landsat 8 pansharpening themselves
- …