43 research outputs found
Scene-adapted plug-and-play algorithm with convergence guarantees
Recent frameworks, such as the so-called plug-and-play, allow us to leverage
the developments in image denoising to tackle other, and more involved,
problems in image processing. As the name suggests, state-of-the-art denoisers
are plugged into an iterative algorithm that alternates between a denoising
step and the inversion of the observation operator. While these tools offer
flexibility, the convergence of the resulting algorithm may be difficult to
analyse. In this paper, we plug a state-of-the-art denoiser, based on a
Gaussian mixture model, in the iterations of an alternating direction method of
multipliers and prove the algorithm is guaranteed to converge. Moreover, we
build upon the concept of scene-adapted priors where we learn a model targeted
to a specific scene being imaged, and apply the proposed method to address the
hyperspectral sharpening problem
A Spectral Diffusion Prior for Hyperspectral Image Super-Resolution
Fusion-based hyperspectral image (HSI) super-resolution aims to produce a
high-spatial-resolution HSI by fusing a low-spatial-resolution HSI and a
high-spatial-resolution multispectral image. Such a HSI super-resolution
process can be modeled as an inverse problem, where the prior knowledge is
essential for obtaining the desired solution. Motivated by the success of
diffusion models, we propose a novel spectral diffusion prior for fusion-based
HSI super-resolution. Specifically, we first investigate the spectrum
generation problem and design a spectral diffusion model to model the spectral
data distribution. Then, in the framework of maximum a posteriori, we keep the
transition information between every two neighboring states during the reverse
generative process, and thereby embed the knowledge of trained spectral
diffusion model into the fusion problem in the form of a regularization term.
At last, we treat each generation step of the final optimization problem as its
subproblem, and employ the Adam to solve these subproblems in a reverse
sequence. Experimental results conducted on both synthetic and real datasets
demonstrate the effectiveness of the proposed approach. The code of the
proposed approach will be available on https://github.com/liuofficial/SDP
Deep Hyperspectral and Multispectral Image Fusion with Inter-image Variability
Hyperspectral and multispectral image fusion allows us to overcome the
hardware limitations of hyperspectral imaging systems inherent to their lower
spatial resolution. Nevertheless, existing algorithms usually fail to consider
realistic image acquisition conditions. This paper presents a general imaging
model that considers inter-image variability of data from heterogeneous sources
and flexible image priors. The fusion problem is stated as an optimization
problem in the maximum a posteriori framework. We introduce an original image
fusion method that, on the one hand, solves the optimization problem accounting
for inter-image variability with an iteratively reweighted scheme and, on the
other hand, that leverages light-weight CNN-based networks to learn realistic
image priors from data. In addition, we propose a zero-shot strategy to
directly learn the image-specific prior of the latent images in an unsupervised
manner. The performance of the algorithm is illustrated with real data subject
to inter-image variability.Comment: IEEE Trans. Geosci. Remote sens., to be published. Manuscript
submitted August 23, 2022; revised Dec. 15, 2022, and Mar. 13, 2023; and
accepted Apr. 07, 202
Convolutional Deblurring for Natural Imaging
In this paper, we propose a novel design of image deblurring in the form of
one-shot convolution filtering that can directly convolve with naturally
blurred images for restoration. The problem of optical blurring is a common
disadvantage to many imaging applications that suffer from optical
imperfections. Despite numerous deconvolution methods that blindly estimate
blurring in either inclusive or exclusive forms, they are practically
challenging due to high computational cost and low image reconstruction
quality. Both conditions of high accuracy and high speed are prerequisites for
high-throughput imaging platforms in digital archiving. In such platforms,
deblurring is required after image acquisition before being stored, previewed,
or processed for high-level interpretation. Therefore, on-the-fly correction of
such images is important to avoid possible time delays, mitigate computational
expenses, and increase image perception quality. We bridge this gap by
synthesizing a deconvolution kernel as a linear combination of Finite Impulse
Response (FIR) even-derivative filters that can be directly convolved with
blurry input images to boost the frequency fall-off of the Point Spread
Function (PSF) associated with the optical blur. We employ a Gaussian low-pass
filter to decouple the image denoising problem for image edge deblurring.
Furthermore, we propose a blind approach to estimate the PSF statistics for two
Gaussian and Laplacian models that are common in many imaging pipelines.
Thorough experiments are designed to test and validate the efficiency of the
proposed method using 2054 naturally blurred images across six imaging
applications and seven state-of-the-art deconvolution methods.Comment: 15 pages, for publication in IEEE Transaction Image Processin
Joint Nonlocal, Spectral, and Similarity Low-Rank Priors for Hyperspectral-Multispectral Image Fusion
The fusion of a low-spatial-and-high-spectral resolution hyperspectral image (HSI) with a high-spatial-and-low-spectral resolution multispectral image (MSI) allows synthesizing a high-resolution image (HRI), supporting remote sensing applications, such as disaster management, material identification, and precision agriculture. Unlike existing variational methods using low-rank regularizations separately, we present an HSI-MSI fusion method promoting various low-rank regularizations jointly. Our method refines the HRI spatial and spectral correlations from the individual HSI and MSI data through the proper plug-and-play (PnP) of a nonlocal patch-based denoiser in the alternating direction method of multipliers (ADMM). Notably, we consider the nonlocal self-similarity, the spectral low-rank, and introduce a rank-one similarity prior. Furthermore, we demonstrate via an extensive empirical study that the rank-one similarity prior is an inherent characteristic of the HRI. Simulations over standard benchmark datasets show the effectiveness of the proposed HSI-MSI fusion outperforming state-of-the-art methods, particularly in recovering low-contrast areas.acceptedVersionPeer reviewe
Recent Advances in Image Restoration with Applications to Real World Problems
In the past few decades, imaging hardware has improved tremendously in terms of resolution, making widespread usage of images in many diverse applications on Earth and planetary missions. However, practical issues associated with image acquisition are still affecting image quality. Some of these issues such as blurring, measurement noise, mosaicing artifacts, low spatial or spectral resolution, etc. can seriously affect the accuracy of the aforementioned applications. This book intends to provide the reader with a glimpse of the latest developments and recent advances in image restoration, which includes image super-resolution, image fusion to enhance spatial, spectral resolution, and temporal resolutions, and the generation of synthetic images using deep learning techniques. Some practical applications are also included
Joint demosaicing and fusion of multiresolution coded acquisitions: A unified image formation and reconstruction method
Novel optical imaging devices allow for hybrid acquisition modalities such as
compressed acquisitions with locally different spatial and spectral resolutions
captured by a single focal plane array. In this work, we propose to model the
capturing system of a multiresolution coded acquisition (MRCA) in a unified
framework, which natively includes conventional systems such as those based on
spectral/color filter arrays, compressed coded apertures, and multiresolution
sensing. We also propose a model-based image reconstruction algorithm
performing a joint demosaicing and fusion (JoDeFu) of any acquisition modeled
in the MRCA framework. The JoDeFu reconstruction algorithm solves an inverse
problem with a proximal splitting technique and is able to reconstruct an
uncompressed image datacube at the highest available spatial and spectral
resolution. An implementation of the code is available at
https://github.com/danaroth83/jodefu.Comment: 15 pages, 7 figures; regular pape
Graph Spectral Image Processing
Recent advent of graph signal processing (GSP) has spurred intensive studies
of signals that live naturally on irregular data kernels described by graphs
(e.g., social networks, wireless sensor networks). Though a digital image
contains pixels that reside on a regularly sampled 2D grid, if one can design
an appropriate underlying graph connecting pixels with weights that reflect the
image structure, then one can interpret the image (or image patch) as a signal
on a graph, and apply GSP tools for processing and analysis of the signal in
graph spectral domain. In this article, we overview recent graph spectral
techniques in GSP specifically for image / video processing. The topics covered
include image compression, image restoration, image filtering and image
segmentation