61 research outputs found
Depth Estimation and Image Restoration by Deep Learning from Defocused Images
Monocular depth estimation and image deblurring are two fundamental tasks in
computer vision, given their crucial role in understanding 3D scenes.
Performing any of them by relying on a single image is an ill-posed problem.
The recent advances in the field of Deep Convolutional Neural Networks (DNNs)
have revolutionized many tasks in computer vision, including depth estimation
and image deblurring. When it comes to using defocused images, the depth
estimation and the recovery of the All-in-Focus (Aif) image become related
problems due to defocus physics. Despite this, most of the existing models
treat them separately. There are, however, recent models that solve these
problems simultaneously by concatenating two networks in a sequence to first
estimate the depth or defocus map and then reconstruct the focused image based
on it. We propose a DNN that solves the depth estimation and image deblurring
in parallel. Our Two-headed Depth Estimation and Deblurring Network (2HDED:NET)
extends a conventional Depth from Defocus (DFD) networks with a deblurring
branch that shares the same encoder as the depth branch. The proposed method
has been successfully tested on two benchmarks, one for indoor and the other
for outdoor scenes: NYU-v2 and Make3D. Extensive experiments with 2HDED:NET on
these benchmarks have demonstrated superior or close performances to those of
the state-of-the-art models for depth estimation and image deblurring
A Multi-scale Generalized Shrinkage Threshold Network for Image Blind Deblurring in Remote Sensing
Remote sensing images are essential for many earth science applications, but
their quality can be degraded due to limitations in sensor technology and
complex imaging environments. To address this, various remote sensing image
deblurring methods have been developed to restore sharp, high-quality images
from degraded observational data. However, most traditional model-based
deblurring methods usually require predefined hand-craft prior assumptions,
which are difficult to handle in complex applications, and most deep
learning-based deblurring methods are designed as a black box, lacking
transparency and interpretability. In this work, we propose a novel blind
deblurring learning framework based on alternating iterations of shrinkage
thresholds, alternately updating blurring kernels and images, with the
theoretical foundation of network design. Additionally, we propose a learnable
blur kernel proximal mapping module to improve the blur kernel evaluation in
the kernel domain. Then, we proposed a deep proximal mapping module in the
image domain, which combines a generalized shrinkage threshold operator and a
multi-scale prior feature extraction block. This module also introduces an
attention mechanism to adaptively adjust the prior importance, thus avoiding
the drawbacks of hand-crafted image prior terms. Thus, a novel multi-scale
generalized shrinkage threshold network (MGSTNet) is designed to specifically
focus on learning deep geometric prior features to enhance image restoration.
Experiments demonstrate the superiority of our MGSTNet framework on remote
sensing image datasets compared to existing deblurring methods.Comment: 12 pages
Towards Interpretable Video Super-Resolution via Alternating Optimization
In this paper, we study a practical space-time video super-resolution (STVSR)
problem which aims at generating a high-framerate high-resolution sharp video
from a low-framerate low-resolution blurry video. Such problem often occurs
when recording a fast dynamic event with a low-framerate and low-resolution
camera, and the captured video would suffer from three typical issues: i)
motion blur occurs due to object/camera motions during exposure time; ii)
motion aliasing is unavoidable when the event temporal frequency exceeds the
Nyquist limit of temporal sampling; iii) high-frequency details are lost
because of the low spatial sampling rate. These issues can be alleviated by a
cascade of three separate sub-tasks, including video deblurring, frame
interpolation, and super-resolution, which, however, would fail to capture the
spatial and temporal correlations among video sequences. To address this, we
propose an interpretable STVSR framework by leveraging both model-based and
learning-based methods. Specifically, we formulate STVSR as a joint video
deblurring, frame interpolation, and super-resolution problem, and solve it as
two sub-problems in an alternate way. For the first sub-problem, we derive an
interpretable analytical solution and use it as a Fourier data transform layer.
Then, we propose a recurrent video enhancement layer for the second sub-problem
to further recover high-frequency details. Extensive experiments demonstrate
the superiority of our method in terms of quantitative metrics and visual
quality.Comment: ECCV 202
Joint deconvolution and demosaicing
International audienceWe present a new method to jointly perform deblurring and color- demosaicing of RGB images. Our method is derived following an inverse problem approach in a MAP framework. To avoid noise am- plification and allow for interpolation of missing data, we make use of edge-preserving spatial regularization and spectral regularization. We demonstrate the improvements brought by our algorithm by processing both simulated and real RGB images obtained with a Bayer's color filter and with different types of blurring
- …