350 research outputs found
Online Video Deblurring via Dynamic Temporal Blending Network
State-of-the-art video deblurring methods are capable of removing non-uniform
blur caused by unwanted camera shake and/or object motion in dynamic scenes.
However, most existing methods are based on batch processing and thus need
access to all recorded frames, rendering them computationally demanding and
time consuming and thus limiting their practical use. In contrast, we propose
an online (sequential) video deblurring method based on a spatio-temporal
recurrent network that allows for real-time performance. In particular, we
introduce a novel architecture which extends the receptive field while keeping
the overall size of the network small to enable fast execution. In doing so,
our network is able to remove even large blur caused by strong camera shake
and/or fast moving objects. Furthermore, we propose a novel network layer that
enforces temporal consistency between consecutive frames by dynamic temporal
blending which compares and adaptively (at test time) shares features obtained
at different time steps. We show the superiority of the proposed method in an
extensive experimental evaluation.Comment: 10 page
Learning Blind Motion Deblurring
As handheld video cameras are now commonplace and available in every
smartphone, images and videos can be recorded almost everywhere at anytime.
However, taking a quick shot frequently yields a blurry result due to unwanted
camera shake during recording or moving objects in the scene. Removing these
artifacts from the blurry recordings is a highly ill-posed problem as neither
the sharp image nor the motion blur kernel is known. Propagating information
between multiple consecutive blurry observations can help restore the desired
sharp image or video. Solutions for blind deconvolution based on neural
networks rely on a massive amount of ground-truth data which is hard to
acquire. In this work, we propose an efficient approach to produce a
significant amount of realistic training data and introduce a novel recurrent
network architecture to deblur frames taking temporal information into account,
which can efficiently handle arbitrary spatial and temporal input sizes. We
demonstrate the versatility of our approach in a comprehensive comparison on a
number of challening real-world examples.Comment: International Conference on Computer Vision (ICCV) (2017
Towards Interpretable Video Super-Resolution via Alternating Optimization
In this paper, we study a practical space-time video super-resolution (STVSR)
problem which aims at generating a high-framerate high-resolution sharp video
from a low-framerate low-resolution blurry video. Such problem often occurs
when recording a fast dynamic event with a low-framerate and low-resolution
camera, and the captured video would suffer from three typical issues: i)
motion blur occurs due to object/camera motions during exposure time; ii)
motion aliasing is unavoidable when the event temporal frequency exceeds the
Nyquist limit of temporal sampling; iii) high-frequency details are lost
because of the low spatial sampling rate. These issues can be alleviated by a
cascade of three separate sub-tasks, including video deblurring, frame
interpolation, and super-resolution, which, however, would fail to capture the
spatial and temporal correlations among video sequences. To address this, we
propose an interpretable STVSR framework by leveraging both model-based and
learning-based methods. Specifically, we formulate STVSR as a joint video
deblurring, frame interpolation, and super-resolution problem, and solve it as
two sub-problems in an alternate way. For the first sub-problem, we derive an
interpretable analytical solution and use it as a Fourier data transform layer.
Then, we propose a recurrent video enhancement layer for the second sub-problem
to further recover high-frequency details. Extensive experiments demonstrate
the superiority of our method in terms of quantitative metrics and visual
quality.Comment: ECCV 202
VJT: A Video Transformer on Joint Tasks of Deblurring, Low-light Enhancement and Denoising
Video restoration task aims to recover high-quality videos from low-quality
observations. This contains various important sub-tasks, such as video
denoising, deblurring and low-light enhancement, since video often faces
different types of degradation, such as blur, low light, and noise. Even worse,
these kinds of degradation could happen simultaneously when taking videos in
extreme environments. This poses significant challenges if one wants to remove
these artifacts at the same time. In this paper, to the best of our knowledge,
we are the first to propose an efficient end-to-end video transformer approach
for the joint task of video deblurring, low-light enhancement, and denoising.
This work builds a novel multi-tier transformer where each tier uses a
different level of degraded video as a target to learn the features of video
effectively. Moreover, we carefully design a new tier-to-tier feature fusion
scheme to learn video features incrementally and accelerate the training
process with a suitable adaptive weighting scheme. We also provide a new
Multiscene-Lowlight-Blur-Noise (MLBN) dataset, which is generated according to
the characteristics of the joint task based on the RealBlur dataset and YouTube
videos to simulate realistic scenes as far as possible. We have conducted
extensive experiments, compared with many previous state-of-the-art methods, to
show the effectiveness of our approach clearly.Comment: 12 pages,8 figure
- …