5,141 research outputs found
Learning to Synthesize Motion Blur
We present a technique for synthesizing a motion blurred image from a pair of
unblurred images captured in succession. To build this system we motivate and
design a differentiable "line prediction" layer to be used as part of a neural
network architecture, with which we can learn a system to regress from image
pairs to motion blurred images that span the capture time of the input image
pair. Training this model requires an abundance of data, and so we design and
execute a strategy for using frame interpolation techniques to generate a
large-scale synthetic dataset of motion blurred images and their respective
inputs. We additionally capture a high quality test set of real motion blurred
images, synthesized from slow motion videos, with which we evaluate our model
against several baseline techniques that can be used to synthesize motion blur.
Our model produces higher accuracy output than our baselines, and is
significantly faster than baselines with competitive accuracy.Comment: http://timothybrooks.com/tech/motion-blur/ . IEEE Conference on
Computer Vision and Pattern Recognition (CVPR), 201
PhaseNet for Video Frame Interpolation
Most approaches for video frame interpolation require accurate dense
correspondences to synthesize an in-between frame. Therefore, they do not
perform well in challenging scenarios with e.g. lighting changes or motion
blur. Recent deep learning approaches that rely on kernels to represent motion
can only alleviate these problems to some extent. In those cases, methods that
use a per-pixel phase-based motion representation have been shown to work well.
However, they are only applicable for a limited amount of motion. We propose a
new approach, PhaseNet, that is designed to robustly handle challenging
scenarios while also coping with larger motion. Our approach consists of a
neural network decoder that directly estimates the phase decomposition of the
intermediate frame. We show that this is superior to the hand-crafted
heuristics previously used in phase-based methods and also compares favorably
to recent deep learning based approaches for video frame interpolation on
challenging datasets.Comment: CVPR 201
Reblur2Deblur: Deblurring Videos via Self-Supervised Learning
Motion blur is a fundamental problem in computer vision as it impacts image
quality and hinders inference. Traditional deblurring algorithms leverage the
physics of the image formation model and use hand-crafted priors: they usually
produce results that better reflect the underlying scene, but present
artifacts. Recent learning-based methods implicitly extract the distribution of
natural images directly from the data and use it to synthesize plausible
images. Their results are impressive, but they are not always faithful to the
content of the latent image. We present an approach that bridges the two. Our
method fine-tunes existing deblurring neural networks in a self-supervised
fashion by enforcing that the output, when blurred based on the optical flow
between subsequent frames, matches the input blurry image. We show that our
method significantly improves the performance of existing methods on several
datasets both visually and in terms of image quality metrics. The supplementary
material is https://goo.gl/nYPjE
Effects of Image Degradations to CNN-based Image Classification
Just like many other topics in computer vision, image classification has
achieved significant progress recently by using deep-learning neural networks,
especially the Convolutional Neural Networks (CNN). Most of the existing works
are focused on classifying very clear natural images, evidenced by the widely
used image databases such as Caltech-256, PASCAL VOCs and ImageNet. However, in
many real applications, the acquired images may contain certain degradations
that lead to various kinds of blurring, noise, and distortions. One important
and interesting problem is the effect of such degradations to the performance
of CNN-based image classification. More specifically, we wonder whether
image-classification performance drops with each kind of degradation, whether
this drop can be avoided by including degraded images into training, and
whether existing computer vision algorithms that attempt to remove such
degradations can help improve the image-classification performance. In this
paper, we empirically study this problem for four kinds of degraded images --
hazy images, underwater images, motion-blurred images and fish-eye images. For
this study, we synthesize a large number of such degraded images by applying
respective physical models to the clear natural images and collect a new hazy
image dataset from the Internet. We expect this work can draw more interests
from the community to study the classification of degraded images
Fast and Full-Resolution Light Field Deblurring using a Deep Neural Network
Restoring a sharp light field image from its blurry input has become
essential due to the increasing popularity of parallax-based image processing.
State-of-the-art blind light field deblurring methods suffer from several
issues such as slow processing, reduced spatial size, and a limited motion blur
model. In this work, we address these challenging problems by generating a
complex blurry light field dataset and proposing a learning-based deblurring
approach. In particular, we model the full 6-degree of freedom (6-DOF) light
field camera motion, which is used to create the blurry dataset using a
combination of real light fields captured with a Lytro Illum camera, and
synthetic light field renderings of 3D scenes. Furthermore, we propose a light
field deblurring network that is built with the capability of large receptive
fields. We also introduce a simple strategy of angular sampling to train on the
large-scale blurry light field effectively. We evaluate our method through both
quantitative and qualitative measurements and demonstrate superior performance
compared to the state-of-the-art method with a massive speedup in execution
time. Our method is about 16K times faster than Srinivasan et. al. [22] and can
deblur a full-resolution light field in less than 2 seconds.Comment: 9 pages, 8 figure
Learn to Model Motion from Blurry Footages
It is difficult to recover the motion field from a real-world footage given a
mixture of camera shake and other photometric effects. In this paper we propose
a hybrid framework by interleaving a Convolutional Neural Network (CNN) and a
traditional optical flow energy. We first conduct a CNN architecture using a
novel learnable directional filtering layer. Such layer encodes the angle and
distance similarity matrix between blur and camera motion, which is able to
enhance the blur features of the camera-shake footages. The proposed CNNs are
then integrated into an iterative optical flow framework, which enable the
capability of modelling and solving both the blind deconvolution and the
optical flow estimation problems simultaneously. Our framework is trained
end-to-end on a synthetic dataset and yields competitive precision and
performance against the state-of-the-art approaches.Comment: Preprint of our paper accepted by Pattern Recognitio
Depth-aware Blending of Smoothed Images for Bokeh Effect Generation
Bokeh effect is used in photography to capture images where the closer
objects look sharp and every-thing else stays out-of-focus. Bokeh photos are
generally captured using Single Lens Reflex cameras using shallow
depth-of-field. Most of the modern smartphones can take bokeh images by
leveraging dual rear cameras or a good auto-focus hardware. However, for
smartphones with single-rear camera without a good auto-focus hardware, we have
to rely on software to generate bokeh images. This kind of system is also
useful to generate bokeh effect in already captured images. In this paper, an
end-to-end deep learning framework is proposed to generate high-quality bokeh
effect from images. The original image and different versions of smoothed
images are blended to generate Bokeh effect with the help of a monocular depth
estimation network. The proposed approach is compared against a saliency
detection based baseline and a number of approaches proposed in AIM 2019
Challenge on Bokeh Effect Synthesis. Extensive experiments are shown in order
to understand different parts of the proposed algorithm. The network is
lightweight and can process an HD image in 0.03 seconds. This approach ranked
second in AIM 2019 Bokeh effect challenge-Perceptual Track
Towards Real Scene Super-Resolution with Raw Images
Most existing super-resolution methods do not perform well in real scenarios
due to lack of realistic training data and information loss of the model input.
To solve the first problem, we propose a new pipeline to generate realistic
training data by simulating the imaging process of digital cameras. And to
remedy the information loss of the input, we develop a dual convolutional
neural network to exploit the originally captured radiance information in raw
images. In addition, we propose to learn a spatially-variant color
transformation which helps more effective color corrections. Extensive
experiments demonstrate that super-resolution with raw data helps recover fine
details and clear structures, and more importantly, the proposed network and
data generation pipeline achieve superior results for single image
super-resolution in real scenarios.Comment: Accepted in CVPR 2019, project page:
https://sites.google.com/view/xiangyuxu/rawsr_cvpr1
Dynamic Scene Deblurring using a Locally Adaptive Linear Blur Model
State-of-the-art video deblurring methods cannot handle blurry videos
recorded in dynamic scenes, since they are built under a strong assumption that
the captured scenes are static. Contrary to the existing methods, we propose a
video deblurring algorithm that can deal with general blurs inherent in dynamic
scenes. To handle general and locally varying blurs caused by various sources,
such as moving objects, camera shake, depth variation, and defocus, we estimate
pixel-wise non-uniform blur kernels. We infer bidirectional optical flows to
handle motion blurs, and also estimate Gaussian blur maps to remove optical
blur from defocus in our new blur model. Therefore, we propose a single energy
model that jointly estimates optical flows, defocus blur maps and latent
frames. We also provide a framework and efficient solvers to minimize the
proposed energy model. By optimizing the energy model, we achieve significant
improvements in removing general blurs, estimating optical flows, and extending
depth-of-field in blurry frames. Moreover, in this work, to evaluate the
performance of non-uniform deblurring methods objectively, we have constructed
a new realistic dataset with ground truths. In addition, extensive experimental
on publicly available challenging video data demonstrate that the proposed
method produces qualitatively superior performance than the state-of-the-art
methods which often fail in either deblurring or optical flow estimation
Deblurring by Realistic Blurring
Existing deep learning methods for image deblurring typically train models
using pairs of sharp images and their blurred counterparts. However,
synthetically blurring images do not necessarily model the genuine blurring
process in real-world scenarios with sufficient accuracy. To address this
problem, we propose a new method which combines two GAN models, i.e., a
learning-to-Blur GAN (BGAN) and learning-to-DeBlur GAN (DBGAN), in order to
learn a better model for image deblurring by primarily learning how to blur
images. The first model, BGAN, learns how to blur sharp images with unpaired
sharp and blurry image sets, and then guides the second model, DBGAN, to learn
how to correctly deblur such images. In order to reduce the discrepancy between
real blur and synthesized blur, a relativistic blur loss is leveraged. As an
additional contribution, this paper also introduces a Real-World Blurred Image
(RWBI) dataset including diverse blurry images. Our experiments show that the
proposed method achieves consistently superior quantitative performance as well
as higher perceptual quality on both the newly proposed dataset and the public
GOPRO dataset.Comment: Accepted by CVPR202
- …