261 research outputs found
Simultaneous Stereo Video Deblurring and Scene Flow Estimation
Videos for outdoor scene often show unpleasant blur effects due to the large
relative motion between the camera and the dynamic objects and large depth
variations. Existing works typically focus monocular video deblurring. In this
paper, we propose a novel approach to deblurring from stereo videos. In
particular, we exploit the piece-wise planar assumption about the scene and
leverage the scene flow information to deblur the image. Unlike the existing
approach [31] which used a pre-computed scene flow, we propose a single
framework to jointly estimate the scene flow and deblur the image, where the
motion cues from scene flow estimation and blur information could reinforce
each other, and produce superior results than the conventional scene flow
estimation or stereo deblurring methods. We evaluate our method extensively on
two available datasets and achieve significant improvement in flow estimation
and removing the blur effect over the state-of-the-art methods.Comment: Accepted to IEEE International Conference on Computer Vision and
Pattern Recognition (CVPR) 201
Learning to Extract a Video Sequence from a Single Motion-Blurred Image
We present a method to extract a video sequence from a single motion-blurred
image. Motion-blurred images are the result of an averaging process, where
instant frames are accumulated over time during the exposure of the sensor.
Unfortunately, reversing this process is nontrivial. Firstly, averaging
destroys the temporal ordering of the frames. Secondly, the recovery of a
single frame is a blind deconvolution task, which is highly ill-posed. We
present a deep learning scheme that gradually reconstructs a temporal ordering
by sequentially extracting pairs of frames. Our main contribution is to
introduce loss functions invariant to the temporal order. This lets a neural
network choose during training what frame to output among the possible
combinations. We also address the ill-posedness of deblurring by designing a
network with a large receptive field and implemented via resampling to achieve
a higher computational efficiency. Our proposed method can successfully
retrieve sharp image sequences from a single motion blurred image and can
generalize well on synthetic and real datasets captured with different cameras
Joint Blind Motion Deblurring and Depth Estimation of Light Field
Removing camera motion blur from a single light field is a challenging task
since it is highly ill-posed inverse problem. The problem becomes even worse
when blur kernel varies spatially due to scene depth variation and high-order
camera motion. In this paper, we propose a novel algorithm to estimate all blur
model variables jointly, including latent sub-aperture image, camera motion,
and scene depth from the blurred 4D light field. Exploiting multi-view nature
of a light field relieves the inverse property of the optimization by utilizing
strong depth cues and multi-view blur observation. The proposed joint
estimation achieves high quality light field deblurring and depth estimation
simultaneously under arbitrary 6-DOF camera motion and unconstrained scene
depth. Intensive experiment on real and synthetic blurred light field confirms
that the proposed algorithm outperforms the state-of-the-art light field
deblurring and depth estimation methods
Bringing Blurry Images Alive: High-Quality Image Restoration and Video Reconstruction
Consumer-level cameras are affordable for customers. While handy and easy to use, images and videos are likely to suffer from motion blur effect, especially under low-lighting conditions. Moreover, it is rather difficult to take high frame-rate videos due to the hardware limitations of conventional RGB-sensors. Therefore, our thesis mainly focuses on restoring high-quality (sharp, and high frame-rate) images and videos, from the low-quality (blur, and low frame-rate) ones for better practical applications. In this thesis, we mainly address the problem of how to restore a sharp image from a blurred stereo video sequence, a blurred RGB-D image, or a single blurred image. Then, by utilizing the faithful information about the motion provided by blurry effects in the image, we reconstruct high frame-rate and sharp videos based on an event camera, that brings blurry frame alive.
Stereo camera systems can provide motion information incorporated to help to remove complex spatially-varying motion blur in dynamic scenes. Given consecutive blurred stereo video frames, we recover the latent images, estimate the 3D scene flow, and segment the multiple moving objects simultaneously. We represent the dynamic scenes with the piece-wise planar model, which exploits the local structure of the scene and expresses various dynamic scenes. These three tasks are naturally connected under our model and expressed as the parameter estimation of 3D scene structure and camera motion (structure and motion for the dynamic scenes).
To tackle the challenging, minimal image deblurring case, namely, single-image deblurring, we first focus on blur caused by camera shake during the exposure time. We propose to jointly estimate the 6 DoF camera motion and remove the non-uniform blur by exploiting their underlying geometric relationships, with a single blurred RGB-D image as input. We formulate our joint deblurring and 6 DoF camera motion estimation as an energy minimization problem solved in an alternative manner.
In general cases, we solve the single-image deblurring task by studying the problem in the frequency domain. We show that the auto-correlation of the absolute phase-only image (phase-only image means the image is reconstructed only from the phase information of the blurry image) can provide faithful information about the motion (e.g., the motion direction and magnitude) that caused the blur, leading to a new and efficient blur kernel estimation approach.
Event cameras are gaining attention for they measure intensity changes (called `events') with microsecond accuracy. The event camera allows the simultaneous output of the intensity frames. However, the images are captured at a relatively low frame-rate and often suffer from motion blur. A blurred image can be regarded as the integral of a sequence of latent images, while the events indicate the changes between the latent images. Therefore, we model the blur-generation process by associating event data to a latent image. We propose a simple and effective approach, the EDI model, to reconstruct a high frame-rate, sharp video (>1000 fps) from a single blurry frame and its event data. The video generation is based on solving a simple non-convex optimization problem in a single scalar variable.
Then, we improved the EDI model by using multiple images and their events to handle flickering effects and noise in the generated video. Also, we provide a more efficient solver to minimize the proposed energy model.
Last, the blurred image and events also contribute to optical flow estimation. We propose a single image and events based optical flow estimation approach to unlock their potential applications.
In summary, this thesis addresses how to recover sharp images from blurred ones and reconstruct a high temporal resolution video from a single image and event. Our extensive experimental results demonstrate our proposed methods outperform the state-of-the-art
- …