2 research outputs found
Adversarial Spatio-Temporal Learning for Video Deblurring
Camera shake or target movement often leads to undesired blur effects in
videos captured by a hand-held camera. Despite significant efforts having been
devoted to video-deblur research, two major challenges remain: 1) how to model
the spatio-temporal characteristics across both the spatial domain (i.e., image
plane) and temporal domain (i.e., neighboring frames), and 2) how to restore
sharp image details w.r.t. the conventionally adopted metric of pixel-wise
errors. In this paper, to address the first challenge, we propose a DeBLuRring
Network (DBLRNet) for spatial-temporal learning by applying a modified 3D
convolution to both spatial and temporal domains. Our DBLRNet is able to
capture jointly spatial and temporal information encoded in neighboring frames,
which directly contributes to improved video deblur performance. To tackle the
second challenge, we leverage the developed DBLRNet as a generator in the GAN
(generative adversarial network) architecture, and employ a content loss in
addition to an adversarial loss for efficient adversarial training. The
developed network, which we name as DeBLuRring Generative Adversarial Network
(DBLRGAN), is tested on two standard benchmarks and achieves the
state-of-the-art performance.Comment: To appear in IEEE Transactions on Image Processing (TIP
Image Stitching and Rectification for Hand-Held Cameras
In this paper, we derive a new differential homography that can account for
the scanline-varying camera poses in Rolling Shutter (RS) cameras, and
demonstrate its application to carry out RS-aware image stitching and
rectification at one stroke. Despite the high complexity of RS geometry, we
focus in this paper on a special yet common input -- two consecutive frames
from a video stream, wherein the inter-frame motion is restricted from being
arbitrarily large. This allows us to adopt simpler differential motion model,
leading to a straightforward and practical minimal solver. To deal with
non-planar scene and camera parallax in stitching, we further propose an
RS-aware spatially-varying homography field in the principle of
As-Projective-As-Possible (APAP). We show superior performance over
state-of-the-art methods both in RS image stitching and rectification,
especially for images captured by hand-held shaking cameras.Comment: ECCV 2020. Project web: https://www.nec-labs.com/~mas/RS-APA