543 research outputs found
Learning how to be robust: Deep polynomial regression
Polynomial regression is a recurrent problem with a large number of
applications. In computer vision it often appears in motion analysis. Whatever
the application, standard methods for regression of polynomial models tend to
deliver biased results when the input data is heavily contaminated by outliers.
Moreover, the problem is even harder when outliers have strong structure.
Departing from problem-tailored heuristics for robust estimation of parametric
models, we explore deep convolutional neural networks. Our work aims to find a
generic approach for training deep regression models without the explicit need
of supervised annotation. We bypass the need for a tailored loss function on
the regression parameters by attaching to our model a differentiable hard-wired
decoder corresponding to the polynomial operation at hand. We demonstrate the
value of our findings by comparing with standard robust regression methods.
Furthermore, we demonstrate how to use such models for a real computer vision
problem, i.e., video stabilization. The qualitative and quantitative
experiments show that neural networks are able to learn robustness for general
polynomial regression, with results that well overpass scores of traditional
robust estimation methods.Comment: 18 pages, conferenc
Fast Full-frame Video Stabilization with Iterative Optimization
Video stabilization refers to the problem of transforming a shaky video into
a visually pleasing one. The question of how to strike a good trade-off between
visual quality and computational speed has remained one of the open challenges
in video stabilization. Inspired by the analogy between wobbly frames and
jigsaw puzzles, we propose an iterative optimization-based learning approach
using synthetic datasets for video stabilization, which consists of two
interacting submodules: motion trajectory smoothing and full-frame outpainting.
First, we develop a two-level (coarse-to-fine) stabilizing algorithm based on
the probabilistic flow field. The confidence map associated with the estimated
optical flow is exploited to guide the search for shared regions through
backpropagation. Second, we take a divide-and-conquer approach and propose a
novel multiframe fusion strategy to render full-frame stabilized views. An
important new insight brought about by our iterative optimization approach is
that the target video can be interpreted as the fixed point of nonlinear
mapping for video stabilization. We formulate video stabilization as a problem
of minimizing the amount of jerkiness in motion trajectories, which guarantees
convergence with the help of fixed-point theory. Extensive experimental results
are reported to demonstrate the superiority of the proposed approach in terms
of computational speed and visual quality. The code will be available on
GitHub.Comment: Accepted by ICCV202
GlobalFlowNet: Video Stabilization using Deep Distilled Global Motion Estimates
Videos shot by laymen using hand-held cameras contain undesirable shaky
motion. Estimating the global motion between successive frames, in a manner not
influenced by moving objects, is central to many video stabilization
techniques, but poses significant challenges. A large body of work uses 2D
affine transformations or homography for the global motion. However, in this
work, we introduce a more general representation scheme, which adapts any
existing optical flow network to ignore the moving objects and obtain a
spatially smooth approximation of the global motion between video frames. We
achieve this by a knowledge distillation approach, where we first introduce a
low pass filter module into the optical flow network to constrain the predicted
optical flow to be spatially smooth. This becomes our student network, named as
\textsc{GlobalFlowNet}. Then, using the original optical flow network as the
teacher network, we train the student network using a robust loss function.
Given a trained \textsc{GlobalFlowNet}, we stabilize videos using a two stage
process. In the first stage, we correct the instability in affine parameters
using a quadratic programming approach constrained by a user-specified cropping
limit to control loss of field of view. In the second stage, we stabilize the
video further by smoothing global motion parameters, expressed using a small
number of discrete cosine transform coefficients. In extensive experiments on a
variety of different videos, our technique outperforms state of the art
techniques in terms of subjective quality and different quantitative measures
of video stability. The source code is publicly available at
\href{https://github.com/GlobalFlowNet/GlobalFlowNet}{https://github.com/GlobalFlowNet/GlobalFlowNet}Comment: Accepted in WACV 202
Source Camera Verification from Strongly Stabilized Videos
Image stabilization performed during imaging and/or post-processing poses one
of the most significant challenges to photo-response non-uniformity based
source camera attribution from videos. When performed digitally, stabilization
involves cropping, warping, and inpainting of video frames to eliminate
unwanted camera motion. Hence, successful attribution requires the inversion of
these transformations in a blind manner. To address this challenge, we
introduce a source camera verification method for videos that takes into
account the spatially variant nature of stabilization transformations and
assumes a larger degree of freedom in their search. Our method identifies
transformations at a sub-frame level, incorporates a number of constraints to
validate their correctness, and offers computational flexibility in the search
for the correct transformation. The method also adopts a holistic approach in
countering disruptive effects of other video generation steps, such as video
coding and downsizing, for more reliable attribution. Tests performed on one
public and two custom datasets show that the proposed method is able to verify
the source of 23-30% of all videos that underwent stronger stabilization,
depending on computation load, without a significant impact on false
attribution
- …