1,619 research outputs found
A Fusion Approach for Multi-Frame Optical Flow Estimation
To date, top-performing optical flow estimation methods only take pairs of
consecutive frames into account. While elegant and appealing, the idea of using
more than two frames has not yet produced state-of-the-art results. We present
a simple, yet effective fusion approach for multi-frame optical flow that
benefits from longer-term temporal cues. Our method first warps the optical
flow from previous frames to the current, thereby yielding multiple plausible
estimates. It then fuses the complementary information carried by these
estimates into a new optical flow field. At the time of writing, our method
ranks first among published results in the MPI Sintel and KITTI 2015
benchmarks. Our models will be available on https://github.com/NVlabs/PWC-Net.Comment: Work accepted at IEEE Winter Conference on Applications of Computer
Vision (WACV 2019
Convective regularization for optical flow
We argue that the time derivative in a fixed coordinate frame may not be the
most appropriate measure of time regularity of an optical flow field. Instead,
for a given velocity field we consider the convective acceleration which describes the acceleration of objects moving according to
. Consequently we investigate the suitability of the nonconvex functional
as a regularization term for optical flow. We
demonstrate that this term acts as both a spatial and a temporal regularizer
and has an intrinsic edge-preserving property. We incorporate it into a
contrast invariant and time-regularized variant of the Horn-Schunck functional,
prove existence of minimizers and verify experimentally that it addresses some
of the problems of basic quadratic models. For the minimization we use an
iterative scheme that approximates the original nonlinear problem with a
sequence of linear ones. We believe that the convective acceleration may be
gainfully introduced in a variety of optical flow models
Optic flow calculations with nonlinear smoothness terms extended into the temporal domain
Non-quadratic variational regularization is a well known and powerful approach for the discontinuity-preserving computation of optical flow. In the present paper, we introduce an extension of nonlinear spatial smoothness terms to nonlinear spatio-temporal and flow-driven regularization. To assess the performance of our approach, the implementation is purely based on the corresponding reaction-diffusion system and dispenses with any pre-smoothing typically used for canceling out noise and estimating partial derivatives. Results for real-world scenes show that our spatio-temporal approach (i) improves optical flow fields significantly, (ii) smoothes out background noise efficiently, and (iii) enhances true motion boundaries. The computational costs required are only twice as high as with a pure 2D spatial approach
Variational optic flow computation with a spatio-temporal smoothness constraint
Nonquadratic variational regularization is a well-known and powerful approach for the discontinuity-preserving computation of optic flow. In the present paper, we consider an extension of flow-driven spatial smoothness terms to spatio-temporal regularizers. Our method leads to a rotationally invariant and time symmetrie convex optimization problem. It has a unique minimum that can be found in a stable way by standard algorithms such as gradient descent. Since the convexity guarantees global convergence, the result does not depend on the flow initialization. An iterative algorithm is presented that is not difficult to implement. Qualitative and quantitative results for synthetic and real-world scenes show that our spatio-temporal approach (i) improves optic flow fields significantly, (ii) smoothes out background noise efficiently, and (iii) preserves true motion boundaries. The computational costs are only 50 % higher than for a pure spatial approach applied to all subsequent image pairs of the sequence
Lucas/Kanade meets Horn/Schunck : combining local and global optic flow methods
Differential methods belong to the most widely used techniques for optic flow computation in image sequences. They can be classified into local methods such as the Lucas-Kanade technique or Bigün\u27s structure tensor method, and into global methods such as the Horn/Schunck approach and its extensions. Often local methods are more robust under noise, while global techniques yield dense flow fields. The goal of this paper is to contribute to a better understanding and the design of differential methods in four ways: (i) We juxtapose the role of smoothing/regularisation processes that are required in local and global differential methods for optic flow computation. (ii) This discussion motivates us to describe and evaluate a novel method that combines important advantages of local and global approaches: It yields dense flow fields that are robust against noise. (iii) Spatiotemproal and nonlinear extensions to this hybrid method are presented. (iv) We propose a simple confidence measure for optic flow methods that minimise energy functionals. It allows to sparsify a dense flow field gradually, depending on the reliability required for the resulting flow. Comparisons with experiments from the literature demonstrate the favourable performance of the proposed methods and the confidence measure
Highly accurate optic flow computation with theoretically justified warping
In this paper, we suggest a variational model for optic flow computation based on non-linearised and higher order constancy assumptions. Besides the common grey value constancy assumption, also gradient constancy, as well as the constancy of the Hessian and the Laplacian are proposed. Since the model strictly refrains from a linearisation of these assumptions, it is also capable to deal with large displacements. For the minimisation of the rather complex energy functional, we present an efficient numerical scheme employing two nested fixed point iterations. Following a coarse-to-fine strategy it turns out that there is a theoretical foundation of so-called warping techniques hitherto justified only on an experimental basis. Since our algorithm consists of the integration of various concepts, ranging from different constancy assumptions to numerical implementation issues, a detailed account of the effect of each of these concepts is included in the experimental section. The superior performance of the proposed method shows up by significantly smaller estimation errors when compared to previous techniques. Further experiments also confirm excellent robustness under noise and insensitivity to parameter variations
Optical Flow on Moving Manifolds
Optical flow is a powerful tool for the study and analysis of motion in a
sequence of images. In this article we study a Horn-Schunck type
spatio-temporal regularization functional for image sequences that have a
non-Euclidean, time varying image domain. To that end we construct a Riemannian
metric that describes the deformation and structure of this evolving surface.
The resulting functional can be seen as natural geometric generalization of
previous work by Weickert and Schn\"orr (2001) and Lef\`evre and Baillet (2008)
for static image domains. In this work we show the existence and wellposedness
of the corresponding optical flow problem and derive necessary and sufficient
optimality conditions. We demonstrate the functionality of our approach in a
series of experiments using both synthetic and real data.Comment: 26 pages, 6 figure
Multi-Scale 3D Scene Flow from Binocular Stereo Sequences
Scene flow methods estimate the three-dimensional motion field for points in the world, using multi-camera video data. Such methods combine multi-view reconstruction with motion estimation. This paper describes an alternative formulation for dense scene flow estimation that provides reliable results using only two cameras by fusing stereo and optical flow estimation into a single coherent framework. Internally, the proposed algorithm generates probability distributions for optical flow and disparity. Taking into account the uncertainty in the intermediate stages allows for more reliable estimation of the 3D scene flow than previous methods allow. To handle the aperture problems inherent in the estimation of optical flow and disparity, a multi-scale method along with a novel region-based technique is used within a regularized solution. This combined approach both preserves discontinuities and prevents over-regularization – two problems commonly associated with the basic multi-scale approaches. Experiments with synthetic and real test data demonstrate the strength of the proposed approach.National Science Foundation (CNS-0202067, IIS-0208876); Office of Naval Research (N00014-03-1-0108
Optical Flow Estimation in the Deep Learning Age
Akin to many subareas of computer vision, the recent advances in deep
learning have also significantly influenced the literature on optical flow.
Previously, the literature had been dominated by classical energy-based models,
which formulate optical flow estimation as an energy minimization problem.
However, as the practical benefits of Convolutional Neural Networks (CNNs) over
conventional methods have become apparent in numerous areas of computer vision
and beyond, they have also seen increased adoption in the context of motion
estimation to the point where the current state of the art in terms of accuracy
is set by CNN approaches. We first review this transition as well as the
developments from early work to the current state of CNNs for optical flow
estimation. Alongside, we discuss some of their technical details and compare
them to recapitulate which technical contribution led to the most significant
accuracy improvements. Then we provide an overview of the various optical flow
approaches introduced in the deep learning age, including those based on
alternative learning paradigms (e.g., unsupervised and semi-supervised methods)
as well as the extension to the multi-frame case, which is able to yield
further accuracy improvements.Comment: To appear as a book chapter in Modelling Human Motion, N. Noceti, A.
Sciutti and F. Rea, Eds., Springer, 202
- …