38,481 research outputs found
Optical Flow Estimation in the Deep Learning Age
Akin to many subareas of computer vision, the recent advances in deep
learning have also significantly influenced the literature on optical flow.
Previously, the literature had been dominated by classical energy-based models,
which formulate optical flow estimation as an energy minimization problem.
However, as the practical benefits of Convolutional Neural Networks (CNNs) over
conventional methods have become apparent in numerous areas of computer vision
and beyond, they have also seen increased adoption in the context of motion
estimation to the point where the current state of the art in terms of accuracy
is set by CNN approaches. We first review this transition as well as the
developments from early work to the current state of CNNs for optical flow
estimation. Alongside, we discuss some of their technical details and compare
them to recapitulate which technical contribution led to the most significant
accuracy improvements. Then we provide an overview of the various optical flow
approaches introduced in the deep learning age, including those based on
alternative learning paradigms (e.g., unsupervised and semi-supervised methods)
as well as the extension to the multi-frame case, which is able to yield
further accuracy improvements.Comment: To appear as a book chapter in Modelling Human Motion, N. Noceti, A.
Sciutti and F. Rea, Eds., Springer, 202
Occlusion Aware Unsupervised Learning of Optical Flow
It has been recently shown that a convolutional neural network can learn
optical flow estimation with unsupervised learning. However, the performance of
the unsupervised methods still has a relatively large gap compared to its
supervised counterpart. Occlusion and large motion are some of the major
factors that limit the current unsupervised learning of optical flow methods.
In this work we introduce a new method which models occlusion explicitly and a
new warping way that facilitates the learning of large motion. Our method shows
promising results on Flying Chairs, MPI-Sintel and KITTI benchmark datasets.
Especially on KITTI dataset where abundant unlabeled samples exist, our
unsupervised method outperforms its counterpart trained with supervised
learning.Comment: CVPR 2018 Camera-read
Optical Flow Requires Multiple Strategies (but only one network)
We show that the matching problem that underlies optical flow requires
multiple strategies, depending on the amount of image motion and other factors.
We then study the implications of this observation on training a deep neural
network for representing image patches in the context of descriptor based
optical flow. We propose a metric learning method, which selects suitable
negative samples based on the nature of the true match. This type of training
produces a network that displays multiple strategies depending on the input and
leads to state of the art results on the KITTI 2012 and KITTI 2015 optical flow
benchmarks
Unsupervised Deep Epipolar Flow for Stationary or Dynamic Scenes
Unsupervised deep learning for optical flow computation has achieved
promising results. Most existing deep-net based methods rely on image
brightness consistency and local smoothness constraint to train the networks.
Their performance degrades at regions where repetitive textures or occlusions
occur. In this paper, we propose Deep Epipolar Flow, an unsupervised optical
flow method which incorporates global geometric constraints into network
learning. In particular, we investigate multiple ways of enforcing the epipolar
constraint in flow estimation. To alleviate a "chicken-and-egg" type of problem
encountered in dynamic scenes where multiple motions may be present, we propose
a low-rank constraint as well as a union-of-subspaces constraint for training.
Experimental results on various benchmarking datasets show that our method
achieves competitive performance compared with supervised methods and
outperforms state-of-the-art unsupervised deep-learning methods.Comment: CVPR 201
- …