33,357 research outputs found
Combining Stereo Disparity and Optical Flow for Basic Scene Flow
Scene flow is a description of real world motion in 3D that contains more
information than optical flow. Because of its complexity there exists no
applicable variant for real-time scene flow estimation in an automotive or
commercial vehicle context that is sufficiently robust and accurate. Therefore,
many applications estimate the 2D optical flow instead. In this paper, we
examine the combination of top-performing state-of-the-art optical flow and
stereo disparity algorithms in order to achieve a basic scene flow. On the
public KITTI Scene Flow Benchmark we demonstrate the reasonable accuracy of the
combination approach and show its speed in computation.Comment: Commercial Vehicle Technology Symposium (CVTS), 201
SceneFlowFields: Dense Interpolation of Sparse Scene Flow Correspondences
While most scene flow methods use either variational optimization or a strong
rigid motion assumption, we show for the first time that scene flow can also be
estimated by dense interpolation of sparse matches. To this end, we find sparse
matches across two stereo image pairs that are detected without any prior
regularization and perform dense interpolation preserving geometric and motion
boundaries by using edge information. A few iterations of variational energy
minimization are performed to refine our results, which are thoroughly
evaluated on the KITTI benchmark and additionally compared to state-of-the-art
on MPI Sintel. For application in an automotive context, we further show that
an optional ego-motion model helps to boost performance and blends smoothly
into our approach to produce a segmentation of the scene into static and
dynamic parts.Comment: IEEE Winter Conference on Applications of Computer Vision (WACV),
201
Lagrangian Motion Magnification with Double Sparse Optical Flow Decomposition
Motion magnification techniques aim at amplifying and hence revealing subtle
motion in videos. There are basically two main approaches to reach this goal,
namely via Eulerian or Lagrangian techniques. While the first one magnifies
motion implicitly by operating directly on image pixels, the Lagrangian
approach uses optical flow techniques to extract and amplify pixel
trajectories. Microexpressions are fast and spatially small facial expressions
that are difficult to detect. In this paper, we propose a novel approach for
local Lagrangian motion magnification of facial micromovements. Our
contribution is three-fold: first, we fine-tune the recurrent all-pairs field
transforms for optical flows (RAFT) deep learning approach for faces by adding
ground truth obtained from the variational dense inverse search (DIS) for
optical flow algorithm applied to the CASME II video set of faces. This enables
us to produce optical flows of facial videos in an efficient and sufficiently
accurate way. Second, since facial micromovements are both local in space and
time, we propose to approximate the optical flow field by sparse components
both in space and time leading to a double sparse decomposition. Third, we use
this decomposition to magnify micro-motions in specific areas of the face,
where we introduce a new forward warping strategy using a triangular splitting
of the image grid and barycentric interpolation of the RGB vectors at the
corners of the transformed triangles. We demonstrate the very good performance
of our approach by various examples
DCTM: Discrete-Continuous Transformation Matching for Semantic Flow
Techniques for dense semantic correspondence have provided limited ability to
deal with the geometric variations that commonly exist between semantically
similar images. While variations due to scale and rotation have been examined,
there lack practical solutions for more complex deformations such as affine
transformations because of the tremendous size of the associated solution
space. To address this problem, we present a discrete-continuous transformation
matching (DCTM) framework where dense affine transformation fields are inferred
through a discrete label optimization in which the labels are iteratively
updated via continuous regularization. In this way, our approach draws
solutions from the continuous space of affine transformations in a manner that
can be computed efficiently through constant-time edge-aware filtering and a
proposed affine-varying CNN-based descriptor. Experimental results show that
this model outperforms the state-of-the-art methods for dense semantic
correspondence on various benchmarks
- …