12,640 research outputs found
Devon: Deformable Volume Network for Learning Optical Flow
State-of-the-art neural network models estimate large displacement optical
flow in multi-resolution and use warping to propagate the estimation between
two resolutions. Despite their impressive results, it is known that there are
two problems with the approach. First, the multi-resolution estimation of
optical flow fails in situations where small objects move fast. Second, warping
creates artifacts when occlusion or dis-occlusion happens. In this paper, we
propose a new neural network module, Deformable Cost Volume, which alleviates
the two problems. Based on this module, we designed the Deformable Volume
Network (Devon) which can estimate multi-scale optical flow in a single high
resolution. Experiments show Devon is more suitable in handling small objects
moving fast and achieves comparable results to the state-of-the-art methods in
public benchmarks
Understanding Deformable Alignment in Video Super-Resolution
Deformable convolution, originally proposed for the adaptation to geometric
variations of objects, has recently shown compelling performance in aligning
multiple frames and is increasingly adopted for video super-resolution. Despite
its remarkable performance, its underlying mechanism for alignment remains
unclear. In this study, we carefully investigate the relation between
deformable alignment and the classic flow-based alignment. We show that
deformable convolution can be decomposed into a combination of spatial warping
and convolution. This decomposition reveals the commonality of deformable
alignment and flow-based alignment in formulation, but with a key difference in
their offset diversity. We further demonstrate through experiments that the
increased diversity in deformable alignment yields better-aligned features, and
hence significantly improves the quality of video super-resolution output.
Based on our observations, we propose an offset-fidelity loss that guides the
offset learning with optical flow. Experiments show that our loss successfully
avoids the overflow of offsets and alleviates the instability problem of
deformable alignment. Aside from the contributions to deformable alignment, our
formulation inspires a more flexible approach to introduce offset diversity to
flow-based alignment, improving its performance.Comment: Tech report, 15 pages, 19 figure
DeepMatching: Hierarchical Deformable Dense Matching
We introduce a novel matching algorithm, called DeepMatching, to compute
dense correspondences between images. DeepMatching relies on a hierarchical,
multi-layer, correlational architecture designed for matching images and was
inspired by deep convolutional approaches. The proposed matching algorithm can
handle non-rigid deformations and repetitive textures and efficiently
determines dense correspondences in the presence of significant changes between
images. We evaluate the performance of DeepMatching, in comparison with
state-of-the-art matching algorithms, on the Mikolajczyk (Mikolajczyk et al
2005), the MPI-Sintel (Butler et al 2012) and the Kitti (Geiger et al 2013)
datasets. DeepMatching outperforms the state-of-the-art algorithms and shows
excellent results in particular for repetitive textures.We also propose a
method for estimating optical flow, called DeepFlow, by integrating
DeepMatching in the large displacement optical flow (LDOF) approach of Brox and
Malik (2011). Compared to existing matching algorithms, additional robustness
to large displacements and complex motion is obtained thanks to our matching
approach. DeepFlow obtains competitive performance on public benchmarks for
optical flow estimation
Classifying Deformable and Non-deformable Video Objects
This paper presents a fully automated approach to classifying deformable and non-deformable moving objects in a video surveillance scene. We estimate an object's motion using Marzat optical-flow algorithm. We filter the motion vectors and attempt to find the transformation that represents the correct mapping between the two positions. The Fundamental transformation is estimated using the Normalized Eight-Point Algorithm. We introduce a new type of graph to set the thresholds between deformable and non-deformable motion. Furthermore, we use temporal consistency to classify deformable and non-deformable objects. For experiments, we used a varied corpus of real surveillance videos. Our proposed approach for motion classification achieved a precision rate of 92 percent
- …