4,724 research outputs found
Proposal Flow: Semantic Correspondences from Object Proposals
Finding image correspondences remains a challenging problem in the presence
of intra-class variations and large changes in scene layout. Semantic flow
methods are designed to handle images depicting different instances of the same
object or scene category. We introduce a novel approach to semantic flow,
dubbed proposal flow, that establishes reliable correspondences using object
proposals. Unlike prevailing semantic flow approaches that operate on pixels or
regularly sampled local regions, proposal flow benefits from the
characteristics of modern object proposals, that exhibit high repeatability at
multiple scales, and can take advantage of both local and geometric consistency
constraints among proposals. We also show that the corresponding sparse
proposal flow can effectively be transformed into a conventional dense flow
field. We introduce two new challenging datasets that can be used to evaluate
both general semantic flow techniques and region-based approaches such as
proposal flow. We use these benchmarks to compare different matching
algorithms, object proposals, and region features within proposal flow, to the
state of the art in semantic flow. This comparison, along with experiments on
standard datasets, demonstrates that proposal flow significantly outperforms
existing semantic flow methods in various settings.Comment: arXiv admin note: text overlap with arXiv:1511.0506
End-to-End Learning of Video Super-Resolution with Motion Compensation
Learning approaches have shown great success in the task of super-resolving
an image given a low resolution input. Video super-resolution aims for
exploiting additionally the information from multiple images. Typically, the
images are related via optical flow and consecutive image warping. In this
paper, we provide an end-to-end video super-resolution network that, in
contrast to previous works, includes the estimation of optical flow in the
overall network architecture. We analyze the usage of optical flow for video
super-resolution and find that common off-the-shelf image warping does not
allow video super-resolution to benefit much from optical flow. We rather
propose an operation for motion compensation that performs warping from low to
high resolution directly. We show that with this network configuration, video
super-resolution can benefit from optical flow and we obtain state-of-the-art
results on the popular test sets. We also show that the processing of whole
images rather than independent patches is responsible for a large increase in
accuracy.Comment: Accepted to GCPR201
A Fusion Approach for Multi-Frame Optical Flow Estimation
To date, top-performing optical flow estimation methods only take pairs of
consecutive frames into account. While elegant and appealing, the idea of using
more than two frames has not yet produced state-of-the-art results. We present
a simple, yet effective fusion approach for multi-frame optical flow that
benefits from longer-term temporal cues. Our method first warps the optical
flow from previous frames to the current, thereby yielding multiple plausible
estimates. It then fuses the complementary information carried by these
estimates into a new optical flow field. At the time of writing, our method
ranks first among published results in the MPI Sintel and KITTI 2015
benchmarks. Our models will be available on https://github.com/NVlabs/PWC-Net.Comment: Work accepted at IEEE Winter Conference on Applications of Computer
Vision (WACV 2019
- …