84,913 research outputs found
MPI-Flow: Learning Realistic Optical Flow with Multiplane Images
The accuracy of learning-based optical flow estimation models heavily relies
on the realism of the training datasets. Current approaches for generating such
datasets either employ synthetic data or generate images with limited realism.
However, the domain gap of these data with real-world scenes constrains the
generalization of the trained model to real-world applications. To address this
issue, we investigate generating realistic optical flow datasets from
real-world images. Firstly, to generate highly realistic new images, we
construct a layered depth representation, known as multiplane images (MPI),
from single-view images. This allows us to generate novel view images that are
highly realistic. To generate optical flow maps that correspond accurately to
the new image, we calculate the optical flows of each plane using the camera
matrix and plane depths. We then project these layered optical flows into the
output optical flow map with volume rendering. Secondly, to ensure the realism
of motion, we present an independent object motion module that can separate the
camera and dynamic object motion in MPI. This module addresses the deficiency
in MPI-based single-view methods, where optical flow is generated only by
camera motion and does not account for any object movement. We additionally
devise a depth-aware inpainting module to merge new images with dynamic objects
and address unnatural motion occlusions. We show the superior performance of
our method through extensive experiments on real-world datasets. Moreover, our
approach achieves state-of-the-art performance in both unsupervised and
supervised training of learning-based models. The code will be made publicly
available at: \url{https://github.com/Sharpiless/MPI-Flow}.Comment: Accepted to ICCV202
A Variational Framework for Structure from Motion inOmnidirectional Image Sequences
We address the problem of depth and ego-motion estimation from omnidirectional images. We propose a correspondence-free structure-from-motion problem for sequences of images mapped on the 2-sphere. A novel graph-based variational framework is first proposed for depth estimation between pairs of images. The estimation is cast as a TV-L1 optimization problem that is solved by a fast graph-based algorithm. The ego-motion is then estimated directly from the depth information without explicit computation of the optical flow. Both problems are finally addressed together in an iterative algorithm that alternates between depth and ego-motion estimation for fast computation of 3D information from motion in image sequences. Experimental results demonstrate the effective performance of the proposed algorithm for 3D reconstruction from synthetic and natural omnidirectional image
- …