2,493 research outputs found
ParticleSfM: Exploiting Dense Point Trajectories for Localizing Moving Cameras in the Wild
Estimating the pose of a moving camera from monocular video is a challenging
problem, especially due to the presence of moving objects in dynamic
environments, where the performance of existing camera pose estimation methods
are susceptible to pixels that are not geometrically consistent. To tackle this
challenge, we present a robust dense indirect structure-from-motion method for
videos that is based on dense correspondence initialized from pairwise optical
flow. Our key idea is to optimize long-range video correspondence as dense
point trajectories and use it to learn robust estimation of motion
segmentation. A novel neural network architecture is proposed for processing
irregular point trajectory data. Camera poses are then estimated and optimized
with global bundle adjustment over the portion of long-range point trajectories
that are classified as static. Experiments on MPI Sintel dataset show that our
system produces significantly more accurate camera trajectories compared to
existing state-of-the-art methods. In addition, our method is able to retain
reasonable accuracy of camera poses on fully static scenes, which consistently
outperforms strong state-of-the-art dense correspondence based methods with
end-to-end deep learning, demonstrating the potential of dense indirect methods
based on optical flow and point trajectories. As the point trajectory
representation is general, we further present results and comparisons on
in-the-wild monocular videos with complex motion of dynamic objects. Code is
available at https://github.com/bytedance/particle-sfm.Comment: ECCV 2022. Project page: http://b1ueber2y.me/projects/ParticleSfM
- …