1 research outputs found
Temporally Robust Global Motion Compensation by Keypoint-based Congealing
Global motion compensation (GMC) removes the impact of camera motion and
creates a video in which the background appears static over the progression of
time. Various vision problems, such as human activity recognition, background
reconstruction, and multi-object tracking can benefit from GMC. Existing GMC
algorithms rely on sequentially processing consecutive frames, by estimating
the transformation mapping the two frames, and obtaining a composite
transformation to a global motion compensated coordinate. Sequential GMC
suffers from temporal drift of frames from the accurate global coordinate, due
to either error accumulation or sporadic failures of motion estimation at a few
frames. We propose a temporally robust global motion compensation (TRGMC)
algorithm which performs accurate and stable GMC, despite complicated and
long-term camera motion. TRGMC densely connects pairs of frames, by matching
local keypoints of each frame. A joint alignment of these frames is formulated
as a novel keypoint-based congealing problem, where the transformation of each
frame is updated iteratively, such that the spatial coordinates for the start
and end points of matched keypoints are identical. Experimental results
demonstrate that TRGMC has superior performance in a wide range of scenarios.Comment: 14 Page