Detecting motion regions in presence of strong parallax from a moving camera by multi-view geometric constraints

Abstract

Abstract — We present a method for detecting motion regions in video sequences observed by a moving camera, in the presence of strong parallax due to static 3D structures. The proposed method classifies each image pixel into planar background, parallax or motion regions by sequentially applying 2D planar homographies, the epipolar constraint and a novel geometric constraint, called “structure consistency constraint”. The structure consistency constraint is the main contribution of this paper, and is derived from the relative camera poses in three consecutive frames and is implemented within the “Plane+Parallax ” framework. Unlike previous planar-parallax constraints proposed in the literature, the structure consistency constraint does not require the reference plane to be constant across multiple views. It directly measures the inconsistency between the projective structures from the same point under camera motion and reference plane change. The structure consistency constraint is capable of detecting moving objects followed by a moving camera in the same direction, a so called degenerate configuration where the epipolar constraint fails. We demonstrate the effectiveness and robustness of our method with experimental results on real-world video sequences. Index Terms — Motion detection, multiple view geometry, epipolar constraint, plane plus parallax I