Tracking Based Motion Segmentation under Relaxed Statistical Assumptions

Many Computer Vision algorithms employ the sum of pixel-wise squared differences between two patches as a statistical measure of similarity. This silently assumes that the noise in every pixel is independent. We present a method that involves a much more general noise model with relaxed independence assumptions but without significant increase in the computational requirements. We apply this technique to the problem of motion segmentation that uses tracking to estimate the motion of each region and then we employ our statistic to classify every pixel as part of a segment or the background. We tested several versions of the algorithm on a variety of image sequences (indoor and outdoor, real and synthetic, constant and varying lighting, stationary and moving camera, one of them with known ground truth) with very good results