3 research outputs found
Robust motion segmentation with subspace constraints
Motion segmentation is an important task in computer vision with
many applications such as dynamic scene understanding and
multi-body structure from motion. When the point correspondences
across frames are given, motion segmentation can be addressed as
a subspace clustering problem under an affine camera model. In
the first two parts of this thesis, we target the general
subspace clustering problem and propose two novel methods, namely
Efficient Dense Subspace Clustering (EDSC) and the Robust Shape
Interaction Matrix (RSIM) method.
Instead of following the standard compressive sensing approach,
in EDSC we formulate subspace clustering as a Frobenius norm
minimization problem, which inherently yields denser connections
between data points. While in the noise-free case we rely on the
self-expressiveness of the observations, in the presence of noise
we recover a clean dictionary to represent the data. Our
formulation lets us solve the subspace clustering problem
efficiently. More specifically, for outlier-free observations,
the solution can be obtained in closed-form, and in the presence
of outliers, we solve the problem by performing a series of
linear operations. Furthermore, we show that our Frobenius norm
formulation shares the same solution as the popular nuclear norm
minimization approach when the data is free of any noise.
In RSIM, we revisit the Shape Interaction Matrix (SIM) method,
one of the earliest approaches for motion segmentation (or
subspace clustering), and reveal its connections to several
recent subspace clustering methods. We derive a simple, yet
effective algorithm to robustify the SIM method and make it
applicable to real-world scenarios where the data is corrupted by
noise. We validate the proposed method by intuitive examples and
justify it with the matrix perturbation theory. Moreover, we show
that RSIM can be extended to handle missing data with a
Grassmannian gradient descent method.
The above subspace clustering methods work well for motion
segmentation, yet they require that point trajectories across
frames are known {\it a priori}. However, finding point
correspondences is in itself a challenging task. Existing
approaches tackle the correspondence estimation and motion
segmentation problems separately. In the third part of this
thesis, given a set of feature points detected in each frame of
the sequence, we develop an approach which simultaneously
performs motion segmentation and finds point correspondences
across the frames. We formulate this problem in terms of Partial
Permutation Matrices (PPMs) and aim to match feature descriptors
while simultaneously encouraging point trajectories to satisfy
subspace constraints. This lets us handle outliers in both point
locations and feature appearance. The resulting optimization
problem is solved via the Alternating Direction Method of
Multipliers (ADMM), where each subproblem has an efficient
solution. In particular, we show that most of the subproblems can
be solved in closed-form, and one binary assignment subproblem
can be solved by the Hungarian algorithm.
Obtaining reliable feature tracks in a frame-by-frame manner is
desirable in applications such as online motion segmentation. In
the final part of the thesis, we introduce a novel multi-body
feature tracker that exploits a multi-body rigidity assumption to
improve tracking robustness under a general perspective camera
model. A conventional approach to addressing this problem would
consist of alternating between solving two subtasks: motion
segmentation and feature tracking under rigidity constraints for
each segment. This approach, however, requires knowing the number
of motions, as well as assigning points to motion groups, which
is typically sensitive to motion estimates. By contrast, we
introduce a segmentation-free solution to multi-body feature
tracking that bypasses the motion assignment step and reduces to
solving a series of subproblems with closed-form solutions.
In summary, in this thesis, we exploit the powerful subspace
constraints and develop robust motion segmentation methods in
different challenging scenarios where the trajectories are either
given as input, or unknown beforehand. We also present a general
robust multi-body feature tracker which can be used as the first
step of motion segmentation to get reliable trajectories