Simultaneous Correspondences Estimation and Non-Rigid Structure Reconstruction

Given multi-view correspondences, it has been shown that 3D non-rigid structure can be recovered through factorization based techniques. However, establishing reliable correspondences across multi-view images of non-rigid structure is not an easy task. Existing methods solve multi-view correspondences and 3D non-rigid structure in sequel, which cannot exploit the crossover constraints in each sub-problem (i.e., constraints in non-rigid structure has not been enforced in establishing multi-view correspondences and verse vise). In this paper, we present a unified framework to simultaneously solve for multi-view correspondences and non-rigid structure. We formulate the problem by using the Partial Permutation Matrices (PPMs) and aim at establishing multi-view correspondences while simultaneously enforcing the low-rank constraint in non-rigid structure deformation. Additionally, our method can handle outliers and missing data elegantly under the same framework. We solve the simultaneous non-rigid structure and correspondences recovery problem via the Alternating Direction Method of Multipliers (ADMM). Experimental results on both synthetic and real images show that the proposed method achieves state-of-the-art performance on both sparse and dense non-rigid reconstruction problems.This work was supported in part by Australian Research Council (ARC) grants (DE140100180, DP120103896, LP100100588, and CE140100016) and Natural Science Foundation of China (61420106007)