Multi-person motion capture can be challenging due to ambiguities caused by
severe occlusion, fast body movement, and complex interactions. Existing
frameworks build on 2D pose estimations and triangulate to 3D coordinates via
reasoning the appearance, trajectory, and geometric consistencies among
multi-camera observations. However, 2D joint detection is usually incomplete
and with wrong identity assignments due to limited observation angle, which
leads to noisy 3D triangulation results. To overcome this issue, we propose to
explore the short-range autoregressive characteristics of skeletal motion using
transformer. First, we propose an adaptive, identity-aware triangulation module
to reconstruct 3D joints and identify the missing joints for each identity. To
generate complete 3D skeletal motion, we then propose a Dual-Masked
Auto-Encoder (D-MAE) which encodes the joint status with both
skeletal-structural and temporal position encoding for trajectory completion.
D-MAE's flexible masking and encoding mechanism enable arbitrary skeleton
definitions to be conveniently deployed under the same framework. In order to
demonstrate the proposed model's capability in dealing with severe data loss
scenarios, we contribute a high-accuracy and challenging motion capture dataset
of multi-person interactions with severe occlusion. Evaluations on both
benchmark and our new dataset demonstrate the efficiency of our proposed model,
as well as its advantage against the other state-of-the-art methods