43 research outputs found
3D Human Pose Estimation using Spatio-Temporal Networks with Explicit Occlusion Training
Estimating 3D poses from a monocular video is still a challenging task,
despite the significant progress that has been made in recent years. Generally,
the performance of existing methods drops when the target person is too
small/large, or the motion is too fast/slow relative to the scale and speed of
the training data. Moreover, to our knowledge, many of these methods are not
designed or trained under severe occlusion explicitly, making their performance
on handling occlusion compromised. Addressing these problems, we introduce a
spatio-temporal network for robust 3D human pose estimation. As humans in
videos may appear in different scales and have various motion speeds, we apply
multi-scale spatial features for 2D joints or keypoints prediction in each
individual frame, and multi-stride temporal convolutional net-works (TCNs) to
estimate 3D joints or keypoints. Furthermore, we design a spatio-temporal
discriminator based on body structures as well as limb motions to assess
whether the predicted pose forms a valid pose and a valid movement. During
training, we explicitly mask out some keypoints to simulate various occlusion
cases, from minor to severe occlusion, so that our network can learn better and
becomes robust to various degrees of occlusion. As there are limited 3D
ground-truth data, we further utilize 2D video data to inject a semi-supervised
learning capability to our network. Experiments on public datasets validate the
effectiveness of our method, and our ablation studies show the strengths of our
network\'s individual submodules.Comment: 8 pages, AAAI 202