We present a solution to egocentric 3D body pose estimation from monocular
images captured from downward looking fish-eye cameras installed on the rim of
a head mounted VR device. This unusual viewpoint leads to images with unique
visual appearance, with severe self-occlusions and perspective distortions that
result in drastic differences in resolution between lower and upper body. We
propose an encoder-decoder architecture with a novel multi-branch decoder
designed to account for the varying uncertainty in 2D predictions. The
quantitative evaluation, on synthetic and real-world datasets, shows that our
strategy leads to substantial improvements in accuracy over state of the art
egocentric approaches. To tackle the lack of labelled data we also introduced a
large photo-realistic synthetic dataset. xR-EgoPose offers high quality
renderings of people with diverse skintones, body shapes and clothing,
performing a range of actions. Our experiments show that the high variability
in our new synthetic training corpus leads to good generalization to real world
footage and to state of theart results on real world datasets with ground
truth. Moreover, an evaluation on the Human3.6M benchmark shows that the
performance of our method is on par with top performing approaches on the more
classic problem of 3D human pose from a third person viewpoint.Comment: 14 pages. arXiv admin note: substantial text overlap with
arXiv:1907.1004