We present PyMAF-X, a regression-based approach to recovering a full-body
parametric model from a single image. This task is very challenging since minor
parametric deviation may lead to noticeable misalignment between the estimated
mesh and the input image. Moreover, when integrating part-specific estimations
to the full-body model, existing solutions tend to either degrade the alignment
or produce unnatural wrist poses. To address these issues, we propose a
Pyramidal Mesh Alignment Feedback (PyMAF) loop in our regression network for
well-aligned human mesh recovery and extend it as PyMAF-X for the recovery of
expressive full-body models. The core idea of PyMAF is to leverage a feature
pyramid and rectify the predicted parameters explicitly based on the mesh-image
alignment status. Specifically, given the currently predicted parameters,
mesh-aligned evidence will be extracted from finer-resolution features
accordingly and fed back for parameter rectification. To enhance the alignment
perception, an auxiliary dense supervision is employed to provide mesh-image
correspondence guidance while spatial alignment attention is introduced to
enable the awareness of the global contexts for our network. When extending
PyMAF for full-body mesh recovery, an adaptive integration strategy is proposed
in PyMAF-X to produce natural wrist poses while maintaining the well-aligned
performance of the part-specific estimations. The efficacy of our approach is
validated on several benchmark datasets for body-only and full-body mesh
recovery, where PyMAF and PyMAF-X effectively improve the mesh-image alignment
and achieve new state-of-the-art results. The project page with code and video
results can be found at https://www.liuyebin.com/pymaf-x.Comment: An eXpressive extension of PyMAF [arXiv:2103.16507], Supporting
SMPL-X, Project page: https://www.liuyebin.com/pymaf-