Expressive Body Capture: 3D Hands, Face, and Body from a Single Image

To facilitate the analysis of human actions, interactions and emotions, we compute a 3D model of human body pose, hand pose, and facial expression from a single monocular image. To achieve this, we use thousands of 3D scans to train a new, unified, 3D model of the human body, SMPL-X, that extends SMPL with fully articulated hands and an expressive face. Learning to regress the parameters of SMPL-X directly from images is challenging without paired images and 3D ground truth. Consequently, we follow the approach of SMPLify, which estimates 2D features and then optimizes model parameters to fit the features. We improve on SMPLify in several significant ways: (1) we detect 2D features corresponding to the face, hands, and feet and fit the full SMPL-X model to these; (2) we train a new neural network pose prior using a large MoCap dataset; (3) we define a new interpenetration penalty that is both fast and accurate; (4) we automatically detect gender and the appropriate body models (male, female, or neutral); (5) our PyTorch implementation achieves a speedup of more than 8x over Chumpy. We use the new method, SMPLify-X, to fit SMPL-X to both controlled images and images in the wild. We evaluate 3D accuracy on a new curated dataset comprising 100 images with pseudo ground-truth. This is a step towards automatic expressive human capture from monocular RGB data. The models, code, and data are available for research purposes at https://smpl-x.is.tue.mpg.de.Comment: To appear in CVPR 201

A Multivariate Surface-Based Analysis of the Putamen in Premature Newborns: Regional Differences within the Ventral Striatum

Many children born preterm exhibit frontal executive dysfunction, behavioral problems including attentional deficit/hyperactivity disorder and attention related learning disabilities. Anomalies in regional specificity of cortico-striato-thalamo-cortical circuits may underlie deficits in these disorders. Nonspecific volumetric deficits of striatal structures have been documented in these subjects, but little is known about surface deformation in these structures. For the first time, here we found regional surface morphological differences in the preterm neonatal ventral striatum. We performed regional group comparisons of the surface anatomy of the striatum (putamen and globus pallidus) between 17 preterm and 19 term-born neonates at term-equivalent age. We reconstructed striatal surfaces from manually segmented brain magnetic resonance images and analyzed them using our in-house conformal mapping program. All surfaces were registered to a template with a new surface fluid registration method. Vertex-based statistical comparisons between the two groups were performed via four methods: univariate and multivariate tensor-based morphometry, the commonly used medial axis distance, and a combination of the last two statistics. We found statistically significant differences in regional morphology between the two groups that are consistent across statistics, but more extensive for multivariate measures. Differences were localized to the ventral aspect of the striatum. In particular, we found abnormalities in the preterm anterior/inferior putamen, which is interconnected with the medial orbital/prefrontal cortex and the midline thalamic nuclei including the medial dorsal nucleus and pulvinar. These findings support the hypothesis that the ventral striatum is vulnerable, within the cortico-stiato-thalamo-cortical neural circuitry, which may underlie the risk for long-term development of frontal executive dysfunction, attention deficit hyperactivity disorder and attention-related learning disabilities in preterm neonates. © 2013 Shi et al

Fully Automatic Expression-Invariant Face Correspondence

We consider the problem of computing accurate point-to-point correspondences among a set of human face scans with varying expressions. Our fully automatic approach does not require any manually placed markers on the scan. Instead, the approach learns the locations of a set of landmarks present in a database and uses this knowledge to automatically predict the locations of these landmarks on a newly available scan. The predicted landmarks are then used to compute point-to-point correspondences between a template model and the newly available scan. To accurately fit the expression of the template to the expression of the scan, we use as template a blendshape model. Our algorithm was tested on a database of human faces of different ethnic groups with strongly varying expressions. Experimental results show that the obtained point-to-point correspondence is both highly accurate and consistent for most of the tested 3D face models

Appl Ergon

The objective of this study was to quantify head-and-face shape variations of U.S. civilian workers using modern methods of shape analysis. The purpose of this study was based on previously highlighted changes in U.S. civilian worker head-and-face shape over the last few decades - touting the need for new and better fitting respirators - as well as the study's usefulness in designing more effective personal protective equipment (PPE) - specifically in the field of respirator design. The raw scan three-dimensional (3D) data for 1169 subjects were parameterized using geometry processing techniques. This process allowed the individual scans to be put in correspondence with each other in such a way that statistical shape analysis could be performed on a dense set of 3D points. This process also cleaned up the original scan data such that the noise was reduced and holes were filled in. The next step, statistical analysis of the variability of the head-and-face shape in the 3D database, was conducted using Principal Component Analysis (PCA) techniques. Through these analyses, it was shown that the space of the head-and-face shape was spanned by a small number of basis vectors. Less than 50 components explained more than 90% of the variability. Furthermore, the main mode of variations could be visualized through animating the shape changes along the PCA axes with computer software in executable form for Windows XP. The results from this study in turn could feed back into respirator design to achieve safer, more efficient product style and sizing. Future study is needed to determine the overall utility of the point cloud-based approach for the quantification of facial morphology variation and its relationship to respirator performance.CC999999/Intramural CDC HHS/United States2016-02-09T00:00:00Z23399025PMC474703

Implicit Shape and Appearance Priors for Few-Shot Full Head Reconstruction

Recent advancements in learning techniques that employ coordinate-based neural representations have yielded remarkable results in multi-view 3D reconstruction tasks. However, these approaches often require a substantial number of input views (typically several tens) and computationally intensive optimization procedures to achieve their effectiveness. In this paper, we address these limitations specifically for the problem of few-shot full 3D head reconstruction. We accomplish this by incorporating a probabilistic shape and appearance prior into coordinate-based representations, enabling faster convergence and improved generalization when working with only a few input images (even as low as a single image). During testing, we leverage this prior to guide the fitting process of a signed distance function using a differentiable renderer. By incorporating the statistical prior alongside parallelizable ray tracing and dynamic caching strategies, we achieve an efficient and accurate approach to few-shot full 3D head reconstruction. Moreover, we extend the H3DS dataset, which now comprises 60 high-resolution 3D full head scans and their corresponding posed images and masks, which we use for evaluation purposes. By leveraging this dataset, we demonstrate the remarkable capabilities of our approach in achieving state-of-the-art results in geometry reconstruction while being an order of magnitude faster than previous approaches

Atlas-Guided Segmentation of Vervet Monkey Brain MRI

The vervet monkey is an important nonhuman primate model that allows the study of isolated environmental factors in a controlled environment. Analysis of monkey MRI often suffers from lower quality images compared with human MRI because clinical equipment is typically used to image the smaller monkey brain and higher spatial resolution is required. This, together with the anatomical differences of the monkey brains, complicates the use of neuroimage analysis pipelines tuned for human MRI analysis. In this paper we developed an open source image analysis framework based on the tools available within the 3D Slicer software to support a biological study that investigates the effect of chronic ethanol exposure on brain morphometry in a longitudinally followed population of male vervets. We first developed a computerized atlas of vervet monkey brain MRI, which was used to encode the typical appearance of the individual brain structures in MRI and their spatial distribution. The atlas was then used as a spatial prior during automatic segmentation to process two longitudinal scans per subject. Our evaluation confirms the consistency and reliability of the automatic segmentation. The comparison of atlas construction strategies reveals that the use of a population-specific atlas leads to improved accuracy of the segmentation for subcortical brain structures. The contribution of this work is twofold. First, we describe an image processing workflow specifically tuned towards the analysis of vervet MRI that consists solely of the open source software tools. Second, we develop a digital atlas of vervet monkey brain MRIs to enable similar studies that rely on the vervet model

Registration and statistical analysis of the tongue shape during speech production

This thesis analyzes the human tongue shape during speech production. First, a semi-supervised approach is derived for estimating the tongue shape from volumetric magnetic resonance imaging data of the human vocal tract. Results of this extraction are used to derive parametric tongue models. Next, a framework is presented for registering sparse motion capture data of the tongue by means of such a model. This method allows to generate full three-dimensional animations of the tongue. Finally, a multimodal and statistical text-to-speech system is developed that is able to synthesize audio and synchronized tongue motion from text.Diese Dissertation beschäftigt sich mit der Analyse der menschlichen Zungenform während der Sprachproduktion. Zunächst wird ein semi-überwachtes Verfahren vorgestellt, mit dessen Hilfe sich Zungenformen von volumetrischen Magnetresonanztomographie- Aufnahmen des menschlichen Vokaltrakts schätzen lassen. Die Ergebnisse dieses Extraktionsverfahrens werden genutzt, um ein parametrisches Zungenmodell zu konstruieren. Danach wird eine Methode hergeleitet, die ein solches Modell nutzt, um spärliche Bewegungsaufnahmen der Zunge zu registrieren. Dieser Ansatz erlaubt es, dreidimensionale Animationen der Zunge zu erstellen. Zuletzt wird ein multimodales und statistisches Text-to-Speech-System entwickelt, das in der Lage ist, Audio und die dazu synchrone Zungenbewegung zu synthetisieren.German Research Foundatio