STAR: Sparse Trained Articulated Human Body Regressor

The SMPL body model is widely used for the estimation, synthesis, and analysis of 3D human pose and shape. While popular, we show that SMPL has several limitations and introduce STAR, which is quantitatively and qualitatively superior to SMPL. First, SMPL has a huge number of parameters resulting from its use of global blend shapes. These dense pose-corrective offsets relate every vertex on the mesh to all the joints in the kinematic tree, capturing spurious long-range correlations. To address this, we define per-joint pose correctives and learn the subset of mesh vertices that are influenced by each joint movement. This sparse formulation results in more realistic deformations and significantly reduces the number of model parameters to 20% of SMPL. When trained on the same data as SMPL, STAR generalizes better despite having many fewer parameters. Second, SMPL factors pose-dependent deformations from body shape while, in reality, people with different shapes deform differently. Consequently, we learn shape-dependent pose-corrective blend shapes that depend on both body pose and BMI. Third, we show that the shape space of SMPL is not rich enough to capture the variation in the human population. We address this by training STAR with an additional 10,000 scans of male and female subjects, and show that this results in better model generalization. STAR is compact, generalizes better to new bodies and is a drop-in replacement for SMPL. STAR is publicly available for research purposes at http://star.is.tue.mpg.de.Comment: ECCV 202

Automatic generation of dynamic skin deformation for animated characters

© 2018 by the authors. Since non-automatic rigging requires heavy human involvements, and various automatic rigging algorithms are less efficient in terms of computational efficiency, especially for current curve-based skin deformation methods, identifying the iso-parametric curves and creating the animation skeleton requires tedious and time-consuming manual work. Although several automatic rigging methods have been developed, but they do not aim at curve-based models. To tackle this issue, this paper proposes a new rigging algorithm for automatic generation of dynamic skin deformation to quickly identify iso-parametric curves and create an animation skeleton in a few milliseconds, which can be seamlessly used in curve-based skin deformation methods to make the rigging process fast enough for highly efficient computer animation applications

Dynamic skin deformation using finite difference solutions for character animation

We present a new skin deformation method to create dynamic skin deformations in this paper. The core elements of our approach are a dynamic deformation model, an efficient data-driven finite difference solution, and a curve-based representation of 3D models. We first reconstruct skin deformation models at different poses from the taken photos of a male human arm movement to achieve real deformed skin shapes. Then, we extract curves from these reconstructed skin deformation models. A new dynamic deformation model is proposed to describe physics of dynamic curve deformations, and its finite difference solution is developed to determine shape changes of the extracted curves. In order to improve visual realism of skin deformations, we employ data-driven methods and introduce skin shapes at the initial and final poses into our proposed dynamic deformation model. Experimental examples and comparisons made in this paper indicate that our proposed dynamic skin deformation technique can create realistic deformed skin shapes efficiently with a small data size

State of the Art in Skinning Techniques for Articulated Deformable Characters

Skinning is an indispensable component of the content creation pipeline for character animation in the context of feature films, video games, and in the special effects industry. Skinning techniques define the deformation of the character skin for every animation frame according to the current state of skeletal joints. In this state of the art report, we focus on the existing research in the areas of skeleton-based deformation, volume preserving techniques and physically based skinning methods. We also summarize the recent research in deformable and soft bodies simulations for articulated characters, and discuss various geometric and examples-based approaches

On some interactive mesh deformations

Techniques devoted to deform 3D models are an important research field in Computer Graphics. They can be used in differentstages: the modelling phase, the animation process and also during some special simulations. Additionally, some applications may require the manipulation of 3D models under certain restrictions to preserve the volume of the modified object. Hence, thepresent PhD Dissertation explores new algorithms to perform flexible, robust and efficient 3D deformations. Apart from this, it also researches on a new methodology to restrict these deformations so that the volume of the manipulated model remains constant. Some of the most used methods to achieve smooth deformations are those included in the Cage-Based Deformation paradigm. Cage-based deformations enclose the model to be deformed in a coarse polyhedron, the cage. Then, they usually rely on Generalized Barycentric Coordinates to relate the model with the vertices, and other geometric elements, of this cage, which are the control points or the deformation handles. Finally, every time that one of these handles is dragged, the model is deformed accordingly. Although this paradigm is simple, elegant and performs efficient deformations, some cage-free space deformation techniques have recently appeared. They increase the flexibility of the deformation handles, which do not need to be connected, and define powerful tools that make the deformation process more versatile and intuitive. In this context, the Dissertation introduces new Generalized Barycentric Coordinate systems specially designed to be used in a cage-free environment. Any user who wants to use the presented schemes only needs to locate a set of control points in the vicinity of the model that he or she wants to deform. These handles can be placed wherever he or she considers mode suitable and the only requirement is that the model has to be enclosed in their convex hull. Up to now, there are few techniques to produce volume-preserving space deformations. However, in recent years there has been a growing interest in performing constrained deformations due to their more realistic and physically plausible results. Our contribution to this research line consists in a deformation framework that preserves the volume of the 3D models by means of its gradient and a control surface to restrict the movement of the handles. Moreover, the proposed methodology is not restricted to the cage-based schemes, but it can also be used in a cage-free environment. Finally, our research can be specially useful for spatial deformations of biological and medical models. This kind of models represent real organs and tissues, which are often soft and lack an internal rigid structure. In addition, they are elastic and incompressible. Any application designed to deal with this group of models and to train or assist doctors must be flexible, robust, efficient and user-friendly. The combination of the proposed cage-free systems with the presented volume-preserving deformation framework satisfiesLes deformacions de models 3D s'utilitzen en diverses etapes de la generació de continguts digitals: durant la fase de modelatge, durant el procés d'animació i en alguns tipus de simulacions. A més a més, hi ha aplicacions que necessiten que la manipulació dels models 3D es faci tenint en compte certes restriccions que permeten la conservació del volum de l'objecte modificat. Tot plegat fa que les tècniques de deformació 3D siguin un camp d'estudi molt important dins del món dels Gràfics. Per aquesta raó, aquesta Tesi Doctoral estudia nous algorismes que permetin realitzar deformacions 3D de manera flexible, robusta i eficient i que, a més a més, permetin conservar el volum dels objectes modificats. Un dels paradigmes més utilitzats per tal de realitzar deformacions suaus és el conegut amb el nom de Deformacions Basades en un Poliedre Englobant. Aquesta família de mètodes embolcalla el model que es vol deformar, normalment representat com una malla de triangles, dins d'un poliedre simple, amb poques cares. Un cop fet això, estableix un sistema de Coordenades Baricèntriques Generalitzades per tal de definir els vèrtexs del model a partir dels vèrtexs del poliedre englobant, els quals s'anomenen punts de control o controls de la deformació. D'aquesta manera, cada cop que s'arrossega o es modifica un d'aquests punts de control, el model que es troba dins del poliedre englobant es deforma segons el sistema de coordenades que s'ha definit. Tot i que aquest paradigma és simple, elegant i eficient, des de fa ja uns anys han començat a aparèixer noves tècniques que no necessiten el poliedre englobant per tal de realitzar la deformació. El seu principal objectiu és augmentar la flexibilitat dels controls de la deformació i definir eines que facin que el procés de deformació sigui més versàtil i intuïtiu. Tenint en compte aquest factor, aquesta Tesi també estudia sistemes de Coordenades Baricèntriques Generalitzades dissenyats per realitzar deformacions sense la necessitat de definir el poliedre englobant. D'aquesta manera, qualsevol usuari que vulgui utilitzar els mètodes que es presenten en aquesta Dissertació només s'ha d'encarregar de definir un conjunt de punts de control al voltant del model que vol deformar, podent-los posar allí on consideri més oportú segons la deformació que vulgui obtenir. L'únic requeriment necessari és que el model ha de quedar dins de l'envolupant convexa d'aquests punts de control. Actualment existeixen pocs mètodes que realitzin deformacions 3D amb preservació del volum. No obstant això, d'un temps ençà ha augmentat l'interès per realitzar deformacions subjectes a certes restriccions que fan que el resultat sigui més realista i físicament versemblant. La contribució d'aquesta Tesi dins d'aquesta línia de recerca consisteix en un sistema de deformació que preserva el volum dels objectes 3D gràcies a còmput del seu gradient i a una superfície de control que restringeix el moviment dels punts de control. Aquest mètode es pot aplicar tant als sistemes de deformació que necessiten un poliedre englobant com als que no el necessiten. Finalment, i ja per acabar, la recerca realitzada pot ser especialment útil per tal de realitzar deformacions de models mèdics i biològics. Aquests tipus de models poden representar òrgans i teixits reals, els quals, normalment, són tous, mancats d'una estructura rígida interna, elàstics i incompressibles. Qualsevol aplicació dissenyada per treballar amb aquest tipus de models i per entrenar i donar assistència a usuaris mèdics hauria de ser flexible, robusta, eficient i fàcil d'utilitzar. La combinació dels mètodes de deformació proposats conjuntament amb el sistema de preservació de volum satisfà totes aquestes condicions. Per aquesta raó es creu que les contribucions realitzades poden esdevenir eines importants per produir deformacions mèdiques.Postprint (published version

Skin deformation and animation of character models based on static and dynamic ordinary differential equations.

Animated characters play an important role in the field of computer animation, simulation and games. The basic criterion of good character animation is that the animated characters should appear realistic. This can be achieve through proper skin deformations for characters. Although various skin deformation approaches (Joint-based, Example-based, Physics-based, Curve-based and PDE-based) have been developed, the problem of generating realistic skin deformations efficiently with a small data set is a big challenge. In order to address the limitations of skin deformation, this thesis presents a workflow consisting of three main steps. First, the research has developed a new statistical method to determine the positions of joints based on available X-ray images. Second, an effective method for transferring the deformations of the curves to the polygonal model with high accuracy has been developed. Lastly, the research has produced a simple and efficient method to animate skin deformations by introducing a curved-based surface manipulation method combined with physics and data-driven approaches. The novelty of this method depends on a new model of dynamic deformations and an efficient finite difference solution of the model. The application examples indicate that the curve-based dynamic method developed in this thesis can achieve good realism and high computational efficiency with small data sets in the creation of skin deformations

Human Shape Estimation using Statistical Body Models

Human body estimation methods transform real-world observations into predictions about human body state. These estimation methods benefit a variety of health, entertainment, clothing, and ergonomics applications. State may include pose, overall body shape, and appearance. Body state estimation is underconstrained by observations; ambiguity presents itself both in the form of missing data within observations, and also in the form of unknown correspondences between observations. We address this challenge with the use of a statistical body model: a data-driven virtual human. This helps resolve ambiguity in two ways. First, it fills in missing data, meaning that incomplete observations still result in complete shape estimates. Second, the model provides a statistically-motivated penalty for unlikely states, which enables more plausible body shape estimates. Body state inference requires more than a body model; we therefore build obser- vation models whose output is compared with real observations. In this thesis, body state is estimated from three types of observations: 3D motion capture markers, depth and color images, and high-resolution 3D scans. In each case, a forward process is proposed which simulates observations. By comparing observations to the results of the forward process, state can be adjusted to minimize the difference between simulated and observed data. We use gradient-based methods because they are critical to the precise estimation of state with a large number of parameters. The contributions of this work include three parts. First, we propose a method for the estimation of body shape, nonrigid deformation, and pose from 3D markers. Second, we present a concise approach to differentiating through the rendering process, with application to body shape estimation. And finally, we present a statistical body model trained from human body scans, with state-of-the-art fidelity, good runtime performance, and compatibility with existing animation packages