Video Capture of Skin Motion using Calibrated Fabrics

International audienceWe introduce a novel approach to capture linear and non linear motions of the skin surface due to muscle bugglings and other complex sub-surface interactions. Our method uses standard camera and consists of a video motion capture of the desired body surface covered by a stretchy and calibrated cloth. We develop a non model-based tracking method using regularity and topology of dedicated fabrics. Our tracking and 3D reconstruction methods support occlusions : we detect occlusions of 3D surfaces using temporal and spatial coherences, and are able to restart surface tracking once the occlusions have ended. The obtained 3D mesh can easily be integrated and adapted in a sequence of an animated virtual human. We demonstrate our method with the skinning of non-linear biceps deformations

Video-based methodology for markerless human motion analysis

International audienceThis study presents a video-based experiment for the study of markerless human motion. Silhouettes are extracted from a multi-camera video system to reconstruct a 3D mesh for each frame using a reconstruction method based on visual hull. For comparison with traditional motion analysis results, we set up an experiment integrating video recordings from 8 video cameras and a marker-based motion capture system (Vicon™). Our preliminary data provided distances between the 3D trajectories from the Vicon system and the 3D mesh extracted from the video cameras. In the long term, the main ambition of this method is to provide measurement of skeleton motion for human motion analyses while eliminating markers

Video-based methodology for markerless human motion analysis

International audienceThis study presents a video-based experiment for the study of markerless human motion. Silhouettes are extracted from a multi-camera video system to reconstruct a 3D mesh for each frame using a reconstruction method based on visual hull. For comparison with traditional motion analysis results, we set up an experiment integrating video recordings from 8 video cameras and a marker-based motion capture system (Vicon™). Our preliminary data provided distances between the 3D trajectories from the Vicon system and the 3D mesh extracted from the video cameras. In the long term, the main ambition of this method is to provide measurement of skeleton motion for human motion analyses while eliminating markers

Principal Geodesic Dynamics

International audienceThis paper presents a new integration of a data-driven approach using dimension reduction and a physically-based simulation for real-time character animation. We exploit Lie group statistical analysis techniques (Principal Geodesic Analysis, PGA) to approximate the pose manifold of a motion capture sequence by a reduced set of pose geodesics. We integrate this kinematic parametrization into a physically-based animation approach of virtual characters, by using the PGA-reduced parametrization directly as generalized coordinates of a Lagrangian formulation of mechanics. In order to achieve real-time without sacrificing stability, we derive an explicit time integrator by approximating existing variational integrators. Finally, we test our approach in task-space motion control. By formulating both physical simulation and inverse kinematics time stepping schemes as two quadratic programs, we propose a features-based control algorithm that interpolates between the two metrics. This allows for an intuitive trade-off between realistic physical simulation and controllable kinematic manipulation

Wind projection basis for real-time animation of trees

International audienceThis scientific report presents a real-time method to animate complex scenes of thousands of trees under a user-controllable wind load. Firstly, modal analysis is applied to extract the main modes of deformation from the mechanical model of a 3D tree. The novelty of our contribution is to precompute a new basis of the modal stress of the tree under wind load. At runtime, this basis allows to replace the modal projection of the external forces by a direct mapping for any directional wind. We show that this approach can be efficiently implemented on a graphics hardware. This modal animation can be simulated at low computation cost even for large scenes containing thousands of trees

Hierarchical retargetting of 2D motion fields to the animation of 3D plant models

International audienceThe complexity of animating trees, shrubs and foliage is an impediment to the efficient and realistic depiction of natural environments. This paper presents an algorithm to extract, from a single video sequence, motion fields of real shrubs under the influence of wind, and to transfer this motion to the animation of complex, synthetic 3D plant models. The extracted motion is retargeted without requiring physical simulation. First, feature tracking is applied to the video footage, allowing the 2D position and velocity of automatically identified features to be clustered. A key contribution of the method is that the hierarchy obtained through statistical clustering can be used to synthesize a 2D hierarchical geometric structure of branches that terminates according to the cut-off threshold of a classification algorithm. This step extracts both the shape and the motion of a hierarchy of features groups that are identified as geometrical branches. The 2D hierarchy is then extended to three dimensions using the estimated spatial distribution of the features within each group. Another key contribution is that this 3D hierarchical structure can be efficiently used as a motion controller to animate any complex 3D model of similar but non-identical plants using a standard skinning algorithm. Thus, a single video source of a moving shrub becomes an input device for a large class of virtual shrubs. We illustrate the results on two examples of shrubs and one outdoor tree. Extensions to other outdoor plants are discussed

Multiple-View Threedimensional Paper Sheet Reconstruction

Smoothly bent paper-like surfaces are developable. They are however difficult to minimally parameterize since the number of meaningful parameters is intrinsically dependent on the actual deformation. Previous generative models are either incomplete, i.e. limited to subsets of developable surfaces, or depend on huge parameter sets. Our first contribution is a generative model governed by a quasi-minimal set of intuitive parameters, namely rules and angles. More precisely, a flat mesh is bent along guiding rules, while a number of extra rules controls the level of smoothness. The generated surface is guaranteed to be developable. The second contribution is an automatic multi-camera 3D reconstruction algorithm. First of all, the cameras and a sparse structure are reconstructed from the images using Structure-from-Motion method. A 2D parametrization of the reconstructed points is computed by dimensionality reduction. This parameterization is used to initialize the proposed model since it easily allows us to estimate the surface curvature. The initial model parameters are eventually tuned through model-based bundle-adjustment.Les surfaces de type papier, lorsqu’elles ne présentent pas de pli franc, sont mathématiquement décrites par des surfaces développables. Ces dernières sont difficiles à paramétrer de manière minimale car le nombre de degrés de liberté significatif dépend de la déformation. Les modèles existants sont incomplets ou dépendent de grands jeux de paramètres redondants. Notre première contribution est un modèle génératif contrôlé par un jeu quasi-minimal de paramètres intuitifs. Le principe est de plier une surface plane autour de règles de guidage. Un certain nombre de règles de lissage contrôle la régularité de la surface ainsi générée, qui par construction approche une surface développable. Notre deuxième contribution est un algorithme d’estimation du modèle proposé à partir de plusieurs images. Tout d’abord, les caméras et une structure 3D éparse de la surface de l’objet sont reconstruites. Une paramétrisation 2D de ces points est ensuite calculée par une méthode non-linéaire de réduction des dimensions. Cette paramétrisation est essentielle pour évaluer la courbure d’une surface passant par les points reconstruits, nécessaire à l’initialisation des paramètres du modèle. Enfin, un ajustement de faisceaux ajuste les paramètres du modèle afin de raffiner la surface en minimisant l’erreur de reprojection

FACSGen: A Tool to Synthesize Emotional Facial Expressions Through Systematic Manipulation of Facial Action Units

To investigate the perception of emotional facial expressions, researchers rely on shared sets of photos or videos, most often generated by actor portrayals. The drawback of such standardized material is a lack of flexibility and controllability, as it does not allow the systematic parametric manipulation of specific features of facial expressions on the one hand, and of more general properties of the facial identity (age, ethnicity, gender) on the other. To remedy this problem, we developed FACSGen: a novel tool that allows the creation of realistic synthetic 3D facial stimuli, both static and dynamic, based on the Facial Action Coding System. FACSGen provides researchers with total control over facial action units, and corresponding informational cues in 3D synthetic faces. We present four studies validating both the software and the general methodology of systematically generating controlled facial expression patterns for stimulus presentatio

Cage-based Motion Recovery using Manifold Learning

International audienceWe present a flexible model-based approach for the recovery of parameterized motion from a sequence of 3D meshes without temporal coherence. Unlike previous model-based approaches using skeletons, we embed the deformation of a reference mesh template within a low polygonal representation of the mesh, namely the cage, using Green Coordinates. The advantage is a less constrained model that more robustly adapts to noisy observations while still providing structured motion information, as required by several applications. The cage is parameterized with a set of 3D features dedicated to the description of human morphology. This allows to formalize a novel representation of 3D meshed and articulated characters, the Oriented Quads Rigging (OQR). To regularize the tracking, the OQR space is subsequently constrained to plausible poses using manifold learning. Results are shown for sequences of meshes, with and without temporal coherence, obtained from multiple view videos preprocessed by visual hull. Motion recovery applications are illustrated with a motion transfer encoding and the extraction of trajectories of anatomical joints. Validation is performed on the HumanEva II database