A parametric deformable model to fit unstructured 3D data

International audienceIn many computer vision and image understanding problems, it is important to find a smooth surface that fits a set of given unstructured 3D data. Although approaches based on general deformable models give satisfactory results, in particular a local description of the surface, they involve large linear systems to solve when dealing with high resolution 3D images. The advantage of parametric deformable templates like superquadrics is their small number of parameters to describe a shape. However, the set of shapes described by superquadrics is too limited to approximate precisely complex surfaces. This is why hybrid models have been introduced to refine the initial approximation. This article introduces a deformable superquadric model based on a superquadric fit followed by a free-form deformation (FFD) to fit unstructured 3D points. At the expense of a reasonable number of additional parameters, free-form deformations provide a much closer fit and a volumetric deformation field. We first present the mathematical and algorithmic details of the method. Then, since we are mainly concerned with applications for medical images, we present a medical application consisting in the reconstruction of the left ventricle of the heart from a number of various 3D cardiac images. The extension of the method to track anatomical structures in spatio-temporal images (4D data) is presented in a companion article

Complex scene modeling and segmentation with deformable simplex meshes

In this thesis we present a system for 3D reconstruction and segmentation of complex real world scenes. The input to the system is an unstructured cloud of 3D points. The output is a 3D model for each object in the scene. The system starts with a model that encloses the input point cloud. A deformation process is applied to the initial model so it gets close to the point cloud in terms of distance, geometry and topology. Once the deformation stops the model is analyzed to check if more than one object is present in the point cloud. If necessary a segmentation process splits the model into several parts that correspond to each object in the scene. Using this segmented model the point cloud is also segmented. Each resulting sub-cloud is treated as a new input to the system. If, after the deformation process, the model is not segmented a refinement process improves the objective and subjective quality of the model by concentrating vertices around high curvature areas. The simplex mesh reconstruction algorithm was modified and extended to suit our application. A novel segmentation algorithm was designed to be applied on the simplex mesh. We test the system with synthetic and real data obtained from single objects, simple. and complex scenes. In the case of the synthetic data different levels of noise are added to examine the performance of the system. The results show that the systems performs well for either of the three cases and also in the presence of low levels of noise

NASA: Neural Articulated Shape Approximation

Efficient representation of articulated objects such as human bodies is an important problem in computer vision and graphics. To efficiently simulate deformation, existing approaches represent 3D objects using polygonal meshes and deform them using skinning techniques. This paper introduces neural articulated shape approximation (NASA), an alternative framework that enables efficient representation of articulated deformable objects using neural indicator functions that are conditioned on pose. Occupancy testing using NASA is straightforward, circumventing the complexity of meshes and the issue of water-tightness. We demonstrate the effectiveness of NASA for 3D tracking applications, and discuss other potential extensions.Comment: ECCV 202

Multi-set canonical correlation analysis for 3D abnormal gait behaviour recognition based on virtual sample generation

Small sample dataset and two-dimensional (2D) approach are challenges to vision-based abnormal gait behaviour recognition (AGBR). The lack of three-dimensional (3D) structure of the human body causes 2D based methods to be limited in abnormal gait virtual sample generation (VSG). In this paper, 3D AGBR based on VSG and multi-set canonical correlation analysis (3D-AGRBMCCA) is proposed. First, the unstructured point cloud data of gait are obtained by using a structured light sensor. A 3D parametric body model is then deformed to fit the point cloud data, both in shape and posture. The features of point cloud data are then converted to a high-level structured representation of the body. The parametric body model is used for VSG based on the estimated body pose and shape data. Symmetry virtual samples, pose-perturbation virtual samples and various body-shape virtual samples with multi-views are generated to extend the training samples. The spatial-temporal features of the abnormal gait behaviour from different views, body pose and shape parameters are then extracted by convolutional neural network based Long Short-Term Memory model network. These are projected onto a uniform pattern space using deep learning based multi-set canonical correlation analysis. Experiments on four publicly available datasets show the proposed system performs well under various conditions

Rendering Deformable Surface Reflectance Fields

Animation of photorealistic computer graphics models is an important goal for many applications. Image-based modeling has emerged as a promising approach to capture and visualize real-world objects. Animating image-based models, however, is still a largely unsolved problem. In this paper, we extend a popular image-based representation called surface reflectance field to animate and render deformable real-world objects under arbitrary illumination. Deforming the surface reflectance field is achieved by modifying the underlying impostor geometry. We augment the impostor by a local parameterization that allows the correct evaluation of acquired reflectance images, preserving the original light model on the deformed surface. We present a deferred shading scheme to handle the increased amount of data involved in shading the deformable surface reflectance field. We show animations of various objects that were acquired with 3D photography.Engineering and Applied Science

Libration driven multipolar instabilities

We consider rotating flows in non-axisymmetric enclosures that are driven by libration, i.e. by a small periodic modulation of the rotation rate. Thanks to its simplicity, this model is relevant to various contexts, from industrial containers (with small oscillations of the rotation rate) to fluid layers of terrestial planets (with length-of-day variations). Assuming a multipolar $n$-fold boundary deformation, we first obtain the two-dimensional basic flow. We then perform a short-wavelength local stability analysis of the basic flow, showing that an instability may occur in three dimensions. We christen it the Libration Driven Multipolar Instability (LDMI). The growth rates of the LDMI are computed by a Floquet analysis in a systematic way, and compared to analytical expressions obtained by perturbation methods. We then focus on the simplest geometry allowing the LDMI, a librating deformed cylinder. To take into account viscous and confinement effects, we perform a global stability analysis, which shows that the LDMI results from a parametric resonance of inertial modes. Performing numerical simulations of this librating cylinder, we confirm that the basic flow is indeed established and report the first numerical evidence of the LDMI. Numerical results, in excellent agreement with the stability results, are used to explore the non-linear regime of the instability (amplitude and viscous dissipation of the driven flow). We finally provide an example of LDMI in a deformed spherical container to show that the instability mechanism is generic. Our results show that the previously studied libration driven elliptical instability simply corresponds to the particular case $n=2$ of a wider class of instabilities. Summarizing, this work shows that any oscillating non-axisymmetric container in rotation may excite intermittent, space-filling LDMI flows, and this instability should thus be easy to observe experimentally