Physics-Based Modeling of Nonrigid Objects for Vision and Graphics (Dissertation)

This thesis develops a physics-based framework for 3D shape and nonrigid motion modeling for computer vision and computer graphics. In computer vision it addresses the problems of complex 3D shape representation, shape reconstruction, quantitative model extraction from biomedical data for analysis and visualization, shape estimation, and motion tracking. In computer graphics it demonstrates the generative power of our framework to synthesize constrained shapes, nonrigid object motions and object interactions for the purposes of computer animation. Our framework is based on the use of a new class of dynamically deformable primitives which allow the combination of global and local deformations. It incorporates physical constraints to compose articulated models from deformable primitives and provides force-based techniques for fitting such models to sparse, noise-corrupted 2D and 3D visual data. The framework leads to shape and nonrigid motion estimators that exploit dynamically deformable models to track moving 3D objects from time-varying observations. We develop models with global deformation parameters which represent the salient shape features of natural parts, and local deformation parameters which capture shape details. In the context of computer graphics, these models represent the physics-based marriage of the parameterized and free-form modeling paradigms. An important benefit of their global/local descriptive power in the context of computer vision is that it can potentially satisfy the often conflicting requirements of shape reconstruction and shape recognition. The Lagrange equations of motion that govern our models, augmented by constraints, make them responsive to externally applied forces derived from input data or applied by the user. This system of differential equations is discretized using finite element methods and simulated through time using standard numerical techniques. We employ these equations to formulate a shape and nonrigid motion estimator. The estimator is a continuous extended Kalman filter that recursively transforms the discrepancy between the sensory data and the estimated model state into generalized forces. These adjust the translational, rotational, and deformational degrees of freedom such that the model evolves in a consistent fashion with the noisy data. We demonstrate the interactive time performance of our techniques in a series of experiments in computer vision, graphics, and visualization

Physics-Based Modeling, Analysis and Animation

The idea of using physics-based models has received considerable interest in computer graphics and computer vision research the last ten years. The interest arises from the fact that simple geometric primitives cannot accurately represent natural objects. In computer graphics physics-based models are used to generate and visualize constrained shapes, motions of rigid and nonrigid objects and object interactions with the environment for the purposes of animation. On the other hand, in computer vision, the method applies to complex 3-D shape representation, shape reconstruction and motion estimation. In this paper we review two models that have been used in computer graphics and two models that apply to both areas. In the area of computer graphics, Miller [48] uses a mass-spring model to animate three forms of locomotion of snakes and worms. To overcome the problem of the multitude of degrees of freedom associated with the mass-spring lattices, Witkin and Welch [87] present a geometric method to model global deformations. To achieve the same result Pentland and Horowitz in [54] delineate the object motion into rigid and nonrigid deformation modes. To overcome problems of these two last approaches, Metaxas and Terzopoulos in [45] successfully combine local deformations with global ones. Modeling based on physical principles is a potent technique for computer graphics and computer vision. It is a rich and fruitful area for research in terms of both theory and applications. It is important, though, to develop concepts, methodologies, and techniques which will be widely applicable to many types of applications

Articular human joint modelling

Copyright @ Cambridge University Press 2009.The work reported in this paper encapsulates the theories and algorithms developed to drive the core analysis modules of the software which has been developed to model a musculoskeletal structure of anatomic joints. Due to local bone surface and contact geometry based joint kinematics, newly developed algorithms make the proposed modeller different from currently available modellers. There are many modellers that are capable of modelling gross human body motion. Nevertheless, none of the available modellers offer complete elements of joint modelling. It appears that joint modelling is an extension of their core analysis capability, which, in every case, appears to be musculoskeletal motion dynamics. It is felt that an analysis framework that is focused on human joints would have significant benefit and potential to be used in many orthopaedic applications. The local mobility of joints has a significant influence in human motion analysis, in understanding of joint loading, tissue behaviour and contact forces. However, in order to develop a bone surface based joint modeller, there are a number of major problems, from tissue idealizations to surface geometry discretization and non-linear motion analysis. This paper presents the following: (a) The physical deformation of biological tissues as linear or non-linear viscoelastic deformation, based on spring-dashpot elements. (b) The linear dynamic multibody modelling, where the linear formulation is established for small motions and is particularly useful for calculating the equilibrium position of the joint. This model can also be used for finding small motion behaviour or loading under static conditions. It also has the potential of quantifying the joint laxity. (c) The non-linear dynamic multibody modelling, where a non-matrix and algorithmic formulation is presented. The approach allows handling complex material and geometrical nonlinearity easily. (d) Shortest path algorithms for calculating soft tissue line of action geometries. The developed algorithms are based on calculating minimum ‘surface mass’ and ‘surface covariance’. An improved version of the ‘surface covariance’ algorithm is described as ‘residual covariance’. The resulting path is used to establish the direction of forces and moments acting on joints. This information is needed for linear or non-linear treatment of the joint motion. (e) The final contribution of the paper is the treatment of the collision. In the virtual world, the difficulty in analysing bodies in motion arises due to body interpenetrations. The collision algorithm proposed in the paper involves finding the shortest projected ray from one body to the other. The projection of the body is determined by the resultant forces acting on it due to soft tissue connections under tension. This enables the calculation of collision condition of non-convex objects accurately. After the initial collision detection, the analysis involves attaching special springs (stiffness only normal to the surfaces) at the ‘potentially colliding points’ and motion of bodies is recalculated. The collision algorithm incorporates the rotation as well as translation. The algorithm continues until the joint equilibrium is achieved. Finally, the results obtained based on the software are compared with experimental results obtained using cadaveric joints

Variational Bonded Discrete Element Method with Manifold Optimization

This paper proposes a novel approach that combines variational integration with the bonded discrete element method (BDEM) to achieve faster and more accurate fracture simulations. The approach leverages the efficiency of implicit integration and the accuracy of BDEM in modeling fracture phenomena. We introduce a variational integrator and a manifold optimization approach utilizing a nullspace operator to speed up the solving of quaternion-constrained systems. Additionally, the paper presents an element packing and surface reconstruction method specifically designed for bonded discrete element methods. Results from the experiments prove that the proposed method offers 2.8 to 12 times faster state-of-the-art methods

Computationally efficient deformable 3D object tracking with a monocular RGB camera

182 p.Monocular RGB cameras are present in most scopes and devices, including embedded environments like robots, cars and home automation. Most of these environments have in common a significant presence of human operators with whom the system has to interact. This context provides the motivation to use the captured monocular images to improve the understanding of the operator and the surrounding scene for more accurate results and applications.However, monocular images do not have depth information, which is a crucial element in understanding the 3D scene correctly. Estimating the three-dimensional information of an object in the scene using a single two-dimensional image is already a challenge. The challenge grows if the object is deformable (e.g., a human body or a human face) and there is a need to track its movements and interactions in the scene.Several methods attempt to solve this task, including modern regression methods based on Deep NeuralNetworks. However, despite the great results, most are computationally demanding and therefore unsuitable for several environments. Computational efficiency is a critical feature for computationally constrained setups like embedded or onboard systems present in robotics and automotive applications, among others.This study proposes computationally efficient methodologies to reconstruct and track three-dimensional deformable objects, such as human faces and human bodies, using a single monocular RGB camera. To model the deformability of faces and bodies, it considers two types of deformations: non-rigid deformations for face tracking, and rigid multi-body deformations for body pose tracking. Furthermore, it studies their performance on computationally restricted devices like smartphones and onboard systems used in the automotive industry. The information extracted from such devices gives valuable insight into human behaviour a crucial element in improving human-machine interaction.We tested the proposed approaches in different challenging application fields like onboard driver monitoring systems, human behaviour analysis from monocular videos, and human face tracking on embedded devices

General Dynamic Surface Reconstruction: Application to the 3D Segmentation of the Left Ventricle

Aquesta tesi descriu la nostra contribució a la reconstrucció tridimensional de les superfícies interna i externa del ventricle esquerre humà. La reconstrucció és un primer procés dins d'una aplicació global de Realitat Virtual dissenyada com una important eina de diagnòstic per a hospitals. L'aplicació parteix de la reconstrucció de les superfícies i proveeix a l'expert de manipulació interactiva del model en temps real, a més de càlculs de volums i de altres paràmetres d'interès. El procés de recuperació de les superfícies es caracteritza per la seva velocitat de convergència, la suavitat a les malles finals i la precisió respecte de les dades recuperades. Donat que el diagnòstic de patologies cardíaques requereix d'experiència, temps i molt coneixement professional, la simulació és un procés clau que millora la eficiència.Els nostres algorismes i implementacions han estat aplicats a dades sintètiques i reals amb diferències relatives a la quantitat de dades inexistents, casuístiques presents a casos patològics i anormals. Els conjunts de dades inclouen adquisicions d'instants concrets i de cicles cardíacs complets. La bondat del sistema de reconstrucció ha estat avaluada mitjançant paràmetres mèdics per a poder comparar els nostres resultats finals amb aquells derivats a partir de programari típic utilitzat pels professionals de la medicina.A més de l'aplicació directa al diagnòstic mèdic, la nostra metodologia permet reconstruccions de tipus genèric en el camp dels Gràfics 3D per ordinador. Les nostres reconstruccions permeten generar models tridimensionals amb un baix cost en quant a la interacció manual necessària i a la càrrega computacional associada. Altrament, el nostre mètode pot entendre's com un robust algorisme de triangularització que construeix superfícies partint de núvols de punts que poden obtenir-se d'escàners làser o sensors magnètics, per exemple.Esta tesis describe nuestra contribución a la reconstrucción tridimensional de las superficies interna y externa del ventrículo izquierdo humano. La reconstrucción es un primer proceso que forma parte de una aplicación global de Realidad Virtual diseñada como una importante herramienta de diagnóstico para hospitales. La aplicación parte de la reconstrucción de las superficies y provee al experto de manipulación interactiva del modelo en tiempo real, además de cálculos de volúmenes y de otros parámetros de interés. El proceso de recuperación de las superficies se caracteriza por su velocidad de convergencia, la suavidad en las mallas finales y la precisión respecto de los datos recuperados. Dado que el diagnóstico de patologías cardíacas requiere experiencia, tiempo y mucho conocimiento profesional, la simulación es un proceso clave que mejora la eficiencia.Nuestros algoritmos e implementaciones han sido aplicados a datos sintéticos y reales con diferencias en cuanto a la cantidad de datos inexistentes, casuística presente en casos patológicos y anormales. Los conjuntos de datos incluyen adquisiciones de instantes concretos y de ciclos cardíacos completos. La bondad del sistema de reconstrucción ha sido evaluada mediante parámetros médicos para poder comparar nuestros resultados finales con aquellos derivados a partir de programario típico utilizado por los profesionales de la medicina.Además de la aplicación directa al diagnóstico médico, nuestra metodología permite reconstrucciones de tipo genérico en el campo de los Gráficos 3D por ordenador. Nuestras reconstrucciones permiten generar modelos tridimensionales con un bajo coste en cuanto a la interacción manual necesaria y a la carga computacional asociada. Por otra parte, nuestro método puede entenderse como un robusto algoritmo de triangularización que construye superficies a partir de nubes de puntos que pueden obtenerse a partir de escáneres láser o sensores magnéticos, por ejemplo.This thesis describes a contribution to the three-dimensional reconstruction of the internal and external surfaces of the human's left ventricle. The reconstruction is a first process fitting in a complete VR application that will serve as an important diagnosis tool for hospitals. Beginning with the surfaces reconstruction, the application will provide volume and interactive real-time manipulation to the model. We focus on speed, precision and smoothness for the final surfaces. As long as heart diseases diagnosis requires experience, time and professional knowledge, simulation is a key-process that enlarges efficiency.The algorithms and implementations have been applied to both synthetic and real datasets with differences regarding missing data, present in cases where pathologies and abnormalities arise. The datasets include single acquisitions and complete cardiac cycles. The goodness of the reconstructions has been evaluated with medical parameters in order to compare our results with those retrieved by typical software used by physicians.Besides the direct application to medicine diagnosis, our methodology is suitable for generic reconstructions in the field of computer graphics. Our reconstructions can serve for getting 3D models at low cost, in terms of manual interaction and CPU computation overhead. Furthermore, our method is a robust tessellation algorithm that builds surfaces from clouds of points that can be retrieved from laser scanners or magnetic sensors, among other available hardware