Physics-Based Modeling of Nonrigid Objects for Vision and Graphics (Dissertation)

This thesis develops a physics-based framework for 3D shape and nonrigid motion modeling for computer vision and computer graphics. In computer vision it addresses the problems of complex 3D shape representation, shape reconstruction, quantitative model extraction from biomedical data for analysis and visualization, shape estimation, and motion tracking. In computer graphics it demonstrates the generative power of our framework to synthesize constrained shapes, nonrigid object motions and object interactions for the purposes of computer animation. Our framework is based on the use of a new class of dynamically deformable primitives which allow the combination of global and local deformations. It incorporates physical constraints to compose articulated models from deformable primitives and provides force-based techniques for fitting such models to sparse, noise-corrupted 2D and 3D visual data. The framework leads to shape and nonrigid motion estimators that exploit dynamically deformable models to track moving 3D objects from time-varying observations. We develop models with global deformation parameters which represent the salient shape features of natural parts, and local deformation parameters which capture shape details. In the context of computer graphics, these models represent the physics-based marriage of the parameterized and free-form modeling paradigms. An important benefit of their global/local descriptive power in the context of computer vision is that it can potentially satisfy the often conflicting requirements of shape reconstruction and shape recognition. The Lagrange equations of motion that govern our models, augmented by constraints, make them responsive to externally applied forces derived from input data or applied by the user. This system of differential equations is discretized using finite element methods and simulated through time using standard numerical techniques. We employ these equations to formulate a shape and nonrigid motion estimator. The estimator is a continuous extended Kalman filter that recursively transforms the discrepancy between the sensory data and the estimated model state into generalized forces. These adjust the translational, rotational, and deformational degrees of freedom such that the model evolves in a consistent fashion with the noisy data. We demonstrate the interactive time performance of our techniques in a series of experiments in computer vision, graphics, and visualization

Superquadric representation of scenes from multi-view range data

Object representation denotes representing three-dimensional (3D) real-world objects with known graphic or mathematic primitives recognizable to computers. This research has numerous applications for object-related tasks in areas including computer vision, computer graphics, reverse engineering, etc. Superquadrics, as volumetric and parametric models, have been selected to be the representation primitives throughout this research. Superquadrics are able to represent a large family of solid shapes by a single equation with only a few parameters. This dissertation addresses superquadric representation of multi-part objects and multiobject scenes. Two issues motivate this research. First, superquadric representation of multipart objects or multi-object scenes has been an unsolved problem due to the complex geometry of objects. Second, superquadrics recovered from single-view range data tend to have low confidence and accuracy due to partially scanned object surfaces caused by inherent occlusions. To address these two problems, this dissertation proposes a multi-view superquadric representation algorithm. By incorporating both part decomposition and multi-view range data, the proposed algorithm is able to not only represent multi-part objects or multi-object scenes, but also achieve high confidence and accuracy of recovered superquadrics. The multi-view superquadric representation algorithm consists of (i) initial superquadric model recovery from single-view range data, (ii) pairwise view registration based on recovered superquadric models, (iii) view integration, (iv) part decomposition, and (v) final superquadric fitting for each decomposed part. Within the multi-view superquadric representation framework, this dissertation proposes a 3D part decomposition algorithm to automatically decompose multi-part objects or multiobject scenes into their constituent single parts consistent with human visual perception. Superquadrics can then be recovered for each decomposed single-part object. The proposed part decomposition algorithm is based on curvature analysis, and includes (i) Gaussian curvature estimation, (ii) boundary labeling, (iii) part growing and labeling, and (iv) post-processing. In addition, this dissertation proposes an extended view registration algorithm based on superquadrics. The proposed view registration algorithm is able to handle deformable superquadrics as well as 3D unstructured data sets. For superquadric fitting, two objective functions primarily used in the literature have been comprehensively investigated with respect to noise, viewpoints, sample resolutions, etc. The objective function proved to have better performance has been used throughout this dissertation. In summary, the three algorithms (contributions) proposed in this dissertation are generic and flexible in the sense of handling triangle meshes, which are standard surface primitives in computer vision and graphics. For each proposed algorithm, the dissertation presents both theory and experimental results. The results demonstrate the efficiency of the algorithms using both synthetic and real range data of a large variety of objects and scenes. In addition, the experimental results include comparisons with previous methods from the literature. Finally, the dissertation concludes with a summary of the contributions to the state of the art in superquadric representation, and presents possible future extensions to this research

Massively Parallel Approach to Modeling 3D Objects in Machine Vision

Electrical Engineerin

The mitral valve computational anatomy and geometry analysis

We present a novel methodology to characterize and quantify the Mitral Valve (MV) geometry and physical attributes in a multi-resolution framework. A multi-scale decomposition was implemented to model the MV geometry by using superquadric shape primitives and spectral reconstruction of the finer-scale geometric details. Superquadrics provide a basis to normalize the size and approximate a basic model of the MV geometry. The point-wise difference between the original geometry and the superquadric model denotes the finer-scale geometric details, which can be modeled as a scalar attribute for the MV model development. The additive decomposition of the basic MV geometry from geometric details (attributes) allows recovering the actual geometry by superposition of the superquadric approximation and the finer-details model. We implemented a lasso optimization algorithm to perform spectral analysis and develop the Fourier reconstruction of the geometric details. The spectral modeling enabled us to resample the geometric details or use spectral filters in order to adjust the spatial resolution in the model reconstruction. It also provides the basis to control the level of detail in the final model reconstruction by applying low-pass filters in the frequency domain. The higher-order attributes such as internal fiber architecture can be integrated with the geometric models using the same framework. We applied our pipeline to create models of three ovine MVs based on computed-tomography 3D images with micrometer resolution. We were able to quantify the MV leaflet geometry, reconstruct models with custom level of geometric details, and develop medial representation of the MV leaflet structure. The results show that our methodology for geometry analysis provides a basis for assessing patient-specific geometries and facilitates developing population-averaged models. Ultimately, this approach allows building personalized image-based computational models for medical device design and surgical treatment simulations.Mechanical Engineerin

\u3cem\u3eGRASP News\u3c/em\u3e, Volume 6, Number 1

A report of the General Robotics and Active Sensory Perception (GRASP) Laboratory, edited by Gregory Long and Alok Gupta

On-Orbit Manoeuvring Using Superquadric Potential Fields

On-orbit manoeuvring represents an essential process in many space missions such as orbital assembly, servicing and reconfiguration. A new methodology, based on the potential field method along with superquadric repulsive potentials, is discussed in this thesis. The methodology allows motion in a cluttered environment by combining translation and rotation in order to avoid collisions. This combination reduces the manoeuvring cost and duration, while allowing collision avoidance through combinations of rotation and translation. Different attractive potential fields are discussed: parabolic, conic, and a new hyperbolic potential. The superquadric model is used to represent the repulsive potential with several enhancements. These enhancements are: accuracy of separation distance estimation, modifying the model to be suitable for moving obstacles, and adding the effect of obstacle rotation through quaternions. Adding dynamic parameters such as object translational velocity and angular velocity to the potential field can lead to unbounded actuator control force. This problem is overcome in this thesis through combining parabolic and conic functions to form an attractive potential or through using a hyperbolic function. The global stability and convergence of the solution is guaranteed through the appropriate choice of the control laws based on Lyapunov's theorem. Several on-orbit manoeuvring problems are then conducted such as on-orbit assembly using impulsive and continuous strategies, structure disassembly and reconfiguration and free-flyer manoeuvring near a space station. Such examples demonstrate the accuracy and robustness of the method for on-orbit motion planning

Model-Based Shape and Motion Analysis: Left Ventricle of a Heart

The accurate and clinically useful estimation of the shape, motion, and deformation of the left ventricle of a heart (LV) is an important yet open research problem. Recently, computer vision techniques for reconstructing the 3-D shape and motion of the LV have been developed. The main drawback of these techniques, however, is that their models are formulated in terms of either too many local parameters that require non-trivial processing to be useful for close to real time diagnosis, or too few parameters to offer an adequate approximation to the LV motion. To address the problem, we present a new class of volumetric primitives for a compact and accurate LV shape representation in which model parameters are functions. Lagrangian dynamics are employed to convert geometric models into dynamic models that can deform according to the forces manifested in the data points. It is thus possible to make a precise estimation of the deformation of the LV shape endocardial, epicardial and anywhere in between with a small number of intuitive parameter functions. We believe that the proposed technique has a wide range of potential applications. In this thesis, we demonstrate the possibility by applying it to the 3-D LV shape and motion characterization from magnetic tagging data (MRI-SPAMM). We show that the results of our experiments with normal and abnormal heart data enable us to quantitatively verify the physicians\u27 qualitative conception of the left ventricular wall motion

Sense, Think, Grasp: A study on visual and tactile information processing for autonomous manipulation

Interacting with the environment using hands is one of the distinctive abilities of humans with respect to other species. This aptitude reflects on the crucial role played by objects\u2019 manipulation in the world that we have shaped for us. With a view of bringing robots outside industries for supporting people during everyday life, the ability of manipulating objects autonomously and in unstructured environments is therefore one of the basic skills they need. Autonomous manipulation is characterized by great complexity especially regarding the processing of sensors information to perceive the surrounding environment. Humans rely on vision for wideranging tridimensional information, prioprioception for the awareness of the relative position of their own body in the space and the sense of touch for local information when physical interaction with objects happens. The study of autonomous manipulation in robotics aims at transferring similar perceptive skills to robots so that, combined with state of the art control techniques, they could be able to achieve similar performance in manipulating objects. The great complexity of this task makes autonomous manipulation one of the open problems in robotics that has been drawing increasingly the research attention in the latest years. In this work of Thesis, we propose possible solutions to some key components of autonomous manipulation, focusing in particular on the perception problem and testing the developed approaches on the humanoid robotic platform iCub. When available, vision is the first source of information to be processed for inferring how to interact with objects. The object modeling and grasping pipeline based on superquadric functions we designed meets this need, since it reconstructs the object 3D model from partial point cloud and computes a suitable hand pose for grasping the object. Retrieving objects information with touch sensors only is a relevant skill that becomes crucial when vision is occluded, as happens for instance during physical interaction with the object. We addressed this problem with the design of a novel tactile localization algorithm, named Memory Unscented Particle Filter, capable of localizing and recognizing objects relying solely on 3D contact points collected on the object surface. Another key point of autonomous manipulation we report on in this Thesis work is bi-manual coordination. The execution of more advanced manipulation tasks in fact might require the use and coordination of two arms. Tool usage for instance often requires a proper in-hand object pose that can be obtained via dual-arm re-grasping. In pick-and-place tasks sometimes the initial and target position of the object do not belong to the same arm workspace, then requiring to use one hand for lifting the object and the other for locating it in the new position. At this regard, we implemented a pipeline for executing the handover task, i.e. the sequences of actions for autonomously passing an object from one robot hand on to the other. The contributions described thus far address specific subproblems of the more complex task of autonomous manipulation. This actually differs from what humans do, in that humans develop their manipulation skills by learning through experience and trial-and-error strategy. Aproper mathematical formulation for encoding this learning approach is given by Deep Reinforcement Learning, that has recently proved to be successful in many robotics applications. For this reason, in this Thesis we report also on the six month experience carried out at Berkeley Artificial Intelligence Research laboratory with the goal of studying Deep Reinforcement Learning and its application to autonomous manipulation

Deformable solids and displacement maps--a multi-scale technique for model recovery and recognition

Thesis (M.S.)--Massachusetts Institute of Technology, Dept. of Architecture, 1991.Includes bibliographical references (leaves 74-78).by Stanley Edward Sclaroff.M.S

Model driven segmentation and the detection of bone fractures

Bibliography: leaves 83-90.The introduction of lower dosage image acquisition devices and the increase in computational power means that there is an increased focus on producing diagnostic aids for the medical trauma environment. The focus of this research is to explore whether geometric criteria can be used to detect bone fractures from Computed Tomography data. Conventional image processing of CT data is aimed at the production of simple iso-surfaces for surgical planning or diagnosis - such methods are not suitable for the automated detection of fractures. Our hypothesis is that through a model-based technique a triangulated surface representing the bone can be speedily and accurately produced. And, that there is sufficient structural information present that by examining the geometric structure of this representation we can accurately detect bone fractures. In this dissertation we describe the algorithms and framework that we built to facilitate the detection of bone fractures and evaluate the validity of our approach