Finite Element Based Tracking of Deforming Surfaces

We present an approach to robustly track the geometry of an object that deforms over time from a set of input point clouds captured from a single viewpoint. The deformations we consider are caused by applying forces to known locations on the object's surface. Our method combines the use of prior information on the geometry of the object modeled by a smooth template and the use of a linear finite element method to predict the deformation. This allows the accurate reconstruction of both the observed and the unobserved sides of the object. We present tracking results for noisy low-quality point clouds acquired by either a stereo camera or a depth camera, and simulations with point clouds corrupted by different error terms. We show that our method is also applicable to large non-linear deformations.Comment: additional experiment

Bridging the computational gap between mesoscopic and continuum modeling of red blood cells for fully resolved blood flow

We present a computational framework for the simulation of blood flow with fully resolved red blood cells (RBCs) using a modular approach that consists of a lattice Boltzmann solver for the blood plasma, a novel finite element based solver for the deformable bodies and an immersed boundary method for the fluid-solid interaction. For the RBCs, we propose a nodal projective FEM (npFEM) solver which has theoretical advantages over the more commonly used mass-spring systems (mesoscopic modeling), such as an unconditional stability, versatile material expressivity, and one set of parameters to fully describe the behavior of the body at any mesh resolution. At the same time, the method is substantially faster than other FEM solvers proposed in this field, and has an efficiency that is comparable to the one of mesoscopic models. At its core, the solver uses specially defined potential energies, and builds upon them a fast iterative procedure based on quasi-Newton techniques. For a known material, our solver has only one free parameter that demands tuning, related to the body viscoelasticity. In contrast, state-of-the-art solvers for deformable bodies have more free parameters, and the calibration of the models demands special assumptions regarding the mesh topology, which restrict their generality and mesh independence. We propose as well a modification to the potential energy proposed by Skalak et al. 1973 for the red blood cell membrane, which enhances the strain hardening behavior at higher deformations. Our viscoelastic model for the red blood cell, while simple enough and applicable to any kind of solver as a post-convergence step, can capture accurately the characteristic recovery time and tank-treading frequencies. The framework is validated using experimental data, and it proves to be scalable for multiple deformable bodies

Calipso: Physics-based Image and Video Editing through CAD Model Proxies

We present Calipso, an interactive method for editing images and videos in a physically-coherent manner. Our main idea is to realize physics-based manipulations by running a full physics simulation on proxy geometries given by non-rigidly aligned CAD models. Running these simulations allows us to apply new, unseen forces to move or deform selected objects, change physical parameters such as mass or elasticity, or even add entire new objects that interact with the rest of the underlying scene. In Calipso, the user makes edits directly in 3D; these edits are processed by the simulation and then transfered to the target 2D content using shape-to-image correspondences in a photo-realistic rendering process. To align the CAD models, we introduce an efficient CAD-to-image alignment procedure that jointly minimizes for rigid and non-rigid alignment while preserving the high-level structure of the input shape. Moreover, the user can choose to exploit image flow to estimate scene motion, producing coherent physical behavior with ambient dynamics. We demonstrate Calipso's physics-based editing on a wide range of examples producing myriad physical behavior while preserving geometric and visual consistency.Comment: 11 page

Occlusion Coherence: Detecting and Localizing Occluded Faces

The presence of occluders significantly impacts object recognition accuracy. However, occlusion is typically treated as an unstructured source of noise and explicit models for occluders have lagged behind those for object appearance and shape. In this paper we describe a hierarchical deformable part model for face detection and landmark localization that explicitly models part occlusion. The proposed model structure makes it possible to augment positive training data with large numbers of synthetically occluded instances. This allows us to easily incorporate the statistics of occlusion patterns in a discriminatively trained model. We test the model on several benchmarks for landmark localization and detection including challenging new data sets featuring significant occlusion. We find that the addition of an explicit occlusion model yields a detection system that outperforms existing approaches for occluded instances while maintaining competitive accuracy in detection and landmark localization for unoccluded instances

Displacement-based grasping of deformable objects

Robotic grasping of deformable objects is inherently different from that of rigid objects, and is an under-researched area. Difficulties arise not only from expensive deformable modeling, but also from the changing object geometry under grasping force. This dissertation studies strategies of grasping deformable objects using two robotic fingers. Discovering the inapplicability of the traditional force-centered grasping strategies for rigid objects, I have designed an approach for grasping deformable objects that specifies finger displacements. This not only ensures equilibrium under the elasticity theory, but also enhances stability and simplifies finger control in the implementation. Deformable modeling is carried out using the Finite Element Method (FEM), for which our analysis establishes uniqueness of the shape of a grasped object given the finger displacements. Meanwhile, preprocessing based on the Singular Value Decomposition greatly reduces the complexity of computation. Grasping strategies have been investigated for both 2D and 3D objects. With a hollow 2D object, the grasping fingers make point contacts. The condition of a successful grasp is that the friction cones at the two contacts must contain the line segment through them before and after the deformation. With a solid planar object, the fingers make area contacts. Grasp computation is carried out by an event-driven algorithm, which has been validated by our robot experiments. For 3D objects, a simple squeeze-and-test strategy has been designed to lift them off the supporting table against gravity with a method that predicts the squeeze amounts. In reality, objects shapes are affected to various degrees by gravity, but such a effect has been ignored in the FEM-based modeling. For accuracy, the gravity-free shape of an object is sometimes needed. I have introduced an iterative algorithm that will converge to such shape as a fixed point . In the last part of my thesis, I study planning of the finger squeeze paths, not to limited by straight movements. The objective is to not only enlarge the range of finger placements for successful grasps, but also improve stability and energy efficiency. I have designed a path planning algorithm based on the Rapidly-exploring Random Trees (RRT) that is able to achieve certain optimalities

Real-time Error Control for Surgical Simulation

Objective: To present the first real-time a posteriori error-driven adaptive finite element approach for real-time simulation and to demonstrate the method on a needle insertion problem. Methods: We use corotational elasticity and a frictional needle/tissue interaction model. The problem is solved using finite elements within SOFA. The refinement strategy relies upon a hexahedron-based finite element method, combined with a posteriori error estimation driven local $h$-refinement, for simulating soft tissue deformation. Results: We control the local and global error level in the mechanical fields (e.g. displacement or stresses) during the simulation. We show the convergence of the algorithm on academic examples, and demonstrate its practical usability on a percutaneous procedure involving needle insertion in a liver. For the latter case, we compare the force displacement curves obtained from the proposed adaptive algorithm with that obtained from a uniform refinement approach. Conclusions: Error control guarantees that a tolerable error level is not exceeded during the simulations. Local mesh refinement accelerates simulations. Significance: Our work provides a first step to discriminate between discretization error and modeling error by providing a robust quantification of discretization error during simulations.Comment: 12 pages, 16 figures, change of the title, submitted to IEEE TBM

Subspace procrustes analysis

Postprint (author's final draft