Calipso: Physics-based Image and Video Editing through CAD Model Proxies

We present Calipso, an interactive method for editing images and videos in a physically-coherent manner. Our main idea is to realize physics-based manipulations by running a full physics simulation on proxy geometries given by non-rigidly aligned CAD models. Running these simulations allows us to apply new, unseen forces to move or deform selected objects, change physical parameters such as mass or elasticity, or even add entire new objects that interact with the rest of the underlying scene. In Calipso, the user makes edits directly in 3D; these edits are processed by the simulation and then transfered to the target 2D content using shape-to-image correspondences in a photo-realistic rendering process. To align the CAD models, we introduce an efficient CAD-to-image alignment procedure that jointly minimizes for rigid and non-rigid alignment while preserving the high-level structure of the input shape. Moreover, the user can choose to exploit image flow to estimate scene motion, producing coherent physical behavior with ambient dynamics. We demonstrate Calipso's physics-based editing on a wide range of examples producing myriad physical behavior while preserving geometric and visual consistency.Comment: 11 page

Shell Model for Reconstruction and Real-Time Simulation of Thin Anatomical Structures

International audienceThis paper presents a new modelling technique for the defor- mation of thin anatomical structures like membranes and hollow organs. We show that the behaviour of this type of surface tissue can be ab- stracted with a modelling of their elastic resistance using shell theory. In order to apply the shell theory in the context of medical simulation, our method propose to base the geometrical reconstruction of the organ on the shape functions of the shell element. Moreover, we also use these continuous shape functions to handle the contacts and the interactions with other types of deformable tissues. The technique is illustrated using several examples including the simulation of an angioplasty procedure

Efficient Contact Modeling using Compliance Warping

International audienceContact handling is the key of deformable objects simulation, since without it, objects can not interact with their environment nor with the user. In this paper, we propose a novel and very efficient approach for precise computation of contact response between various types of objects commonly used in computer animation. Being constraint based, this method ensures physical correctness, and respects Singorini s law. It can be used with any deformation model, and is based on the use of the initial compliance matrix and contact warping. Thus, the contact response can be computed efficiently, and the object deformation can still be done in a physically plausible way provided the underlying model is physical

Improving depth perception during surgical augmented reality

International audienceThis study suggests a method to compensate the loss of depth perception while enhancing organ vessels and tumors to surgeons. This method relies on a combination of contour rendering technique and adaptive alpha blending to effectively perceive the vessels and tumors depth. In addition, this technique is designed to achieve real-time to satisfy the requirements of clinical routines, and has been tested on real human surgery

Segmentation and Labelling of Intra-operative Laparoscopic Images using Structure from Point Cloud

International audienceWe present in this paper an automatic method for segmenting and labelling of liver its surrounding tissues in intra-operative laparoscopic images. The goal is to be able to distinguished between the different structure that compose a common intra-operative hepatic surgery scene. This will permits to improve the registration between pre-operative data and intra-operative images for task such as Augmented Reality. Our segmentation method consider the scene as a 3D structured point cloud instead of a laparoscopic images in order to exploit powerful informations such as curvature and normals, in addition to visual cues that permits to efficiently classify the scene. Our approach works well on sparse and noisy point clouds, thanks to a surface approximation stage, and unlike existing approaches, is independent of organs texture in the image. Experiements performed on challenging human hepatic surgery confirm that accurate segmentation and labelling are possible using 3D structure information and appropriate visual cues

Simultaneous Pose Estimation and Augmentation of Elastic Surfaces from a Moving Monocular Camera

International audienceWe present in this paper an original method to estimate the pose of a monocular camera while simultaneously modeling and capturing the elastic deformation of the object to be augmented. Our method tackles a challenging problem where ambiguities between rigid motion and non-rigid deformation are present. This issue represents a major lock for the establishment of an efficient surgical augmented reality where endoscopic camera moves and organs deform. Using an underlying physical model to estimate the low stressed regions our algorithm separates the rigid body motion from the elastic deformations using polar decomposition of the strain tensor. Following this decomposition, a constrained minimization, that encodes both the optical and the physical constraints, is resolved at each frame. Results on real and simulated data are exposed to show the effectiveness of our approach

Fracture in Augmented Reality

International audienceWe propose in this study an image-guided mesh cutting method to handle real-time augmentation of paper tearing. This method relieson the combination of visually-based fracture tracking algorithm and a physics-based model that is dynamically superimposed on the image

Surgery Training, Planning and Guidance Using the SOFA Framework

International audienceIn recent years, an active development of novel technologies dealing with medical training, planning and guidance has become an increasingly important area of interest in both research and health-care manufacturing. A combination of advanced physical models, realistic human-computer interaction and growing computational power is bringing new solutions in order to help both medical students and experts to achieve a higher degree of accuracy and reliability in surgical interventions. In this paper, we present three different examples of medical physically-based simulations implemented in a common software platform called SOFA. Each example represents a different application: training for cardiac electrophysiology, pre-operative planning of cryosurgery and per-operative guidance for laparoscopy. The goal of this presentation is to evaluate the realism, accuracy and efficiency of the simulations, as well as to demonstrate the potential and flexibility of the SOFA platform

Augmented Reality during Cutting and Tearing of Deformable Objects

International audienceCurrent methods dealing with non-rigid augmented reality only provide an augmented view when the topology of the tracked object is not modified, which is an important limitation. In this paper we solve this shortcoming by introducing a method for physics-based non-rigid augmented reality. Singularities caused by topological changes are detected by analyzing the displacement field of the underlying deformable model. These topological changes are then applied to the physics-based model to approximate the real cut. All these steps, from deformation to cutting simulation, are performed in real-time. This significantly improves the coherence between the actual view and the model, and provides added value