Calipso: Physics-based Image and Video Editing through CAD Model Proxies

We present Calipso, an interactive method for editing images and videos in a physically-coherent manner. Our main idea is to realize physics-based manipulations by running a full physics simulation on proxy geometries given by non-rigidly aligned CAD models. Running these simulations allows us to apply new, unseen forces to move or deform selected objects, change physical parameters such as mass or elasticity, or even add entire new objects that interact with the rest of the underlying scene. In Calipso, the user makes edits directly in 3D; these edits are processed by the simulation and then transfered to the target 2D content using shape-to-image correspondences in a photo-realistic rendering process. To align the CAD models, we introduce an efficient CAD-to-image alignment procedure that jointly minimizes for rigid and non-rigid alignment while preserving the high-level structure of the input shape. Moreover, the user can choose to exploit image flow to estimate scene motion, producing coherent physical behavior with ambient dynamics. We demonstrate Calipso's physics-based editing on a wide range of examples producing myriad physical behavior while preserving geometric and visual consistency.Comment: 11 page

Volumetric modeling and interactive cutting of deformable bodies

Special Issue on Biomechanical Modelling of Soft Tissue MotionInternational audienceA new approach for the interactive simulation of viscoelastic object cutting is presented. Two synchronized geo- metrical models at different resolutions are used, both derived from medical images. In contrast with most previous approaches, the blade deforms the object, and cutting occurs once a contact pressure threshold is exceeded. Moreover, we achieve interactive simulation rates by embedding a high resolution geometry within a regular grid with arbitrary resolution. This allows to trade off accuracy for speed in the computation of deformations. The input data is a high- resolution volumetric model of the objects. The surface model of the object, used for rendering as well as collision detection and response, is a polygonal level set of the volumetric data. It is embedded in the volume model using barycentric coordinates. Cutting is performed by removing voxels at the fine level, and updating the surface and volume models accordingly. We introduce a new data structure, which we call a Dynamic Branched Grid, in order to preserve the fine level topology at the coarse level. When an element of the coarse volumetric model is cut, it is replaced by a number of superimposed elements with the same size and at the same rest position as the original one. Each new element is assigned a part of material contained in the original one, and the mass and stiffness are recomputed accordingly. The well-known problem of creating small, ill-shaped finite elements while remeshing is thus completely avoided

Accurate Real-Time Framework for Complex Pre-defined Cuts in Finite Element Modeling

PhD ThesisAchieving detailed pre-defined cuts on deformable materials is vitally pivotal for many commercial applications, such as cutting scenes in games and vandalism effects in virtual movies. In these types of applications, the majority of resources are allocated to achieve high-fidelity representations of materials and the virtual environments. In the case of limited computing resources, it is challenging to achieve a convincing cutting effect. On the premise of sacrificing realism effects or computational cost, a considerable amount of research work has been carried out, but the best solution that can be compatible with both cases has not yet been identified. This doctoral dissertation is dedicated to developing a unique framework for representing pre-defined cuts of deformable surface models, which can achieve real-time, detailed cutting while maintaining the realistic physical behaviours. In order to achieve this goal, we have made in-depth explorations from geometric and numerical perspectives. From a geometric perspective, we propose a robust subdivision mechanism that allows users to make arbitrary predetermined cuts on elastic surface models based on the finite element method (FEM). Specifically, after the user separates the elements in an arbitrary manner (i.e., linear or non-linear) on the topological mesh, we then optimise the resulting mesh by regenerating the triangulation within the element based on the Delaunay triangulation principle. The optimisation of regenerated triangles, as a process of refining the ill-shaped elements that have small Aspect Ratio, greatly improves the realism of physical behaviours and guarantees that the refinement process is balanced with real-time requirements. The above subdivision mechanism can improve the visual effect of cutting, but it neglects the fact that elements cannot be perfectly cut through any pre-defined trajectories. The number of ill-shaped elements generated yield a significant impact on the optimisation time: a large number of ill-shaped elements will render the cutting slow or even collapse, and vice versa. Our idea is based on the core observation that the producing of ill-shaped elements is largely attributed to the condition number of the global stiffness matrix. Practically, for a stiffness matrix, a large condition number means that it is almost singular, and the calculation of its inverse or the solution of a system of linear equations are prone to large numerical errors and time-consuming. It motivates us to alleviate the impact of condition number of the global stiffness matrix from the numerical aspects. Specifically, we address this issue in a novel manner by converting the global stiffness matrix into the form of a covariance matrix, in which the number of conditions of the matrix can be reduced by exploiting appropriate matrix normalisation to the eigenvalues. Furthermore, we investigated the efficiency of two different scenarios: an exact square-root normalisation and its approximation based on the Newton-Schulz iteration. Experimental tests of the proposed framework demonstrate that it can successfully reproduce competitive visuals of detailed pre-defined cuts compared with the state-of-the-art method (Manteaux et al. 2015) while obtaining a significant improvement on the FPS, increasing up to 46.49 FPS and 21.93 FPS during and after the cuts, respectively. Also, the new refinement method can stably maintain the average Aspect Ratio of the model mesh after the cuts at less than 3 and the average Area Ratio around 3%. Besides, the proposed two matrix normalisation strategies, including ES-CGM and AS-CGM, have shown the superiority of time efficiency compared with the baseline method (Xin et al. 2018). Specifically, the ES-CGM and AS-CGM methods obtained 5 FPS and 10 FPS higher than the baseline method, respectively. These experimental results strongly support our conclusion which is that this new framework would provide significant benefits when implemented for achieving detailed pre-defined cuts at a real-time rate

Adaptive Physically Based Models in Computer Graphics

International audienceOne of the major challenges in physically-based modeling is making simulations efficient. Adaptive models provide an essential solution to these efficiency goals. These models are able to self-adapt in space and time, attempting to provide the best possible compromise between accuracy and speed. This survey reviews the adaptive solutions proposed so far in computer graphics. Models are classified according to the strategy they use for adaptation, from time-stepping and freezing techniques to geometric adaptivity in the form of structured grids, meshes, and particles. Applications range from fluids, through deformable bodies, to articulated solids

Meshless Mechanics and Point-Based Visualization Methods for Surgical Simulations

Computer-based modeling and simulation practices have become an integral part of the medical education field. For surgical simulation applications, realistic constitutive modeling of soft tissue is considered to be one of the most challenging aspects of the problem, because biomechanical soft-tissue models need to reflect the correct elastic response, have to be efficient in order to run at interactive simulation rates, and be able to support operations such as cuts and sutures. Mesh-based solutions, where the connections between the individual degrees of freedom (DoF) are defined explicitly, have been the traditional choice to approach these problems. However, when the problem under investigation contains a discontinuity that disrupts the connectivity between the DoFs, the underlying mesh structure has to be reconfigured in order to handle the newly introduced discontinuity correctly. This reconfiguration for mesh-based techniques is typically called dynamic remeshing, and most of the time it causes the performance bottleneck in the simulation. In this dissertation, the efficiency of point-based meshless methods is investigated for both constitutive modeling of elastic soft tissues and visualization of simulation objects, where arbitrary discontinuities/cuts are applied to the objects in the context of surgical simulation. The point-based deformable object modeling problem is examined in three functional aspects: modeling continuous elastic deformations with, handling discontinuities in, and visualizing a point-based object. Algorithmic and implementation details of the presented techniques are discussed in the dissertation. The presented point-based techniques are implemented as separate components and integrated into the open-source software framework SOFA. The presented meshless continuum mechanics model of elastic tissue were verified by comparing it to the Hertzian non-adhesive frictionless contact theory. Virtual experiments were setup with a point-based deformable block and a rigid indenter, and force-displacement curves obtained from the virtual experiments were compared to the theoretical solutions. The meshless mechanics model of soft tissue and the integrated novel discontinuity treatment technique discussed in this dissertation allows handling cuts of arbitrary shape. The implemented enrichment technique not only modifies the internal mechanics of the soft tissue model, but also updates the point-based visual representation in an efficient way preventing the use of costly dynamic remeshing operations

Découpage virtuel interactif de corps élastiques pour simulation chirurgicale

''RÉSUMÉ : La simulation chirurgicale dans un environnement de réalité virtuelle fournit un moyen de pratiquer certaines opérations sans les risques associés à une intervention sur un patient ou le coût d’un mannequin. Afin de générer un sentiment de présence, on cherche à produire un environnement le plus complet possible, incluant une vision 3D, un retour haptique, des interactions crédibles et un comportement physique réaliste des objets présents. La recherche présentée ici porte sur la simulation physique du comportement d’organes mous, comme le foie ou le cerveau, ainsi que sur le découpage de ces organes à l’aide d’un scalpel, une interaction particulièrement difficile à reproduire virtuellement de façon réaliste. L’objectif principal est de développer une méthode de déformation à la fois réaliste et efficace, et de permettre à un utilisateur de découper interactivement un objet simulé par cette méthode, à l’aide d’un outil tranchant virtuel. De plus, nous voulons que la déformation et les interactions soient décrites avec une grande précision, tout en permettant d’effectuer les calculs très rapidement, pour une interaction fluide qui maintient le sentiment de présence.''----------''ABSTRACT : Surgery simulation in a virtual reality environment provides a way to practice certain operations without the risks associated with performing surgery on a patient or the cost of using arealistic dummy. To facilitate immersion, we seek to produce an environment as complete as possible, including 3D vision, haptic feedback, credible interactions and a realistic physical behavior of simulated objects. The research presented in this document focuses on the physical behavior of soft organs, like the brain or liver, and on cutting these organs using a scalpel. It is especially difficult to reproduce virtually that interaction in a realistic way. The main objective is to develop a deformation method that is both realistic and efficient, and to allow a user to interactively cut an object simulated through this method, using a virtual sharp tool. Furthermore, we want the deformation and interactions to be described with high precision while allowing for fast computations, for a smooth interaction that maintains immersion.'

Design and development of a VR system for exploration of medical data using haptic rendering and high quality visualization

[no abstract

Surgical Augmented Reality with Topological Changes

International audienceThe visualization of internal structures of organs in minimally invasive surgery is an important avenue for improving the perception of the surgeon, or for supporting planning and decision systems. However, current methods dealing with non-rigid augmented reality only provide augmentation when the topology of the organ is not modified. In this paper we solve this shortcoming by introducing a method for physics-based non-rigid augmented reality. Singularities caused by topo-logical changes are detected and propagated to the pre-operative model. This significantly improves the coherence between the actual laparascopic view and the model, and provides added value in terms of navigation and decision making. Our real time augmentation algorithm is assessed on a video showing the cut of a porcine liver's lobe in minimal invasive surgery

Aeronautical engineering. A continuing bibliography with indexes

This bibliography lists 326 reports, articles, and other documents introduced into the NASA scientific and technical information system in January 1982. Topics on aeronautical engineering and aerodynamics such as flight control systems, avionics, computer programs, computational fluid dynamics and composite structures are covered

A biomechanics-based articulation model for medical applications

Computer Graphics came into the medical world especially after the arrival of 3D medical imaging. Computer Graphics techniques are already integrated in the diagnosis procedure by means of the visual tridimensional analysis of computer tomography, magnetic resonance and even ultrasound data. The representations they provide, nevertheless, are static pictures of the patients' body, lacking in functional information. We believe that the next step in computer assisted diagnosis and surgery planning depends on the development of functional 3D models of human body. It is in this context that we propose a model of articulations based on biomechanics. Such model is able to simulate the joint functionality in order to allow for a number of medical applications. It was developed focusing on the following requirements: it must be at the same time simple enough to be implemented on computer, and realistic enough to allow for medical applications; it must be visual in order for applications to be able to explore the joint in a 3D simulation environment. Then, we propose to combine kinematical motion for the parts that can be considered as rigid, such as bones, and physical simulation of the soft tissues. We also deal with the interaction between the different elements of the joint, and for that we propose a specific contact management model. Our kinematical skeleton is based on anatomy. Special considerations have been taken to include anatomical features like axis displacements, range of motion control, and joints coupling. Once a 3D model of the skeleton is built, it can be simulated by data coming from motion capture or can be specified by a specialist, a clinician for instance. Our deformation model is an extension of the classical mass-spring systems. A spherical volume is considered around mass points, and mechanical properties of real materials can be used to parameterize the model. Viscoelasticity, anisotropy and non-linearity of the tissues are simulated. We particularly proposed a method to configure the mass-spring matrix such that the objects behave according to a predefined Young's modulus. A contact management model is also proposed to deal with the geometric interactions between the elements inside the joint. After having tested several approaches, we proposed a new method for collision detection which measures in constant time the signed distance to the closest point for each point of two meshes subject to collide. We also proposed a method for collision response which acts directly on the surfaces geometry, in a way that the physical behavior relies on the propagation of reaction forces produced inside the tissue. Finally, we proposed a 3D model of a joint combining the three elements: anatomical skeleton motion, biomechanical soft tissues deformation, and contact management. On the top of that we built a virtual hip joint and implemented a set of medical applications prototypes. Such applications allow for assessment of stress distribution on the articular surfaces, range of motion estimation based on ligament constraint, ligament elasticity estimation from clinically measured range of motion, and pre- and post-operative evaluation of stress distribution. Although our model provides physicians with a number of useful variables for diagnosis and surgery planning, it should be improved for effective clinical use. Validation has been done partially. However, a global clinical validation is necessary. Patient specific data are still difficult to obtain, especially individualized mechanical properties of tissues. The characterization of material properties in our soft tissues model can also be improved by including control over the shear modulus