Calipso: Physics-based Image and Video Editing through CAD Model Proxies

We present Calipso, an interactive method for editing images and videos in a physically-coherent manner. Our main idea is to realize physics-based manipulations by running a full physics simulation on proxy geometries given by non-rigidly aligned CAD models. Running these simulations allows us to apply new, unseen forces to move or deform selected objects, change physical parameters such as mass or elasticity, or even add entire new objects that interact with the rest of the underlying scene. In Calipso, the user makes edits directly in 3D; these edits are processed by the simulation and then transfered to the target 2D content using shape-to-image correspondences in a photo-realistic rendering process. To align the CAD models, we introduce an efficient CAD-to-image alignment procedure that jointly minimizes for rigid and non-rigid alignment while preserving the high-level structure of the input shape. Moreover, the user can choose to exploit image flow to estimate scene motion, producing coherent physical behavior with ambient dynamics. We demonstrate Calipso's physics-based editing on a wide range of examples producing myriad physical behavior while preserving geometric and visual consistency.Comment: 11 page

Meshless Mechanics and Point-Based Visualization Methods for Surgical Simulations

Computer-based modeling and simulation practices have become an integral part of the medical education field. For surgical simulation applications, realistic constitutive modeling of soft tissue is considered to be one of the most challenging aspects of the problem, because biomechanical soft-tissue models need to reflect the correct elastic response, have to be efficient in order to run at interactive simulation rates, and be able to support operations such as cuts and sutures. Mesh-based solutions, where the connections between the individual degrees of freedom (DoF) are defined explicitly, have been the traditional choice to approach these problems. However, when the problem under investigation contains a discontinuity that disrupts the connectivity between the DoFs, the underlying mesh structure has to be reconfigured in order to handle the newly introduced discontinuity correctly. This reconfiguration for mesh-based techniques is typically called dynamic remeshing, and most of the time it causes the performance bottleneck in the simulation. In this dissertation, the efficiency of point-based meshless methods is investigated for both constitutive modeling of elastic soft tissues and visualization of simulation objects, where arbitrary discontinuities/cuts are applied to the objects in the context of surgical simulation. The point-based deformable object modeling problem is examined in three functional aspects: modeling continuous elastic deformations with, handling discontinuities in, and visualizing a point-based object. Algorithmic and implementation details of the presented techniques are discussed in the dissertation. The presented point-based techniques are implemented as separate components and integrated into the open-source software framework SOFA. The presented meshless continuum mechanics model of elastic tissue were verified by comparing it to the Hertzian non-adhesive frictionless contact theory. Virtual experiments were setup with a point-based deformable block and a rigid indenter, and force-displacement curves obtained from the virtual experiments were compared to the theoretical solutions. The meshless mechanics model of soft tissue and the integrated novel discontinuity treatment technique discussed in this dissertation allows handling cuts of arbitrary shape. The implemented enrichment technique not only modifies the internal mechanics of the soft tissue model, but also updates the point-based visual representation in an efficient way preventing the use of costly dynamic remeshing operations

SOFA: A Multi-Model Framework for Interactive Physical Simulation

International audienceSOFA (Simulation Open Framework Architecture) is an open-source C++ library primarily targeted at interactive computational medical simulation. SOFA facilitates collaborations between specialists from various domains, by decomposing complex simulators into components designed independently and organized in a scenegraph data structure. Each component encapsulates one of the aspects of a simulation, such as the degrees of freedom, the forces and constraints, the differential equations, the main loop algorithms, the linear solvers, the collision detection algorithms or the interaction devices. The simulated objects can be represented using several models, each of them optimized for a different task such as the computation of internal forces, collision detection, haptics or visual display. These models are synchronized during the simulation using a mapping mechanism. CPU and GPU implementations can be transparently combined to exploit the computational power of modern hardware architectures. Thanks to this flexible yet efficient architecture, \sofa{} can be used as a test-bed to compare models and algorithms, or as a basis for the development of complex, high-performance simulators

Design and development of a VR system for exploration of medical data using haptic rendering and high quality visualization

[no abstract

Pre-operative Planning and Intra-operative Guidance for Shoulder Replacement Surgery

Shoulder joint replacement, or arthroplasty, is indicated in cases where arthritis or trauma has resulted in severe joint damage that in turn causes increased pain and decreased function. However, shoulder arthroplasty is less successful than hip and knee replacement, mostly due to the complexity of the shoulder joint and the resultant complexity of the replacement operation. In this paper we present a complete visualization-oriented pre-operative planning and intra-operative guidance approach for shoulder joint replacement. Our system assists the surgeon by allowing a virtual arthroplasty procedure whilst giving feedback, primarily via patient- and procedure-specific joint range of motion (ROM) simulation and visualization. After a successful planning, our system automatically generates a 3D model of a patient-specific mechanical guidance device that is then produced by a rapid prototyping machine and can be used during the operation. In this way, a computer-based guidance system is not required in the operating room

Woodification of polygonal meshes

An evolving polygonal mesh based on stem\u27s tree growth coupled with a physical simulation of bark\u27s cracking is presented. This process is denominated woodification. Whereas previous approaches use a fixed resolution voxel grid, woodification is built on the deformable simplicial complex representation, which robustly simulates growth with adaptive subdivision. The approach allows any meshed object to be grown and textured. Features, such as interaction with obstacles, attributes interpolation, and sketching tools, are added to provide control during the woodifible process

Design of decorative 3D models: from geodesic ornaments to tangible assemblies

L'obiettivo di questa tesi è sviluppare strumenti utili per creare opere d'arte decorative digitali in 3D. Uno dei processi decorativi più comunemente usati prevede la creazione di pattern decorativi, al fine di abbellire gli oggetti. Questi pattern possono essere dipinti sull'oggetto di base o realizzati con l'applicazione di piccoli elementi decorativi. Tuttavia, la loro realizzazione nei media digitali non è banale. Da un lato, gli utenti esperti possono eseguire manualmente la pittura delle texture o scolpire ogni decorazione, ma questo processo può richiedere ore per produrre un singolo pezzo e deve essere ripetuto da zero per ogni modello da decorare. D'altra parte, gli approcci automatici allo stato dell'arte si basano sull'approssimazione di questi processi con texturing basato su esempi o texturing procedurale, o con sistemi di riproiezione 3D. Tuttavia, questi approcci possono introdurre importanti limiti nei modelli utilizzabili e nella qualità dei risultati. Il nostro lavoro sfrutta invece i recenti progressi e miglioramenti delle prestazioni nel campo dell'elaborazione geometrica per creare modelli decorativi direttamente sulle superfici. Presentiamo una pipeline per i pattern 2D e una per quelli 3D, e dimostriamo come ognuna di esse possa ricreare una vasta gamma di risultati con minime modifiche dei parametri. Inoltre, studiamo la possibilità di creare modelli decorativi tangibili. I pattern 3D generati possono essere stampati in 3D e applicati a oggetti realmente esistenti precedentemente scansionati. Discutiamo anche la creazione di modelli con mattoncini da costruzione, e la possibilità di mescolare mattoncini standard e mattoncini custom stampati in 3D. Ciò consente una rappresentazione precisa indipendentemente da quanto la voxelizzazione sia approssimativa. I principali contributi di questa tesi sono l'implementazione di due diverse pipeline decorative, un approccio euristico alla costruzione con mattoncini e un dataset per testare quest'ultimo.The aim of this thesis is to develop effective tools to create digital decorative 3D artworks. Real-world art often involves the use of decorative patterns to enrich objects. These patterns can be painted on the base or might be realized with the application of small decorative elements. However, their creation in digital media is not trivial. On the one hand, users can manually perform texture paint or sculpt each decoration, in a process that can take hours to produce a single piece and needs to be repeated from the ground up for every model that needs to be decorated. On the other hand, automatic approaches in state of the art rely on approximating these processes with procedural or by-example texturing or with 3D reprojection. However, these approaches can introduce significant limitations in the models that can be used and in the quality of the results. Instead, our work exploits the recent advances and performance improvements in the geometry processing field to create decorative patterns directly on surfaces. We present a pipeline for 2D and one for 3D patterns and demonstrate how each of them can recreate a variety of results with minimal tweaking of the parameters. Furthermore, we investigate the possibility of creating decorative tangible models. The 3D patterns we generate can be 3D printed and applied to previously scanned real-world objects. We also discuss the creation of models with standard building bricks and the possibility of mixing standard and custom 3D-printed bricks. This allows for a precise representation regardless of the coarseness of the voxelization. The main contributions of this thesis are the implementation of two different decorative pipelines, a heuristic approach to brick construction, and a dataset to test the latter