Real-time hybrid cutting with dynamic fluid visualization for virtual surgery

It is widely accepted that a reform in medical teaching must be made to meet today's high volume training requirements. Virtual simulation offers a potential method of providing such trainings and some current medical training simulations integrate haptic and visual feedback to enhance procedure learning. The purpose of this project is to explore the capability of Virtual Reality (VR) technology to develop a training simulator for surgical cutting and bleeding in a general surgery

Research on generic interactive deformable 3D models: focus on the human inguinal region

The goal of this project is to research for real-time approximate methods of physicallybased animation in conjunction with static polygonal meshes with the aim of deforming them and simulating an elastic behaviour for these meshes. Because of this, in this project it has been developed a software suite capable of doing a lot of tasks, each one from different computer graphics research fields, conforming a versatile capability project

Appearance Preserving Rendering of Out-of-Core Polygon and NURBS Models

In Computer Aided Design (CAD) trimmed NURBS surfaces are widely used due to their flexibility. For rendering and simulation however, piecewise linear representations of these objects are required. A relatively new field in CAD is the analysis of long-term strain tests. After such a test the object is scanned with a 3d laser scanner for further processing on a PC. In all these areas of CAD the number of primitives as well as their complexity has grown constantly in the recent years. This growth is exceeding the increase of processor speed and memory size by far and posing the need for fast out-of-core algorithms. This thesis describes a processing pipeline from the input data in the form of triangular or trimmed NURBS models until the interactive rendering of these models at high visual quality. After discussing the motivation for this work and introducing basic concepts on complex polygon and NURBS models, the second part of this thesis starts with a review of existing simplification and tessellation algorithms. Additionally, an improved stitching algorithm to generate a consistent model after tessellation of a trimmed NURBS model is presented. Since surfaces need to be modified interactively during the design phase, a novel trimmed NURBS rendering algorithm is presented. This algorithm removes the bottleneck of generating and transmitting a new tessellation to the graphics card after each modification of a surface by evaluating and trimming the surface on the GPU. To achieve high visual quality, the appearance of a surface can be preserved using texture mapping. Therefore, a texture mapping algorithm for trimmed NURBS surfaces is presented. To reduce the memory requirements for the textures, the algorithm is modified to generate compressed normal maps to preserve the shading of the original surface. Since texturing is only possible, when a parametric mapping of the surface - requiring additional memory - is available, a new simplification and tessellation error measure is introduced that preserves the appearance of the original surface by controlling the deviation of normal vectors. The preservation of normals and possibly other surface attributes allows interactive visualization for quality control applications (e.g. isophotes and reflection lines). In the last part out-of-core techniques for processing and rendering of gigabyte-sized polygonal and trimmed NURBS models are presented. Then the modifications necessary to support streaming of simplified geometry from a central server are discussed and finally and LOD selection algorithm to support interactive rendering of hard and soft shadows is described

VISIO-HAPTIC DEFORMABLE MODEL FOR HAPTIC DOMINANT PALPATION SIMULATOR

Vision and haptic are two most important modalities in a medical simulation. While visual cues assist one to see his actions when performing a medical procedure, haptic cues enable feeling the object being manipulated during the interaction. Despite their importance in a computer simulation, the combination of both modalities has not been adequately assessed, especially that in a haptic dominant environment. Thus, resulting in poor emphasis in resource allocation management in terms of effort spent in rendering the two modalities for simulators with realistic real-time interactions. Addressing this problem requires an investigation on whether a single modality (haptic) or a combination of both visual and haptic could be better for learning skills in a haptic dominant environment such as in a palpation simulator. However, before such an investigation could take place one main technical implementation issue in visio-haptic rendering needs to be addresse

Meshless Mechanics and Point-Based Visualization Methods for Surgical Simulations

Computer-based modeling and simulation practices have become an integral part of the medical education field. For surgical simulation applications, realistic constitutive modeling of soft tissue is considered to be one of the most challenging aspects of the problem, because biomechanical soft-tissue models need to reflect the correct elastic response, have to be efficient in order to run at interactive simulation rates, and be able to support operations such as cuts and sutures. Mesh-based solutions, where the connections between the individual degrees of freedom (DoF) are defined explicitly, have been the traditional choice to approach these problems. However, when the problem under investigation contains a discontinuity that disrupts the connectivity between the DoFs, the underlying mesh structure has to be reconfigured in order to handle the newly introduced discontinuity correctly. This reconfiguration for mesh-based techniques is typically called dynamic remeshing, and most of the time it causes the performance bottleneck in the simulation. In this dissertation, the efficiency of point-based meshless methods is investigated for both constitutive modeling of elastic soft tissues and visualization of simulation objects, where arbitrary discontinuities/cuts are applied to the objects in the context of surgical simulation. The point-based deformable object modeling problem is examined in three functional aspects: modeling continuous elastic deformations with, handling discontinuities in, and visualizing a point-based object. Algorithmic and implementation details of the presented techniques are discussed in the dissertation. The presented point-based techniques are implemented as separate components and integrated into the open-source software framework SOFA. The presented meshless continuum mechanics model of elastic tissue were verified by comparing it to the Hertzian non-adhesive frictionless contact theory. Virtual experiments were setup with a point-based deformable block and a rigid indenter, and force-displacement curves obtained from the virtual experiments were compared to the theoretical solutions. The meshless mechanics model of soft tissue and the integrated novel discontinuity treatment technique discussed in this dissertation allows handling cuts of arbitrary shape. The implemented enrichment technique not only modifies the internal mechanics of the soft tissue model, but also updates the point-based visual representation in an efficient way preventing the use of costly dynamic remeshing operations

Modelling and simulation of flexible instruments for minimally invasive surgical training in virtual reality

Improvements in quality and safety standards in surgical training, reduction in training hours and constant technological advances have challenged the traditional apprenticeship model to create a competent surgeon in a patient-safe way. As a result, pressure on training outside the operating room has increased. Interactive, computer based Virtual Reality (VR) simulators offer a safe, cost-effective, controllable and configurable training environment free from ethical and patient safety issues. Two prototype, yet fully-functional VR simulator systems for minimally invasive procedures relying on flexible instruments were developed and validated. NOViSE is the first force-feedback enabled VR simulator for Natural Orifice Transluminal Endoscopic Surgery (NOTES) training supporting a flexible endoscope. VCSim3 is a VR simulator for cardiovascular interventions using catheters and guidewires. The underlying mathematical model of flexible instruments in both simulator prototypes is based on an established theoretical framework – the Cosserat Theory of Elastic Rods. The efficient implementation of the Cosserat Rod model allows for an accurate, real-time simulation of instruments at haptic-interactive rates on an off-the-shelf computer. The behaviour of the virtual tools and its computational performance was evaluated using quantitative and qualitative measures. The instruments exhibited near sub-millimetre accuracy compared to their real counterparts. The proposed GPU implementation further accelerated their simulation performance by approximately an order of magnitude. The realism of the simulators was assessed by face, content and, in the case of NOViSE, construct validity studies. The results indicate good overall face and content validity of both simulators and of virtual instruments. NOViSE also demonstrated early signs of construct validity. VR simulation of flexible instruments in NOViSE and VCSim3 can contribute to surgical training and improve the educational experience without putting patients at risk, raising ethical issues or requiring expensive animal or cadaver facilities. Moreover, in the context of an innovative and experimental technique such as NOTES, NOViSE could potentially facilitate its development and contribute to its popularization by keeping practitioners up to date with this new minimally invasive technique.Open Acces

Simulating molecular docking with haptics

Intermolecular binding underlies various metabolic and regulatory processes of the cell, and the therapeutic and pharmacological properties of drugs. Molecular docking systems model and simulate these interactions in silico and allow the study of the binding process. In molecular docking, haptics enables the user to sense the interaction forces and intervene cognitively in the docking process. Haptics-assisted docking systems provide an immersive virtual docking environment where the user can interact with the molecules, feel the interaction forces using their sense of touch, identify visually the binding site, and guide the molecules to their binding pose. Despite a forty-year research e�ort however, the docking community has been slow to adopt this technology. Proprietary, unreleased software, expensive haptic hardware and limits on processing power are the main reasons for this. Another signi�cant factor is the size of the molecules simulated, limited to small molecules. The focus of the research described in this thesis is the development of an interactive haptics-assisted docking application that addresses the above issues, and enables the rigid docking of very large biomolecules and the study of the underlying interactions. Novel methods for computing the interaction forces of binding on the CPU and GPU, in real-time, have been developed. The force calculation methods proposed here overcome several computational limitations of previous approaches, such as precomputed force grids, and could potentially be used to model molecular exibility at haptic refresh rates. Methods for force scaling, multipoint collision response, and haptic navigation are also reported that address newfound issues, particular to the interactive docking of large systems, e.g. force stability at molecular collision. The i ii result is a haptics-assisted docking application, Haptimol RD, that runs on relatively inexpensive consumer level hardware, (i.e. there is no need for specialized/proprietary hardware)

Techniques for Realtime Viewing and Manipulation of Volumetric Data

Visualizing and manipulating volumetric data is a major component in many areas including anatomical registration in biomedical fields, seismic data analysis in the oil industry, machine part design in computer-aided geometric design, character animation in the movie industry, and fluid simulation. These industries have to meet the demands of the times and be able to make meaningful assertions about the data they generate. The shear size of this data presents many challenges to facilitating realtime interaction. In the recent decade, graphics hardware has become increasingly powerful and more sophisticated which has introduced a new realm of possibilities for processing volumetric data. This thesis focuses on a suite of techniques for viewing and editing volumetric data that efficiently use the processing power of central processing units (CPUs) as well as the large processing power of the graphics hardware (GPUs). This work begins with an algorithm to improve the efficiency of a texture-based volume rendering. We continue with a framework for performing realtime constructive solid geometry (CSG) with complex shapes and smoothing operations on watertight meshes based on a variation of Depth Peeling. We then move to an intuitive technique for deforming volumetric data using a collection of control points. Finally, we apply this technique to image registration of 3-dimensional computed tomography (CT) images used for lung cancel treatment, planning

VISIO-HAPTIC DEFORMABLE MODEL FOR HAPTIC DOMINANT PALPATION SIMULATOR

Vision and haptic are two most important modalities in a medical simulation. While visual cues assist one to see his actions when performing a medical procedure, haptic cues enable feeling the object being manipulated during the interaction. Despite their importance in a computer simulation, the combination of both modalities has not been adequately assessed, especially that in a haptic dominant environment. Thus, resulting in poor emphasis in resource allocation management in terms of effort spent in rendering the two modalities for simulators with realistic real-time interactions. Addressing this problem requires an investigation on whether a single modality (haptic) or a combination of both visual and haptic could be better for learning skills in a haptic dominant environment such as in a palpation simulator. However, before such an investigation could take place one main technical implementation issue in visio-haptic rendering needs to be addresse

Research on real-time physics-based deformation for haptic-enabled medical simulation

This study developed a multiple effective visuo-haptic surgical engine to handle a variety of surgical manipulations in real-time. Soft tissue models are based on biomechanical experiment and continuum mechanics for greater accuracy. Such models will increase the realism of future training systems and the VR/AR/MR implementations for the operating room