Advanced Visualization and Interaction Systems for Surgical Pre-operative Planning

The visualization of 3D models of the patient’s body emerges as a priority in surgery. In this paper two different visualization and interaction systems are presented: a virtual interface and a low cost multi-touch screen. The systems are able to interpret in real-time the user’s movements and can be used in the surgical pre-operative planning for the navigation and manipulation of 3D models of the human body built from CT images. The surgeon can visualize both the standard patient information, such as the CT image dataset, and the 3D model of the patient’s organs built from these images. The developed virtual interface is the first prototype of a system designed to avoid any contact with the computer so that the surgeon is able to visualize models of the patient’s organs and to interact with these, moving the finger in the free space. The multi-touch screen provides a custom user interface developed for doctors’ needs that allows users to interact, for surgical pre-operative planning purposes, both with the 3D model of the patient’s body built from medical images, and with the image dataset

A haptic-enabled multimodal interface for the planning of hip arthroplasty

Multimodal environments help fuse a diverse range of sensory modalities, which is particularly important when integrating the complex data involved in surgical preoperative planning. The authors apply a multimodal interface for preoperative planning of hip arthroplasty with a user interface that integrates immersive stereo displays and haptic modalities. This article overviews this multimodal application framework and discusses the benefits of incorporating the haptic modality in this area

Grid simulation services for the medical community

The first part of this paper presents a selection of medical simulation applications, including image reconstruction, near real-time registration for neuro-surgery, enhanced dose distribution calculation for radio-therapy, inhaled drug delivery prediction, plastic surgery planning and cardio-vascular system simulation. The latter two topics are discussed in some detail. In the second part, we show how such services can be made available to the clinical practitioner using Grid technology. We discuss the developments and experience made during the EU project GEMSS, which provides reliable, efficient, secure and lawful medical Grid services

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Computer- and robot-assisted Medical Intervention

Medical robotics includes assistive devices used by the physician in order to make his/her diagnostic or therapeutic practices easier and more efficient. This chapter focuses on such systems. It introduces the general field of Computer-Assisted Medical Interventions, its aims, its different components and describes the place of robots in that context. The evolutions in terms of general design and control paradigms in the development of medical robots are presented and issues specific to that application domain are discussed. A view of existing systems, on-going developments and future trends is given. A case-study is detailed. Other types of robotic help in the medical environment (such as for assisting a handicapped person, for rehabilitation of a patient or for replacement of some damaged/suppressed limbs or organs) are out of the scope of this chapter.Comment: Handbook of Automation, Shimon Nof (Ed.) (2009) 000-00

Retopologizing MRI and Diffusion Tensor Tractography Datasets for Real-time Interactivity

Current technology allows MRI and other patient data to be translated into voxel-based 3D models for the purpose of visualization. However, these voxel models are extremely complex and are not suitable for rapid real-time manipulation. For true “on-the-fly” interactivity, polygon-based models must be hand-built using other methods and imported into a game engine.This project develops an algorithm to translate complex datasets into optimized models for real-time interactivity without sacrificing accuracy of the original imaging modality. A working prototype, ready for integration into game engines, has been built with brain tumor data exported from OSIRIX1 and 3D Slicer2 via Mudbox3, retopologized in 3D Coat4 and re-imported to Maya5. White matter tracts detected by Diffusion Tensor Tractography are exported as volume models using 3D Slicer.The model has been integrated into the Unreal Development Kit (UDK)6 game engine to facilitate real-time interactivity across multiple platforms, including Mac, PC, Apple iOS, Google Android, Xbox 360, and SONY PlayStation. New techniques are being explored to automate and accelerate the process of retopologizing models. 1.Osirix – Advanced Open-Source PACS Workstation DICOM Viewer http://www.osirix-viewer.com/ 2. 3D Slicer- A multi-platform, free and open source software package for visualization and medical image computing http://www.3dslicer.org 3.MudBox – Autodesk¨ Mudbox™ 3D digital sculpting and digital painting software http://usa.autodesk. com 4.3D Coat – Retopologizing and 3D sculpting software http://3d-coat.com 5.MAYA - Autodesk¨Maya¨ 3D animation software delivers an end-to-end creative workflow with compression tools for animation, modeling, simulation, visual effects, rendering, matchmoving, and compositing on a highly extensible production platform http://usa.autodesk.com 6.Unreal Engine 3 – a complete development framework for PCs, iOS, Xbox 360¨, and PlayStation¨ 3, providing a vast array of core technologies, content creation tools and support infrastructure http://www. unrealengine.com