Exploiting Temporal Image Information in Minimally Invasive Surgery

Minimally invasive procedures rely on medical imaging instead of the surgeons direct vision. While preoperative images can be used for surgical planning and navigation, once the surgeon arrives at the target site real-time intraoperative imaging is needed. However, acquiring and interpreting these images can be challenging and much of the rich temporal information present in these images is not visible. The goal of this thesis is to improve image guidance for minimally invasive surgery in two main areas. First, by showing how high-quality ultrasound video can be obtained by integrating an ultrasound transducer directly into delivery devices for beating heart valve surgery. Secondly, by extracting hidden temporal information through video processing methods to help the surgeon localize important anatomical structures. Prototypes of delivery tools, with integrated ultrasound imaging, were developed for both transcatheter aortic valve implantation and mitral valve repair. These tools provided an on-site view that shows the tool-tissue interactions during valve repair. Additionally, augmented reality environments were used to add more anatomical context that aids in navigation and in interpreting the on-site video. Other procedures can be improved by extracting hidden temporal information from the intraoperative video. In ultrasound guided epidural injections, dural pulsation provides a cue in finding a clear trajectory to the epidural space. By processing the video using extended Kalman filtering, subtle pulsations were automatically detected and visualized in real-time. A statistical framework for analyzing periodicity was developed based on dynamic linear modelling. In addition to detecting dural pulsation in lumbar spine ultrasound, this approach was used to image tissue perfusion in natural video and generate ventilation maps from free-breathing magnetic resonance imaging. A second statistical method, based on spectral analysis of pixel intensity values, allowed blood flow to be detected directly from high-frequency B-mode ultrasound video. Finally, pulsatile cues in endoscopic video were enhanced through Eulerian video magnification to help localize critical vasculature. This approach shows particular promise in identifying the basilar artery in endoscopic third ventriculostomy and the prostatic artery in nerve-sparing prostatectomy. A real-time implementation was developed which processed full-resolution stereoscopic video on the da Vinci Surgical System

INTERFACE DESIGN FOR A VIRTUAL REALITY-ENHANCED IMAGE-GUIDED SURGERY PLATFORM USING SURGEON-CONTROLLED VIEWING TECHNIQUES

Initiative has been taken to develop a VR-guided cardiac interface that will display and deliver information without affecting the surgeons’ natural workflow while yielding better accuracy and task completion time than the existing setup. This paper discusses the design process, the development of comparable user interface prototypes as well as an evaluation methodology that can measure user performance and workload for each of the suggested display concepts. User-based studies and expert recommendations are used in conjunction to es­ tablish design guidelines for our VR-guided surgical platform. As a result, a better understanding of autonomous view control, depth display, and use of virtual context, is attained. In addition, three proposed interfaces have been developed to allow a surgeon to control the view of the virtual environment intra-operatively. Comparative evaluation of the three implemented interface prototypes in a simulated surgical task scenario, revealed performance advantages for stereoscopic and monoscopic biplanar display conditions, as well as the differences between three types of control modalities. One particular interface prototype demonstrated significant improvement in task performance. Design recommendations are made for this interface as well as the others as we prepare for prospective development iterations

Medical Robotics

The first generation of surgical robots are already being installed in a number of operating rooms around the world. Robotics is being introduced to medicine because it allows for unprecedented control and precision of surgical instruments in minimally invasive procedures. So far, robots have been used to position an endoscope, perform gallbladder surgery and correct gastroesophogeal reflux and heartburn. The ultimate goal of the robotic surgery field is to design a robot that can be used to perform closed-chest, beating-heart surgery. The use of robotics in surgery will expand over the next decades without any doubt. Minimally Invasive Surgery (MIS) is a revolutionary approach in surgery. In MIS, the operation is performed with instruments and viewing equipment inserted into the body through small incisions created by the surgeon, in contrast to open surgery with large incisions. This minimizes surgical trauma and damage to healthy tissue, resulting in shorter patient recovery time. The aim of this book is to provide an overview of the state-of-art, to present new ideas, original results and practical experiences in this expanding area. Nevertheless, many chapters in the book concern advanced research on this growing area. The book provides critical analysis of clinical trials, assessment of the benefits and risks of the application of these technologies. This book is certainly a small sample of the research activity on Medical Robotics going on around the globe as you read it, but it surely covers a good deal of what has been done in the field recently, and as such it works as a valuable source for researchers interested in the involved subjects, whether they are currently “medical roboticists” or not

Estimating Target Vessel Location on Robot-Assisted CABG using Feature-based CT to US Registration

Although robot-assisted coronary artery bypass grafting (RA-CABG) has gained more acceptance worldwide, its success still depends on the surgeon’s experience and expertise, and the conversion rate to full sternotomy is in the order of 15%—25%. One of the reasons for conversion is poor pre-operative planning, which is based solely on pre-operative computed tomography (CT) images. This thesis proposes a technique to estimate the global peri-operative displacement of the heart and to predict the intra-operative target vessel location. The technique has been validated via both an in vitro and a clinical study, and predicted the position of the peri-operative target vessel location with ~ 3.5 mm RMS accuracy in the in vitro study while it yielded ~ 5.0 mm accuracy for the clinical validation. As the desired clinical accuracy imposed by this procedure is on the order of one intercostal space (10 - 15 mm), our technique suits the clinical requirements. It is therefore believed that this technique has the potential to improve the pre-operative planning by updating peri-operative migration patterns of the heart and, consequently, will lead to reduced conversion to conventional open thoracic procedures

Robotic Ultrasound Imaging: State-of-the-Art and Future Perspectives

Ultrasound (US) is one of the most widely used modalities for clinical intervention and diagnosis due to the merits of providing non-invasive, radiation-free, and real-time images. However, free-hand US examinations are highly operator-dependent. Robotic US System (RUSS) aims at overcoming this shortcoming by offering reproducibility, while also aiming at improving dexterity, and intelligent anatomy and disease-aware imaging. In addition to enhancing diagnostic outcomes, RUSS also holds the potential to provide medical interventions for populations suffering from the shortage of experienced sonographers. In this paper, we categorize RUSS as teleoperated or autonomous. Regarding teleoperated RUSS, we summarize their technical developments, and clinical evaluations, respectively. This survey then focuses on the review of recent work on autonomous robotic US imaging. We demonstrate that machine learning and artificial intelligence present the key techniques, which enable intelligent patient and process-specific, motion and deformation-aware robotic image acquisition. We also show that the research on artificial intelligence for autonomous RUSS has directed the research community toward understanding and modeling expert sonographers' semantic reasoning and action. Here, we call this process, the recovery of the "language of sonography". This side result of research on autonomous robotic US acquisitions could be considered as valuable and essential as the progress made in the robotic US examination itself. This article will provide both engineers and clinicians with a comprehensive understanding of RUSS by surveying underlying techniques.Comment: Accepted by Medical Image Analysi

Art and Medicine: A Collaborative Project Between Virginia Commonwealth University in Qatar and Weill Cornell Medicine in Qatar

Four faculty researchers, two from Virginia Commonwealth University in Qatar, and two from Weill Cornell Medicine in Qatar developed a one semester workshop-based course in Qatar exploring the connections between art and medicine in a contemporary context. Students (6 art / 6 medicine) were enrolled in the course. The course included presentations by clinicians, medical engineers, artists, computing engineers, an art historian, a graphic designer, a painter, and other experts from the fields of art, design, and medicine. To measure the student experience of interdisciplinarity, the faculty researchers employed a mixed methods approach involving psychometric tests and observational ethnography. Data instruments included pre- and post-course semi-structured audio interviews, pre-test / post-test psychometric instruments (Budner Scale and Torrance Tests of Creativity), observational field notes, self-reflective blogging, and videography. This book describes the course and the experience of the students. It also contains images of the interdisciplinary work they created for a culminating class exhibition. Finally, the book provides insight on how different fields in a Middle Eastern context can share critical /analytical thinking tools to refine their own professional practices

A minimally invasive surgical system for 3D ultrasound guided robotic retrieval of foreign bodies from a beating heart

The result of various medical conditions and trauma, foreign bodies in the heart pose a serious health risk as they may interfere with cardiovascular function. Particles such as thrombi, bullet fragments, and shrapnel can become trapped in a person's heart after migrating through the venous system, or by direct penetration. The severity of disruption can range from benign to fatal, with associated symptoms including anxiety, fever, cardiac tamponade, hemorrhage, infection, embolism, arrhythmia, and valve dysfunction. Injuries of this nature are common in both civilian and military populations. For symptomatic cases, conventional treatment is removal of the foreign body through open surgery via a median sternotomy, the use of cardiopulmonary bypass, and a wide incision in the heart muscle; these methods incur pronounced perioperative risks and long recovery periods. In order to improve upon the standard of care, we propose an image guided robotic system and a corresponding minimally invasive surgical approach. The system employs a dexterous robotic capture device that can maneuver inside the heart through a small incision. Visualization and guidance within the otherwise occluded internal regions are provided by 3D transesophageal echocardiography (TEE), an emerging form of intraoperative medical imaging used in interventions such as mitral valve repair and device implantation. A robotic approach, as opposed to a manual procedure using rigid instruments, is motivated by the various challenges inherent in minimally invasive surgery, which arise from attempts to perform skilled surgical tasks through small incisions without direct vision. Challenges include reduced dexterity, constrained workspace, limited visualization, and difficult hand-eye coordination, which ultimately lead to poor manipulability. A dexterous robotic end effector with real-time image guidance can help overcome these challenges and potentially improve surgical performance. However promising, such a system and approach require that several technical hurdles be resolved. The foreign body must be automatically tracked as it travels about the dynamic environment of the heart. The erratically moving particle must then be captured using a dexterous robot that moves much more slowly in comparison. Furthermore, retrieval must be performed under 3D ultrasound guidance, amidst the uncertainties presented by both the turbulent flow and by the imaging modality itself. In addressing such barriers, this thesis explores the development of a prototype system capable of retrieving a foreign body from a beating heart, culminating in a set of demonstrative in vitro experiments

Intracardiac Ultrasound Guided Systems for Transcatheter Cardiac Interventions

Transcatheter cardiac interventions are characterized by their percutaneous nature, increased patient safety, and low hospitalization times. Transcatheter procedures involve two major stages: navigation towards the target site and the positioning of tools to deliver the therapy, during which the interventionalists face the challenge of visualizing the anatomy and the relative position of the tools such as a guidewire. Fluoroscopic and transesophageal ultrasound (TEE) imaging are the most used techniques in cardiac procedures; however, they possess the disadvantage of radiation exposure and suboptimal imaging. This work explores the potential of intracardiac ultrasound (ICE) within an image guidance system (IGS) to facilitate the two stages of cardiac interventions. First, a novel 2.5D side-firing, conical Foresight ICE probe (Conavi Medical Inc., Toronto) is characterized, calibrated, and tracked using an electromagnetic sensor. The results indicate an acceptable tracking accuracy within some limitations. Next, an IGS is developed for navigating the vessels without fluoroscopy. A forward-looking, tracked ICE probe is used to reconstruct the vessel on a phantom which mimics the ultrasound imaging of an animal vena cava. Deep learning methods are employed to segment the complex vessel geometry from ICE imaging for the first time. The ICE-reconstructed vessel showed a clinically acceptable range of accuracy. Finally, a guidance system was developed to facilitate the positioning of tools during a tricuspid valve repair. The designed system potentially facilitates the positioning of the TriClip at the coaptation gap by pre-mapping the corresponding site of regurgitation in 3D tracking space