Image Guided Robotic Radical Prostatectomy

We present results of early trials of a system to overlay preoperative MRI onto endoscope video taken from a daVinci endoscope system during robotic radical prostatectomy. The endoscope is calibrated and tracked so that the MRI can be overlaid in the same coordinate system as the video data and projected onto the screen with the correct camera parameters. The system has two potential applications. The first is that it enables the surgeon to easily refer to the preoperative MRI during surgery. Secondly, it may serve as an initialisation for a model to video registration method to enable the preoperative data to be updated during the procedure. The system has been trialled in five patients, with overlay images provided to the surgeon during surgery in two cases. Further trials are ongoing

Image Guidance in Telemanipulator Assisted Urology Surgery

This thesis outlines the development of an image guided surgery system, intended for use in \davinci assisted radical prostatectomy but more generally applicable to laparoscopic urology surgery. We defined the key performance parameter of the system as the accuracy of overlaying modelled anatomy onto the surgical scene. This thesis is primarily concerned with determining the system accuracy based on an analysis of the system's components. A common error measure was defined for all system components. This is an on screen error (measured in pixels) based on the error in projecting a single point lying near the apex of the prostate with the endoscope in a typical surgical pose. In this case the projected point was approximately 200 mm from the endoscope lens. An intraoperative coordinate system is first defined as the coordinate system of an optical tracking system used to track the endoscope. The MRI image of the patient is transformed into the intraoperative coordinate system. Prior to surgery the endoscope is calibrated and during surgery the endoscope is tracked, defining a transform from the coordinates of the optical tracking system to the endoscope screen. This transform is used to project the MRI image onto the endoscope video display. The early part of the thesis describes a novel algorithm for registering MRI to ultrasound images of the bone which was used to put the MRI image into the intraoperative coordinate system. Using this algorithm avoids the need for fiducial markers. The table below shows the errors (as on screen pixel RMS) due to using this algorithm. An approximate value as RMS distance error at the prostate apex point is also included

Augmented Reality Assistance for Surgical Interventions using Optical See-Through Head-Mounted Displays

Augmented Reality (AR) offers an interactive user experience via enhancing the real world environment with computer-generated visual cues and other perceptual information. It has been applied to different applications, e.g. manufacturing, entertainment and healthcare, through different AR media. An Optical See-Through Head-Mounted Display (OST-HMD) is a specialized hardware for AR, where the computer-generated graphics can be overlaid directly onto the user's normal vision via optical combiners. Using OST-HMD for surgical intervention has many potential perceptual advantages. As a novel concept, many technical and clinical challenges exist for OST-HMD-based AR to be clinically useful, which motivates the work presented in this thesis. From the technical aspects, we first investigate the display calibration of OST-HMD, which is an indispensable procedure to create accurate AR overlay. We propose various methods to reduce the user-related error, improve robustness of the calibration, and remodel the calibration as a 3D-3D registration problem. Secondly, we devise methods and develop hardware prototype to increase the user's visual acuity of both real and virtual content through OST-HMD, to aid them in tasks that require high visual acuity, e.g. dental procedures. Thirdly, we investigate the occlusion caused by the OST-HMD hardware, which limits the user's peripheral vision. We propose to use alternative indicators to remind the user of unattended environment motion. From the clinical perspective, we identified many clinical use cases where OST-HMD-based AR is potentially helpful, developed applications integrated with current clinical systems, and conducted proof-of-concept evaluations. We first present a "virtual monitor'' for image-guided surgery. It can replace real radiology monitors in the operating room with easier user control and more flexibility in positioning. We evaluated the "virtual monitor'' for simulated percutaneous spine procedures. Secondly, we developed ARssist, an application for the bedside assistant in robotic surgery. The assistant can see the robotic instruments and endoscope within the patient body with ARssist. We evaluated the efficiency, safety and ergonomics of the assistant during two typical tasks: instrument insertion and manipulation. The performance for inexperienced users is significantly improved with ARssist, and for experienced users, the system significantly enhanced their confidence level. Lastly, we developed ARAMIS, which utilizes real-time 3D reconstruction and visualization to aid the laparoscopic surgeon. It demonstrates the concept of "X-ray see-through'' surgery. Our preliminary evaluation validated the application via a peg transfer task, and also showed significant improvement in hand-eye coordination. Overall, we have demonstrated that OST-HMD based AR application provides ergonomic improvements, e.g. hand-eye coordination. In challenging situations or for novice users, the improvements in ergonomic factors lead to improvement in task performance. With continuous effort as a community, optical see-through augmented reality technology will be a useful interventional aid in the near future