Exploiting Temporal Image Information in Minimally Invasive Surgery

Minimally invasive procedures rely on medical imaging instead of the surgeons direct vision. While preoperative images can be used for surgical planning and navigation, once the surgeon arrives at the target site real-time intraoperative imaging is needed. However, acquiring and interpreting these images can be challenging and much of the rich temporal information present in these images is not visible. The goal of this thesis is to improve image guidance for minimally invasive surgery in two main areas. First, by showing how high-quality ultrasound video can be obtained by integrating an ultrasound transducer directly into delivery devices for beating heart valve surgery. Secondly, by extracting hidden temporal information through video processing methods to help the surgeon localize important anatomical structures. Prototypes of delivery tools, with integrated ultrasound imaging, were developed for both transcatheter aortic valve implantation and mitral valve repair. These tools provided an on-site view that shows the tool-tissue interactions during valve repair. Additionally, augmented reality environments were used to add more anatomical context that aids in navigation and in interpreting the on-site video. Other procedures can be improved by extracting hidden temporal information from the intraoperative video. In ultrasound guided epidural injections, dural pulsation provides a cue in finding a clear trajectory to the epidural space. By processing the video using extended Kalman filtering, subtle pulsations were automatically detected and visualized in real-time. A statistical framework for analyzing periodicity was developed based on dynamic linear modelling. In addition to detecting dural pulsation in lumbar spine ultrasound, this approach was used to image tissue perfusion in natural video and generate ventilation maps from free-breathing magnetic resonance imaging. A second statistical method, based on spectral analysis of pixel intensity values, allowed blood flow to be detected directly from high-frequency B-mode ultrasound video. Finally, pulsatile cues in endoscopic video were enhanced through Eulerian video magnification to help localize critical vasculature. This approach shows particular promise in identifying the basilar artery in endoscopic third ventriculostomy and the prostatic artery in nerve-sparing prostatectomy. A real-time implementation was developed which processed full-resolution stereoscopic video on the da Vinci Surgical System

Augmented Image-Guidance for Transcatheter Aortic Valve Implantation

The introduction of transcatheter aortic valve implantation (TAVI), an innovative stent-based technique for delivery of a bioprosthetic valve, has resulted in a paradigm shift in treatment options for elderly patients with aortic stenosis. While there have been major advancements in valve design and access routes, TAVI still relies largely on single-plane fluoroscopy for intraoperative navigation and guidance, which provides only gross imaging of anatomical structures. Inadequate imaging leading to suboptimal valve positioning contributes to many of the early complications experienced by TAVI patients, including valve embolism, coronary ostia obstruction, paravalvular leak, heart block, and secondary nephrotoxicity from contrast use. A potential method of providing improved image-guidance for TAVI is to combine the information derived from intra-operative fluoroscopy and TEE with pre-operative CT data. This would allow the 3D anatomy of the aortic root to be visualized along with real-time information about valve and prosthesis motion. The combined information can be visualized as a `merged\u27 image where the different imaging modalities are overlaid upon each other, or as an `augmented\u27 image, where the location of key target features identified on one image are displayed on a different imaging modality. This research develops image registration techniques to bring fluoroscopy, TEE, and CT models into a common coordinate frame with an image processing workflow that is compatible with the TAVI procedure. The techniques are designed to be fast enough to allow for real-time image fusion and visualization during the procedure, with an intra-procedural set-up requiring only a few minutes. TEE to fluoroscopy registration was achieved using a single-perspective TEE probe pose estimation technique. The alignment of CT and TEE images was achieved using custom-designed algorithms to extract aortic root contours from XPlane TEE images, and matching the shape of these contours to a CT-derived surface model. Registration accuracy was assessed on porcine and human images by identifying targets (such as guidewires or coronary ostia) on the different imaging modalities and measuring the correspondence of these targets after registration. The merged images demonstrated good visual alignment of aortic root structures, and quantitative assessment measured an accuracy of less than 1.5mm error for TEE-fluoroscopy registration and less than 6mm error for CT-TEE registration. These results suggest that the image processing techniques presented have potential for development into a clinical tool to guide TAVI. Such a tool could potentially reduce TAVI complications, reducing morbidity and mortality and allowing for a safer procedure

Development of a Surgical Assistance System for Guiding Transcatheter Aortic Valve Implantation

Development of image-guided interventional systems is growing up rapidly in the recent years. These new systems become an essential part of the modern minimally invasive surgical procedures, especially for the cardiac surgery. Transcatheter aortic valve implantation (TAVI) is a recently developed surgical technique to treat severe aortic valve stenosis in elderly and high-risk patients. The placement of stented aortic valve prosthesis is crucial and typically performed under live 2D fluoroscopy guidance. To assist the placement of the prosthesis during the surgical procedure, a new fluoroscopy-based TAVI assistance system has been developed. The developed assistance system integrates a 3D geometrical aortic mesh model and anatomical valve landmarks with live 2D fluoroscopic images. The 3D aortic mesh model and landmarks are reconstructed from interventional angiographic and fluoroscopic C-arm CT system, and a target area of valve implantation is automatically estimated using these aortic mesh models. Based on template-based tracking approach, the overlay of visualized 3D aortic mesh model, landmarks and target area of implantation onto fluoroscopic images is updated by approximating the aortic root motion from a pigtail catheter motion without contrast agent. A rigid intensity-based registration method is also used to track continuously the aortic root motion in the presence of contrast agent. Moreover, the aortic valve prosthesis is tracked in fluoroscopic images to guide the surgeon to perform the appropriate placement of prosthesis into the estimated target area of implantation. An interactive graphical user interface for the surgeon is developed to initialize the system algorithms, control the visualization view of the guidance results, and correct manually overlay errors if needed. Retrospective experiments were carried out on several patient datasets from the clinical routine of the TAVI in a hybrid operating room. The maximum displacement errors were small for both the dynamic overlay of aortic mesh models and tracking the prosthesis, and within the clinically accepted ranges. High success rates of the developed assistance system were obtained for all tested patient datasets. The results show that the developed surgical assistance system provides a helpful tool for the surgeon by automatically defining the desired placement position of the prosthesis during the surgical procedure of the TAVI.Die Entwicklung bildgeführter interventioneller Systeme wächst rasant in den letzten Jahren. Diese neuen Systeme werden zunehmend ein wesentlicher Bestandteil der technischen Ausstattung bei modernen minimal-invasiven chirurgischen Eingriffen. Diese Entwicklung gilt besonders für die Herzchirurgie. Transkatheter Aortenklappen-Implantation (TAKI) ist eine neue entwickelte Operationstechnik zur Behandlung der schweren Aortenklappen-Stenose bei alten und Hochrisiko-Patienten. Die Platzierung der Aortenklappenprothese ist entscheidend und wird in der Regel unter live-2D-fluoroskopischen Bildgebung durchgeführt. Zur Unterstützung der Platzierung der Prothese während des chirurgischen Eingriffs wurde in dieser Arbeit ein neues Fluoroskopie-basiertes TAKI Assistenzsystem entwickelt. Das entwickelte Assistenzsystem überlagert eine 3D-Geometrie des Aorten-Netzmodells und anatomischen Landmarken auf live-2D-fluoroskopische Bilder. Das 3D-Aorten-Netzmodell und die Landmarken werden auf Basis der interventionellen Angiographie und Fluoroskopie mittels eines C-Arm-CT-Systems rekonstruiert. Unter Verwendung dieser Aorten-Netzmodelle wird das Zielgebiet der Klappen-Implantation automatisch geschätzt. Mit Hilfe eines auf Template Matching basierenden Tracking-Ansatzes wird die Überlagerung des visualisierten 3D-Aorten-Netzmodells, der berechneten Landmarken und der Zielbereich der Implantation auf fluoroskopischen Bildern korrekt überlagert. Eine kompensation der Aortenwurzelbewegung erfolgt durch Bewegungsverfolgung eines Pigtail-Katheters in Bildsequenzen ohne Kontrastmittel. Eine starrere Intensitätsbasierte Registrierungsmethode wurde verwendet, um kontinuierlich die Aortenwurzelbewegung in Bildsequenzen mit Kontrastmittelgabe zu detektieren. Die Aortenklappenprothese wird in die fluoroskopischen Bilder eingeblendet und dient dem Chirurg als Leitfaden für die richtige Platzierung der realen Prothese. Eine interaktive Benutzerschnittstelle für den Chirurg wurde zur Initialisierung der Systemsalgorithmen, zur Steuerung der Visualisierung und für manuelle Korrektur eventueller Überlagerungsfehler entwickelt. Retrospektive Experimente wurden an mehreren Patienten-Datensätze aus der klinischen Routine der TAKI in einem Hybrid-OP durchgeführt. Hohe Erfolgsraten des entwickelten Assistenzsystems wurden für alle getesteten Patienten-Datensätze erzielt. Die Ergebnisse zeigen, dass das entwickelte chirurgische Assistenzsystem ein hilfreiches Werkzeug für den Chirurg bei der Platzierung Position der Prothese während des chirurgischen Eingriffs der TAKI bietet

The Role of Visualization, Force Feedback, and Augmented Reality in Minimally Invasive Heart Valve Repair

New cardiovascular techniques have been developed to address the unique requirements of high risk, elderly, surgical patients with heart valve disease by avoiding both sternotomy and cardiopulmonary bypass. However, these technologies pose new challenges in visualization, force application, and intracardiac navigation. Force feedback and augmented reality (AR) can be applied to minimally invasive mitral valve repair and transcatheter aortic valve implantation (TAVI) techniques to potentially surmount these challenges. Our study demonstrated shorter operative times with three dimensional (3D) visualization compared to two dimensional (2D) visualization; however, both experts and novices applied significantly more force to cardiac tissue during 3D robotics-assisted mitral valve annuloplasty than during conventional open mitral valve annuloplasty. This finding suggests that 3D visualization does not fully compensate for the absence of haptic feedback in robotics-assisted cardiac surgery. Subsequently, using an innovative robotics-assisted surgical system design, we determined that direct haptic feedback may improve both expert and trainee performance using robotics-assisted techniques. We determined that during robotics-assisted mitral valve annuloplasty the use of either visual or direct force feedback resulted in a significant decrease in forces applied to cardiac tissue when compared to robotics-assisted mitral valve annuloplasty without force feedback. We presented NeoNav, an AR-enhanced echocardiograpy intracardiac guidance system for NeoChord off-pump mitral valve repair. Our study demonstrated superior tool navigation accuracy, significantly shorter navigation times, and reduced potential for injury with AR enhanced intracardiac navigation for off-pump transapical mitral valve repair with neochordae implantation. In addition, we applied the NeoNav system as a safe and inexpensive alternative imaging modality for TAVI guidance. We found that our proposed AR guidance system may achieve similar or better results than the current standard of care, contrast enhanced fluoroscopy, while eliminating the use of nephrotoxic contrast and ionizing radiation. These results suggest that the addition of both force feedback and augmented reality image guidance can improve both surgical performance and safety during minimally invasive robotics assisted and beating heart valve surgery, respectively

Evaluation of a Patient-Specific, Low-Cost, 3-Dimensional–Printed Transesophageal Echocardiography Human Heart Phantom

Simulation based education has been shown to increase the task-specific capability of medical trainees. Transesophageal echocardiography training greatly benefits from the use of simulators. They allow real time scanning of a beating heart and generation of ultrasound images side by side with anatomically accurate virtual model. These simulators are costly and have many limitations. 3D printing technologies have enabled the creation of bespoke phantoms capable of being used as task-trainers. This study aims to compare the ease of use and accuracy of a low-cost patient-specific, Computer-tomography based, 3D printed, echogenic TEE phantom compared to a commercially available echocardiography training mannequin. We hypothesized that a low-cost, 3D printed custom-made, cardiac phantom has comparable image quality, accuracy and usability as existing commercially available echocardiographic phantoms. After Institutional Ethic Research Board approval, we recruited ten American Board – Certified cardiac anesthesiologists and conducted a blinded comparative study divided into two stages. Stage one consisted of image assessment. A set of basic TEE views obtained from the 3D printed and commercial phantom were presented to the participants on a computer screen in random order. For each image, participants will be asked to identify the view, identify the quality of the image on a 1-5 Likert scale compared to the corresponding human view and guess with which phantom it was acquired (1 not at all realistic to patients view and 5 realistic to patients view). Stage two, participants will be asked to use the 3D printed and the commercially available phantom to obtain basic TEE views. In a maximum of 30 minutes. Each view was recorded and assessed for accuracy by two certified echocardiographers. Time needed to acquire each basic view and number of correct views was recorded. Overall usability of the phantoms was assessed through a questionnaire. For all continuous variables, we will calculate mean, median and standard deviation. We use Wilcoxon Signed-Rank test to assess significant differences in the rating of each phantom. All ten participants completed all part of the study. All participants could recognize all of the standard views. The average Likert scale was 3.2 for the 3D printed and 2.9 for the commercial Phantom with no significant difference. The average time to obtain views was 24.5 and 30 sec for the 3D printed and the commercial phantoms respectively statistically significantly in favor of the 3D printed phantom. The qualitative user assessment for ease to obtain the views, probe manipulation, image quality and overall experience were in great favor of the 3D printed phantom. Our Study suggest that the quality of TEE images obtained on the 3D printed phantom are not significantly different from those obtained on the commercial Phantom. The ease of use and time required to complete a basic TEE exam were in favor of the 3D Printed phantom.:Table of Content 1. Bibliographic Description 3 2. Introduction 4 2.1. Perioperative transesophageal echocardiography 4 2.2. Transesophageal echocardiography training 5 2.3. Transesophageal echocardiography simulation 6 2.4. 3D Heart Printing 13 2.5. 3D Segmentation 16 2.6. Development of the study phantom 17 2.7. Study Rationale 18 3. Publication 22 4. Summary 30 5. References 33 6. Appendices 37 6.1. Darstellung des eigenes Beitrags 38 6.2. Erklärung über die eigenständige Abfassung der Arbeit 39 6.3. Lebenslauf 40 6.4. Publikationen und Vorträge 44 6.5. Danksagung 61

SCEM+: Real-Time Robust Simultaneous Catheter and Environment Modeling for Endovascular Navigation

© 2016 IEEE. Endovascular procedures are characterised by significant challenges mainly due to the complexity in catheter control and navigation. Real-time recovery of the 3-D structure of the vasculature is necessary to visualise the interaction between the catheter and its surrounding environment to facilitate catheter manipulations. State-of-the-art intraoperative vessel reconstruction approaches are increasingly relying on nonionising imaging techniques such as optical coherence tomography (OCT) and intravascular ultrasound (IVUS). To enable accurate recovery of vessel structures and to deal with sensing errors and abrupt catheter motions, this letter presents a robust and real-time vessel reconstruction scheme for endovascular navigation based on IVUS and electromagnetic (EM) tracking. It is formulated as a nonlinear optimisation problem, which considers the uncertainty in both the IVUS contour and the EM pose, as well as vessel morphology provided by preoperative data. Detailed phantom validation is performed and the results demonstrate the potential clinical value of the technique

Interventional closure of persistent foetal cardiac shunts including PDA and PFO : study of outcome, complications and novel methods

Background: Persistent foramen ovale (PFO) and persistent ductus arteriosus (PDA) are two of the most common congenital heart defects (CHD). The incidence of PFO is reported to be between 10% and 35% (1, 2). In term infants, a PDA is seen in around one in 2000 births, accounting for 5% to 10% of all congenital heart disease (3, 4). Transcatheter closure of these lesions has become standard procedure in children and adults and has largely replaced surgery for these congenital cardiac defects. Cardiac catheterisation techniques require ionising radiation and are generally classified as high-radiation dose procedures according to the European Directive 2013/59/Euratom (5). New methods and materials have improved outcomes and safety, and have shortened periprocedural hospital stays for patients undergoing catheterisation. Aim: This thesis aimed to study outcome and complications of new methods in patients undergoing heart catheterisation. The included studies aimed to evaluate new techniques, starting with vascular access and preclosure devices in the accessed vessel, to improve bleeding control and facilitate same-day discharge (SDD). A new method for surveillance of cancer risk during catheterisation procedures was introduced and can be used to alert the operator when the radiation has exceeded a certain cancer risk level. Lastly, several next-generation PDA devices were studied. Methods: Four retrospective studies were conducted. In Study I, data from 238 paediatric patients were collected to estimate the risk of radiation-induced cancer death. Study II reviewed all patients undergoing PDA closure with an Amplatzer device over a fourteen-year period in a large centre in Tel Aviv. Study III investigated same-day discharge (SDD) of adult patients undergoing PFO closure. All patients who underwent transcatheter closure of a PFO at the Karolinska University Hospital in Stockholm between March 2017 and June 2020 were included. Study IV included all patients who underwent transcatheter PDA closure with a 5/7 Occlutech® duct occluder in three European centres in the UK, France and Sweden. Results: More than 90% of the retrospective study cohort in Study I was within the range of very low (1–10 in 100,000) or low cancer risk level (1–10 in 10,000). No patient exceeded the high cancer risk level (> 1 in 100). In addition, a new concept of age- and gender-specific risk reference values (RRVs) related to population cancer risk was introduced. The results showed that the RRV for males was a factor 2–3 higher than that for females. In Study II, all Amplatzer devices demonstrated very good closure rates (> 99.5%) with a low rate of complications, such as device embolisation or left pulmonary artery (LPA) stenosis. A tendency toward less LPA stenosis with the Piccolo™ device was noted, and no aortic flow disturbance occurred in this study. The majority of complications (device embolisation and LPA stenosis) occurred in patients with a bodyweight < 15 kg. Study III focused on SDD of patients undergoing percutaneous closure of PFO. A total of 246 of 262 patients (94%) had SDD. In 166 (63%) patients, a Perclose ProGlide™ system was used for femoral vein access closure. Post-interventional arrhythmias were noted in 17 (6%) of the patients, and vascular complications in nine patients (3%). There was no difference in SDD between patients who received ProGlide (n=159, 96%) and patients who did not receive ProGlide™ (n=87, 91%, p=0.10). Eighteen paediatric patients with heart failure were included retrospectively in three study sites in Study IV. Eleven of them had a bodyweight below 12 kg, and pulmonary hypertension was noted in seven of the 18 patients. All patients underwent successful PDA closure with no complications with a 5/7 Occlutech® duct occluder. Conclusions: Various aspects of cardiac catheterisation, from radiation risks, device choice, SDD and access site closure, have been studied in this thesis. Transcatheter closure of persistent foetal cardiac shunts is the standard treatment in full-term children and adults with low morbidity and mortality rates. New device development has improved PDA closure outcomes, even in small children with large PDAs. PFO closure has increased in the last five years due to several randomised controlled trials that catheterisations to treat CDH can be carried out with reasonably low radiation have reported a lower risk of recurrent ischemic stroke after PFO closure compared with medical therapy. Cardiac levels. Radiation-reducing tactics and the risk of radiation-induced cancer death must be taken into consideration, especially when treating younger patients

Dynamic Image Processing for Guidance of Off-pump Beating Heart Mitral Valve Repair

Compared to conventional open heart procedures, minimally invasive off-pump beating heart mitral valve repair aims to deliver equivalent treatment for mitral regurgitation with reduced trauma and side effects. However, minimally invasive approaches are often limited by the lack of a direct view to surgical targets and/or tools, a challenge that is compounded by potential movement of the target during the cardiac cycle. For this reason, sophisticated image guidance systems are required in achieving procedural efficiency and therapeutic success. The development of such guidance systems is associated with many challenges. For example, the system should be able to provide high quality visualization of both cardiac anatomy and motion, as well as augmenting it with virtual models of tracked tools and targets. It should have the capability of integrating pre-operative images to the intra-operative scenario through registration techniques. The computation speed must be sufficiently fast to capture the rapid cardiac motion. Meanwhile, the system should be cost effective and easily integrated into standard clinical workflow. This thesis develops image processing techniques to address these challenges, aiming to achieve a safe and efficient guidance system for off-pump beating heart mitral valve repair. These techniques can be divided into two categories, using 3D and 2D image data respectively. When 3D images are accessible, a rapid multi-modal registration approach is proposed to link the pre-operative CT images to the intra-operative ultrasound images. The ultrasound images are used to display the real time cardiac motion, enhanced by CT data serving as high quality 3D context with annotated features. I also developed a method to generate synthetic dynamic CT images, aiming to replace real dynamic CT data in such a guidance system to reduce the radiation dose applied to the patients. When only 2D images are available, an approach is developed to track the feature of interest, i.e. the mitral annulus, based on bi-plane ultrasound images and a magnetic tracking system. The concept of modern GPU-based parallel computing is employed in most of these approaches to accelerate the computation in order to capture the rapid cardiac motion with desired accuracy. Validation experiments were performed on phantom, animal and human data. The overall accuracy of registration and feature tracking with respect to the mitral annulus was about 2-3mm with computation time of 60-400ms per frame, sufficient for one update per cardiac cycle. It was also demonstrated in the results that the synthetic CT images can provide very similar anatomical representations and registration accuracy compared to that of the real dynamic CT images. These results suggest that the approaches developed in the thesis have good potential for a safer and more effective guidance system for off-pump beating heart mitral valve repair

3D Localization of Vena Contracta using Doppler ICE Imaging in Tricuspid Valve Interventions

PURPOSE: Tricuspid valve (TV) interventions face the challenge of imaging the anatomy and tools because of the ‘TEE-unfriendly’ nature of the TV. In edge-to-edge TV repair, a core step is to position the clip perpendicular to the coaptation gap. In this study, we provide a semi-automated method to localize the VC from Doppler intracardiac echo (ICE) imaging in a tracked 3D space, thus providing a pre-mapped location of the coaptation gap to assist device positioning. METHODS: A magnetically tracked ICE probe with Doppler imaging capabilities is employed in this study for imaging three patient-specific TVs placed in a pulsatile heart phantom. For each of the valves, the ICE probe is positioned to image the maximum regurgitant flow for five cardiac cycles. An algorithm then extracts the regurgitation imaging and computes the exact location of the vena contracta on the image. RESULTS: Across the three pathological, patient-specific valves, the average distance error between the detected VC and the ground truth model is [Formula: see text] mm. For each of the valves, one case represented the outlier where the algorithm misidentified the vena contracta to be near the annulus. In such cases, it is recommended to retake the five-second imaging data. CONCLUSION: This study presented a method for ultrasound-based localization of vena contracta in 3D space. Mapping such anatomical landmarks has the potential to assist with device positioning and to simplify tricuspid valve interventions by providing more contextual information to the interventionalists, thus enhancing their spatial awareness. Additionally, ICE can be used to provide live US and Doppler imaging of the complex TV anatomy throughout the procedure