Virtual and Augmented Reality Techniques for Minimally Invasive Cardiac Interventions: Concept, Design, Evaluation and Pre-clinical Implementation

While less invasive techniques have been employed for some procedures, most intracardiac interventions are still performed under cardiopulmonary bypass, on the drained, arrested heart. The progress toward off-pump intracardiac interventions has been hampered by the lack of adequate visualization inside the beating heart. This thesis describes the development, assessment, and pre-clinical implementation of a mixed reality environment that integrates pre-operative imaging and modeling with surgical tracking technologies and real-time ultrasound imaging. The intra-operative echo images are augmented with pre-operative representations of the cardiac anatomy and virtual models of the delivery instruments tracked in real time using magnetic tracking technologies. As a result, the otherwise context-less images can now be interpreted within the anatomical context provided by the anatomical models. The virtual models assist the user with the tool-to-target navigation, while real-time ultrasound ensures accurate positioning of the tool on target, providing the surgeon with sufficient information to ``see\u27\u27 and manipulate instruments in absence of direct vision. Several pre-clinical acute evaluation studies have been conducted in vivo on swine models to assess the feasibility of the proposed environment in a clinical context. Following direct access inside the beating heart using the UCI, the proposed mixed reality environment was used to provide the necessary visualization and navigation to position a prosthetic mitral valve on the the native annulus, or to place a repair patch on a created septal defect in vivo in porcine models. Following further development and seamless integration into the clinical workflow, we hope that the proposed mixed reality guidance environment may become a significant milestone toward enabling minimally invasive therapy on the beating heart

Modelling mitral valvular dynamics–current trend and future directions

Dysfunction of mitral valve causes morbidity and premature mortality and remains a leading medical problem worldwide. Computational modelling aims to understand the biomechanics of human mitral valve and could lead to the development of new treatment, prevention and diagnosis of mitral valve diseases. Compared with the aortic valve, the mitral valve has been much less studied owing to its highly complex structure and strong interaction with the blood flow and the ventricles. However, the interest in mitral valve modelling is growing, and the sophistication level is increasing with the advanced development of computational technology and imaging tools. This review summarises the state-of-the-art modelling of the mitral valve, including static and dynamics models, models with fluid-structure interaction, and models with the left ventricle interaction. Challenges and future directions are also discussed

INTERFACE DESIGN FOR A VIRTUAL REALITY-ENHANCED IMAGE-GUIDED SURGERY PLATFORM USING SURGEON-CONTROLLED VIEWING TECHNIQUES

Initiative has been taken to develop a VR-guided cardiac interface that will display and deliver information without affecting the surgeons’ natural workflow while yielding better accuracy and task completion time than the existing setup. This paper discusses the design process, the development of comparable user interface prototypes as well as an evaluation methodology that can measure user performance and workload for each of the suggested display concepts. User-based studies and expert recommendations are used in conjunction to es­ tablish design guidelines for our VR-guided surgical platform. As a result, a better understanding of autonomous view control, depth display, and use of virtual context, is attained. In addition, three proposed interfaces have been developed to allow a surgeon to control the view of the virtual environment intra-operatively. Comparative evaluation of the three implemented interface prototypes in a simulated surgical task scenario, revealed performance advantages for stereoscopic and monoscopic biplanar display conditions, as well as the differences between three types of control modalities. One particular interface prototype demonstrated significant improvement in task performance. Design recommendations are made for this interface as well as the others as we prepare for prospective development iterations

NOVEL STRATEGIES FOR THE MORPHOLOGICAL AND BIOMECHANICAL ANALYSIS OF THE CARDIAC VALVES BASED ON VOLUMETRIC CLINICAL IMAGES

This work was focused on the morphological and biomechanical analysis of the heart valves exploiting the volumetric data. Novel methods were implemented to perform cardiac valve structure and sub-structure segmentation by defining long axis planes evenly rotated around the long axis of the valve. These methods were exploited to successfully reconstruct the 3D geometry of the mitral, tricuspid and aortic valve structures. Firstly, the reconstructed models were used for the morphological analysis providing a detailed description of the geometry of the valve structures, also computing novel indexes that could improve the description of the valvular apparatus and help their clinical assessment. Additionally, the models obtained for the mitral valve complex were adopted for the development of a novel biomechanical approach to simulate the systolic closure of the valve, relying on highly-efficient mass-spring models thus obtaining a good trade-off between the accuracy and the computational cost of the numerical simulations. In specific: \u2022 First, an innovative and semi-automated method was implemented to generate the 3D model of the aortic valve and of its calcifications, to quantitively describe its 3D morphology and to compute the anatomical aortic valve area (AVA) based on multi-detector computed tomography images. The comparison of the obtained results vs. effective AVA measurements showed a good correlation. Additionally, these methods accounted for asymmetries or anatomical derangements, which would be difficult to correctly capture through either effective AVA or planimetric AVA. \u2022 Second, a tool to quantitively assess the geometry of the tricuspid valve during the cardiac cycle using multidetector CT was developed, in particular focusing on the 3D spatial relationship between the tricuspid annulus and the right coronary artery. The morphological analysis of the annulus and leaflets confirmed data reported in literature. The qualitative and quantitative analysis of the spatial relationship could standardize the analysis protocol and be pivotal in the procedure planning of the percutaneous device implantation that interact with the tricuspid annulus. \u2022 Third, we simulated the systolic closure of three patient specific mitral valve models, derived from CMR datasets, by means of the mass spring model approach. The comparison of the obtained results vs. finite element analyses (considered as the gold-standard) was performed tuning the parameters of the mass spring model, so to obtain the best trade-off between computational expense and accuracy of the results. A configuration mismatch between the two models lower than two times the in-plane resolution of starting imaging data was yielded using a mass spring model set-up that requires, on average, only ten minutes to simulate the valve closure. \u2022 Finally, in the last chapter, we performed a comprehensive analysis which aimed at exploring the morphological and mechanical changes induced by the myxomatous pathologies in the mitral valve tissue. The analysis of mitral valve thickness confirmed the data and patterns reported in literature, while the mechanical test accurately described the behavior of the pathological tissue. A preliminary implementation of this data into finite element simulations suggested that the use of more reliable patient-specific and pathology-specific characterization of the model could improve the realism and the accuracy of the biomechanical simulations

Investigation and Validation of Imaging Techniques for Mitral Valve Disease Diagnosis and Intervention

Mitral Valve Disease (MVD) describes a variety of pathologies that result in regurgitation of blood during the systolic phase of the cardiac cycle. Decisions in valvular disease management rely heavily on non-invasive imaging. Transesophageal echocardiography (TEE) is widely recognized as the key evaluation technique where backflow of high velocity blood can be visualized under Doppler. In most cases, TEE imaging is adequate for identifying mitral valve pathology, though the modality is often limited from signal dropout, artifacts and a restricted field of view. Quantitative analysis is an integral part of the overall assessment of valve morphology and gives objective evidence for both classification and guiding intervention of regurgitation. In addition, patient-specific models derived from diagnostic TEE images allow clinicians to gain insight into uniquely intricate anatomy prior to surgery. However, the heavy reliance on TEE segmentation for diagnosis and modelling has necessitated an evaluation of the accuracy of the oft-used mitral valve imaging modality. Dynamic cardiac 4D-Computed Tomography (4D-CT) is emerging as a valuable tool for diagnosis, quantification and assessment of cardiac diseases. This modality has the potential to provide a high quality rendering of the mitral valve and subvalvular apparatus, to provide a more complete picture of the underlying morphology. However, application of dynamic CT to mitral valve imaging is especially challenging due to the large and rapid motion of the valve leaflets. It is therefore necessary to investigate the accuracy and level of precision by which dynamic CT captures mitral valve motion throughout the cardiac cycle. To do this, we design and construct a silicone and bovine quasi-static mitral valve phantom which can simulate a range of ECG-gated heart rates and reproduce physiologic valve motion over the cardiac cycle. In this study, we discovered that the dynamic CT accurately captures the underlying valve movement, but with a higher prevalence of image artifacts as leaflet and chordae motion increases due to elevated heart rates. In a subsequent study, we acquire simultaneous CT and TEE images of both a silicone mitral valve phantom and an iodine-stained bovine mitral valve. We propose a pipeline to use CT as the ground truth to study the relationship between TEE intensities and the underlying valve morphology. Preliminary results demonstrate that with an optimized threshold selection based solely on TEE pixel intensities, only 40\% of pixels are correctly classified as part of the valve. In addition, we have shown that emphasizing the centre-line rather than the boundaries of high intensity TEE image regions provides a better representation and segmentation of the valve morphology. This work has the potential to inform and augment the use of TEE for diagnosis and modelling of the mitral valve in the clinical workflow for MVD

Exploiting Temporal Image Information in Minimally Invasive Surgery

Minimally invasive procedures rely on medical imaging instead of the surgeons direct vision. While preoperative images can be used for surgical planning and navigation, once the surgeon arrives at the target site real-time intraoperative imaging is needed. However, acquiring and interpreting these images can be challenging and much of the rich temporal information present in these images is not visible. The goal of this thesis is to improve image guidance for minimally invasive surgery in two main areas. First, by showing how high-quality ultrasound video can be obtained by integrating an ultrasound transducer directly into delivery devices for beating heart valve surgery. Secondly, by extracting hidden temporal information through video processing methods to help the surgeon localize important anatomical structures. Prototypes of delivery tools, with integrated ultrasound imaging, were developed for both transcatheter aortic valve implantation and mitral valve repair. These tools provided an on-site view that shows the tool-tissue interactions during valve repair. Additionally, augmented reality environments were used to add more anatomical context that aids in navigation and in interpreting the on-site video. Other procedures can be improved by extracting hidden temporal information from the intraoperative video. In ultrasound guided epidural injections, dural pulsation provides a cue in finding a clear trajectory to the epidural space. By processing the video using extended Kalman filtering, subtle pulsations were automatically detected and visualized in real-time. A statistical framework for analyzing periodicity was developed based on dynamic linear modelling. In addition to detecting dural pulsation in lumbar spine ultrasound, this approach was used to image tissue perfusion in natural video and generate ventilation maps from free-breathing magnetic resonance imaging. A second statistical method, based on spectral analysis of pixel intensity values, allowed blood flow to be detected directly from high-frequency B-mode ultrasound video. Finally, pulsatile cues in endoscopic video were enhanced through Eulerian video magnification to help localize critical vasculature. This approach shows particular promise in identifying the basilar artery in endoscopic third ventriculostomy and the prostatic artery in nerve-sparing prostatectomy. A real-time implementation was developed which processed full-resolution stereoscopic video on the da Vinci Surgical System

Computer Vision Techniques for Transcatheter Intervention

Minimally invasive transcatheter technologies have demonstrated substantial promise for the diagnosis and treatment of cardiovascular diseases. For example, TAVI is an alternative to AVR for the treatment of severe aortic stenosis and TAFA is widely used for the treatment and cure of atrial fibrillation. In addition, catheter-based IVUS and OCT imaging of coronary arteries provides important information about the coronary lumen, wall and plaque characteristics. Qualitative and quantitative analysis of these cross-sectional image data will be beneficial for the evaluation and treatment of coronary artery diseases such as atherosclerosis. In all the phases (preoperative, intraoperative, and postoperative) during the transcatheter intervention procedure, computer vision techniques (e.g., image segmentation, motion tracking) have been largely applied in the field to accomplish tasks like annulus measurement, valve selection, catheter placement control, and vessel centerline extraction. This provides beneficial guidance for the clinicians in surgical planning, disease diagnosis, and treatment assessment. In this paper, we present a systematical review on these state-of-the-art methods.We aim to give a comprehensive overview for researchers in the area of computer vision on the subject of transcatheter intervention. Research in medical computing is multi-disciplinary due to its nature, and hence it is important to understand the application domain, clinical background, and imaging modality so that methods and quantitative measurements derived from analyzing the imaging data are appropriate and meaningful. We thus provide an overview on background information of transcatheter intervention procedures, as well as a review of the computer vision techniques and methodologies applied in this area

Functional and Biomechanical Effects of the Edge-to-Edge Repair in the Setting of Mitral Regurgitation: Consolidated Knowledge and Novel Tools to Gain Insight into Its Percutaneous Implementation

Mitral regurgitation is the most prevalent heart valve disease in the western population. When severe, it requires surgical treatment, repair being the preferred option. The edge-to-edge repair technique treats mitral regurgitation by suturing the leaflets together and creating a double-orifice valve. Due to its relative simplicity and versatility, it has become progressively more widespread. Recently, its percutaneous version has become feasible, and has raised interest thanks to the positive results of the Mitraclip(\uae) device. Edge-to-edge features and evolution have stimulated debate and multidisciplinary research by both clinicians and engineers. After providing an overview of representative studies in the field, here we propose a novel computational approach to the most recent percutaneous evolution of the edge-to-edge technique. Image-based structural finite element models of three mitral valves affected by posterior prolapse were derived from cine-cardiac magnetic resonance imaging. The models accounted for the patient-specific 3D geometry of the valve, including leaflet compound curvature pattern, patient-specific motion of annulus and papillary muscles, and hyperelastic and anisotropic mechanical properties of tissues. The biomechanics of the three valves throughout the entire cardiac cycle was simulated before and after Mitraclip(\uae) implantation, assessing the biomechanical impact of the procedure. For all three simulated MVs, Mitraclip(\uae) implantation significantly improved systolic leaflets coaptation, without inducing major alterations in systolic peak stresses. Diastolic orifice area was decreased, by up to 58.9%, and leaflets diastolic stresses became comparable, although lower, to systolic ones. Despite established knowledge on the edge-to-edge surgical repair, latest technological advances make its percutanoues implementation a challenging field of research. The modeling approach herein proposed may be expanded to analyze clinical scenarios that are currently critical for Mitraclip(\uae) implantation, helping the search for possible solutions

Towards Patient Specific Mitral Valve Modelling via Dynamic 3D Transesophageal Echocardiography

Mitral valve disease is a common pathologic problem occurring increasingly in an aging population, and many patients suffering from mitral valve disease require surgical intervention. Planning an interventional approach from diagnostic imaging alone remains a significant clinical challenge. Transesophageal echocardiography (TEE) is the primary imaging modality used diagnostically, it has limitations in image quality and field-of-view. Recently, developments have been made towards modelling patient-specific deformable mitral valves from TEE imaging, however, a major barrier to producing accurate valve models is the need to derive the leaflet geometry through segmentation of diagnostic TEE imaging. This work explores the development of volume compounding and automated image analysis to more accurately and quickly capture the relevant valve geometry needed to produce patient-specific mitral valve models. Volume compounding enables multiple ultrasound acquisitions from different orientations and locations to be aligned and blended to form a single volume with improved resolution and field-of-view. A series of overlapping transgastric views are acquired that are then registered together with the standard en-face image and are combined using a blending function. The resulting compounded ultrasound volumes allow the visualization of a wider range of anatomical features within the left heart, enhancing the capabilities of a standard TEE probe. In this thesis, I first describe a semi-automatic segmentation algorithm based on active contours designed to produce segmentations from end-diastole suitable for deriving 3D printable molds. Subsequently I describe the development of DeepMitral, a fully automatic segmentation pipeline which leverages deep learning to produce very accurate segmentations with a runtime of less than ten seconds. DeepMitral is the first reported method using convolutional neural networks (CNNs) on 3D TEE for mitral valve segmentations. The results demonstrate very accurate leaflet segmentations, and a reduction in the time and complexity to produce a patient-specific mitral valve replica. Finally, a real-time annulus tracking system using CNNs to predict the annulus coordinates in the spatial frequency domain was developed. This method facilitates the use of mitral annulus tracking in real-time guidance systems, and further simplifies mitral valve modelling through the automatic detection of the annulus, which is a key structure for valve quantification, and reproducing accurate leaflet dynamics