The Role of Visualization, Force Feedback, and Augmented Reality in Minimally Invasive Heart Valve Repair

New cardiovascular techniques have been developed to address the unique requirements of high risk, elderly, surgical patients with heart valve disease by avoiding both sternotomy and cardiopulmonary bypass. However, these technologies pose new challenges in visualization, force application, and intracardiac navigation. Force feedback and augmented reality (AR) can be applied to minimally invasive mitral valve repair and transcatheter aortic valve implantation (TAVI) techniques to potentially surmount these challenges. Our study demonstrated shorter operative times with three dimensional (3D) visualization compared to two dimensional (2D) visualization; however, both experts and novices applied significantly more force to cardiac tissue during 3D robotics-assisted mitral valve annuloplasty than during conventional open mitral valve annuloplasty. This finding suggests that 3D visualization does not fully compensate for the absence of haptic feedback in robotics-assisted cardiac surgery. Subsequently, using an innovative robotics-assisted surgical system design, we determined that direct haptic feedback may improve both expert and trainee performance using robotics-assisted techniques. We determined that during robotics-assisted mitral valve annuloplasty the use of either visual or direct force feedback resulted in a significant decrease in forces applied to cardiac tissue when compared to robotics-assisted mitral valve annuloplasty without force feedback. We presented NeoNav, an AR-enhanced echocardiograpy intracardiac guidance system for NeoChord off-pump mitral valve repair. Our study demonstrated superior tool navigation accuracy, significantly shorter navigation times, and reduced potential for injury with AR enhanced intracardiac navigation for off-pump transapical mitral valve repair with neochordae implantation. In addition, we applied the NeoNav system as a safe and inexpensive alternative imaging modality for TAVI guidance. We found that our proposed AR guidance system may achieve similar or better results than the current standard of care, contrast enhanced fluoroscopy, while eliminating the use of nephrotoxic contrast and ionizing radiation. These results suggest that the addition of both force feedback and augmented reality image guidance can improve both surgical performance and safety during minimally invasive robotics assisted and beating heart valve surgery, respectively

Image fusion performed with noncontrast computed tomography scans during endovascular aneurysm repair

We report two endovascular aneurysm repair procedures achieved under image fusion guidance accomplished with noncontrast injected preoperative computed tomography scans. Such use of this advanced imaging application reduces contrast media injection volume (respectively, 27 and 24 mL throughout the patients' hospital course). No changes in creatinine clearance occurred after the procedures. Contrast-enhanced ultrasound imaging confirmed technical success in both cases

Evaluation of robotic catheter technology in complex endovascular intervention

The past four decades have witnessed tremendous strides in the evolution of endovascular devices and techniques. Catheter-based intervention has revolutionized the management of arterial disease allowing treatment of aortic and peripheral pathologies via a minimally invasive approach. Despite the exponential advances in endovascular equipment, devices and techniques, catheter-based endovascular intervention has certain morphological and technological constraints. Complex patient anatomy, technological impediments and suboptimal fluoroscopic imaging, can make endovascular intervention challenging using traditional endovascular means. Conventional endovascular catheters lack active manoeuvrability of the tip. Manual control can hinder overall stability and control at key target areas, leading to significantly prolonged overall procedure and fluoroscopic times. Repeated instrumentation increases the risk of vessel trauma and distal embolization. More importantly, guidewire-catheter skills are not necessarily intuitive but must be developed and are highly dependent on operator skill with long training pathways as a result. Recognizing the pressing need to address some of the limitations of standard catheter technology this thesis aims to evaluate the role of advanced robotic endovascular catheters in the aortic arch and the visceral segment. Clinical use of this technology is currently limited to transvenous cardiac mapping and ablation procedures. A comprehensive pre-clinical comparison and analysis of robotic versus manual catheter techniques is presented to reveal both their advantages and limitations, with particular emphasis on the potential of robotic catheter technology to reduce the manual skill required for complex tasks, improve stability at key target areas, reduce the risk of vessel trauma, embolization and radiation exposure, whilst improving overall operator performance. The worlds first clinical report of robot-assisted aortic aneurysm repair, a “proof - of - concept” resulting from this research, is also presented, and the potential for future advanced applications in order to increase the applicability of endovascular therapy to a larger cohort of patients discussed

3D shape instantiation for intra-operative navigation from a single 2D projection

Unlike traditional open surgery where surgeons can see the operation area clearly, in robot-assisted Minimally Invasive Surgery (MIS), a surgeon’s view of the region of interest is usually limited. Currently, 2D images from fluoroscopy, Magnetic Resonance Imaging (MRI), endoscopy or ultrasound are used for intra-operative guidance as real-time 3D volumetric acquisition is not always possible due to the acquisition speed or exposure constraints. 3D reconstruction, however, is key to navigation in complex in vivo geometries and can help resolve this issue. Novel 3D shape instantiation schemes are developed in this thesis, which can reconstruct the high-resolution 3D shape of a target from limited 2D views, especially a single 2D projection or slice. To achieve a complete and automatic 3D shape instantiation pipeline, segmentation schemes based on deep learning are also investigated. These include normalization schemes for training U-Nets and network architecture design of Atrous Convolutional Neural Networks (ACNNs). For U-Net normalization, four popular normalization methods are reviewed, then Instance-Layer Normalization (ILN) is proposed. It uses a sigmoid function to linearly weight the feature map after instance normalization and layer normalization, and cascades group normalization after the weighted feature map. Detailed validation results potentially demonstrate the practical advantages of the proposed ILN for effective and robust segmentation of different anatomies. For network architecture design in training Deep Convolutional Neural Networks (DCNNs), the newly proposed ACNN is compared to traditional U-Net where max-pooling and deconvolutional layers are essential. Only convolutional layers are used in the proposed ACNN with different atrous rates and it has been shown that the method is able to provide a fully-covered receptive field with a minimum number of atrous convolutional layers. ACNN enhances the robustness and generalizability of the analysis scheme by cascading multiple atrous blocks. Validation results have shown the proposed method achieves comparable results to the U-Net in terms of medical image segmentation, whilst reducing the trainable parameters, thus improving the convergence and real-time instantiation speed. For 3D shape instantiation of soft and deforming organs during MIS, Sparse Principle Component Analysis (SPCA) has been used to analyse a 3D Statistical Shape Model (SSM) and to determine the most informative scan plane. Synchronized 2D images are then scanned at the most informative scan plane and are expressed in a 2D SSM. Kernel Partial Least Square Regression (KPLSR) has been applied to learn the relationship between the 2D and 3D SSM. It has been shown that the KPLSR-learned model developed in this thesis is able to predict the intra-operative 3D target shape from a single 2D projection or slice, thus permitting real-time 3D navigation. Validation results have shown the intrinsic accuracy achieved and the potential clinical value of the technique. The proposed 3D shape instantiation scheme is further applied to intra-operative stent graft deployment for the robot-assisted treatment of aortic aneurysms. Mathematical modelling is first used to simulate the stent graft characteristics. This is then followed by the Robust Perspective-n-Point (RPnP) method to instantiate the 3D pose of fiducial markers of the graft. Here, Equally-weighted Focal U-Net is proposed with a cross-entropy and an additional focal loss function. Detailed validation has been performed on patient-specific stent grafts with an accuracy between 1-3mm. Finally, the relative merits and potential pitfalls of all the methods developed in this thesis are discussed, followed by potential future research directions and additional challenges that need to be tackled.Open Acces

Augmented Image-Guidance for Transcatheter Aortic Valve Implantation

The introduction of transcatheter aortic valve implantation (TAVI), an innovative stent-based technique for delivery of a bioprosthetic valve, has resulted in a paradigm shift in treatment options for elderly patients with aortic stenosis. While there have been major advancements in valve design and access routes, TAVI still relies largely on single-plane fluoroscopy for intraoperative navigation and guidance, which provides only gross imaging of anatomical structures. Inadequate imaging leading to suboptimal valve positioning contributes to many of the early complications experienced by TAVI patients, including valve embolism, coronary ostia obstruction, paravalvular leak, heart block, and secondary nephrotoxicity from contrast use. A potential method of providing improved image-guidance for TAVI is to combine the information derived from intra-operative fluoroscopy and TEE with pre-operative CT data. This would allow the 3D anatomy of the aortic root to be visualized along with real-time information about valve and prosthesis motion. The combined information can be visualized as a `merged\u27 image where the different imaging modalities are overlaid upon each other, or as an `augmented\u27 image, where the location of key target features identified on one image are displayed on a different imaging modality. This research develops image registration techniques to bring fluoroscopy, TEE, and CT models into a common coordinate frame with an image processing workflow that is compatible with the TAVI procedure. The techniques are designed to be fast enough to allow for real-time image fusion and visualization during the procedure, with an intra-procedural set-up requiring only a few minutes. TEE to fluoroscopy registration was achieved using a single-perspective TEE probe pose estimation technique. The alignment of CT and TEE images was achieved using custom-designed algorithms to extract aortic root contours from XPlane TEE images, and matching the shape of these contours to a CT-derived surface model. Registration accuracy was assessed on porcine and human images by identifying targets (such as guidewires or coronary ostia) on the different imaging modalities and measuring the correspondence of these targets after registration. The merged images demonstrated good visual alignment of aortic root structures, and quantitative assessment measured an accuracy of less than 1.5mm error for TEE-fluoroscopy registration and less than 6mm error for CT-TEE registration. These results suggest that the image processing techniques presented have potential for development into a clinical tool to guide TAVI. Such a tool could potentially reduce TAVI complications, reducing morbidity and mortality and allowing for a safer procedure

Endovascular aneurysm repair: prevention and treatment of complications

The introduction of endovascular aneurysm repair (EVAR) procedures has greatly influenced the treatment of abdominal aortic aneurysms (AAAs). At first, EVAR was introduced as an alternative treatment for patients that could not receive an open surgical AAA repair procedure. Nowadays, EVAR is preferred because of the lower 30-day mortality rates compared to open surgical repairs. However, the durability of EVAR is a major problem, EVAR is associated with graft-related complications and the percentage of reinterventions can be as high as 20%. Because more complex endovascular procedures are being performed, the number of early and late complications may increase. The goal of this research can be divided into two parts. The first was to investigate how new imaging techniques can help to prevent complications during and after treatment of obstructive and aneurysmatic aortoiliac diseases. This thesis shows that the use of 3D image fusion during EVAR procedures can lead to a reduction in the amount of contrast agent used and that new prototype software can be used to visualize small changes in apposition and position of the endograft postoperatively.The second part of this thesis was aimed at outcomes after EVAR. It was investigated how EndoAnchors need to be implanted in order to prevent and treat EVAR complications. The thesis furthermore analyses the risks and effects of the location and placement of EndoAnchors and the risks associated with late open conversion after EVAR

Apport de l'assistance par ordinateur lors de la pose d'endoprothèse aortique

The development of endovascular aortic procedures is growing. These mini-invasive techniques allow a reduction of surgical trauma, usually important in conventional open surgery. The technical limitations of endovascular repair are pushed to special aortic localizations which were in the past decade indication for open repair. Success and efficiency of such procedures are based on the development and the implementation of decision-making tools. This work aims to improve endovascular procedures thanks to a better utilization of pre and intraoperative imaging. This approach is in the line with the framework of computer-assisted surgery whose concepts are applied to vascular surgery. The optimization of endograft deployment is considered in three steps. The first part is dedicated to preoperative imaging analysis and shows the limits of the current sizing tools. The accuracy of a new measurement criterion is assessed (outer curvature length). The second part deals with intraoperative imaging and shows the contribution of augmented reality in endovascular aortic repair. In the last part, image guided surgery on soft tissues is addressed, especially the arterial deformations occurring during endovascular procedures which disprove rigid registration in fusion imaging. The use of finite element simulation to deal with this issue is presented. We report an original approach based on a predictive model of deformations using finite element simulation with geometrical and anatomo-mechanical patient specific parameters extracted from the preoperative CT-scan.Les techniques endovasculaires, particulièrement pour l’aorte, sont en plein essor en chirurgie vasculaire. Ces techniques mini-invasives permettent de diminuer l’agression chirurgicale habituellement importante lors de la chirurgie conventionnelle. Les limites techniques sont repoussées à certaines localisations de l’aorte qui étaient il y a encore peu de temps inaccessibles aux endoprothèses. Le succès et l’efficience de ces interventions reposent en partie sur l'élaboration et la mise en œuvre de nouveaux outils d'aide à la décision. Ce travail entend contribuer à l’amélioration des procédures interventionnelles aortiques grâce à une meilleure exploitation de l’imagerie pré et peropératoire. Cette démarche s’inscrit dans le cadre plus général des Gestes Médico-Chirurgicaux Assistés par Ordinateur, dont les concepts sont revisités pour les transposer au domaine de la chirurgie endovasculaire. Trois axes sont développés afin de sécuriser et optimiser la pose d'endoprothèse. Le premier est focalisé sur l’analyse préopératoire du scanner (sizing) et montre les limites des outils de mesure actuels et évalue la précision d’un nouveau critère de mesure des longueurs de l’aorte (courbure externe). Le deuxième axe se positionne sur le versant peropératoire et montre la contribution de la réalité augmentée dans la pose d’une endoprothèse aortique. Le troisième axe s’intéresse au problème plus général des interventions sur les tissus mous et particulièrement aux déformations artérielles qui surviennent au cours des procédures interventionnelles qui mettent en défaut le recalage rigide lors de la fusion d’images. Nous présentons une approche originale basée sur un modèle numérique de prédiction des déformations qui utilise la simulation par éléments finis en y intégrant des paramètres géométriques et anatomo-mécaniques spécifique-patient extraits du scanner préopératoire