Virtual and Augmented Reality Techniques for Minimally Invasive Cardiac Interventions: Concept, Design, Evaluation and Pre-clinical Implementation

While less invasive techniques have been employed for some procedures, most intracardiac interventions are still performed under cardiopulmonary bypass, on the drained, arrested heart. The progress toward off-pump intracardiac interventions has been hampered by the lack of adequate visualization inside the beating heart. This thesis describes the development, assessment, and pre-clinical implementation of a mixed reality environment that integrates pre-operative imaging and modeling with surgical tracking technologies and real-time ultrasound imaging. The intra-operative echo images are augmented with pre-operative representations of the cardiac anatomy and virtual models of the delivery instruments tracked in real time using magnetic tracking technologies. As a result, the otherwise context-less images can now be interpreted within the anatomical context provided by the anatomical models. The virtual models assist the user with the tool-to-target navigation, while real-time ultrasound ensures accurate positioning of the tool on target, providing the surgeon with sufficient information to ``see\u27\u27 and manipulate instruments in absence of direct vision. Several pre-clinical acute evaluation studies have been conducted in vivo on swine models to assess the feasibility of the proposed environment in a clinical context. Following direct access inside the beating heart using the UCI, the proposed mixed reality environment was used to provide the necessary visualization and navigation to position a prosthetic mitral valve on the the native annulus, or to place a repair patch on a created septal defect in vivo in porcine models. Following further development and seamless integration into the clinical workflow, we hope that the proposed mixed reality guidance environment may become a significant milestone toward enabling minimally invasive therapy on the beating heart

Preparing for the future of cardiothoracic surgery with virtual reality simulation and surgical planning:a narrative review

Background and Objective: Virtual reality (VR) technology in cardiothoracic surgery has been an area of interest for almost three decades, but computational limitations had restricted its implementation. Recent advances in computing power have facilitated the creation of high-fidelity VR simulations and anatomy visualisation tools. We undertook a non-systematic narrative review of literature on VR simulations and preoperative planning tools in cardiothoracic surgery and present the state-of-the-art, and a future outlook. Methods: A comprehensive search through MEDLINE database was performed in November 2022 for all publications that describe the use of VR in cardiothoracic surgery regarding training purposes, education, simulation, and procedural planning. We excluded papers that were not in English or Dutch, and that used two-dimensional (2D) screens, augmented, and simulated reality. Key Content and Findings: Results were categorised as simulators and preoperative planning tools. Current surgical simulators include the lobectomy module in the LapSim for video assisted thorascopic surgery which has been extensively validated, and the more recent robotic assisted lobectomy simulators from Robotix Mentor and Da Vinci SimNow, which are increasingly becoming integrated into the robotic surgery curriculum. Other perioperative simulators include the CardioPulmonary VR Resuscitation simulator for advanced life support after cardiac surgery, and the VR Extracorporeal Circulation (ECC) simulator for perfusionists to simulate the use of a heart-lung machine (HLM). For surgical planning, there are many small-scale tools available, and many case/pilot studies have been published utilising the visualisation possibilities provided by VR, including congenital cardiac, congenital thoracic, adult cardiac, and adult thoracic diseases. Conclusions: There are many promising tools becoming available to leverage the immersive power of VR in cardiothoracic surgery. The path to validate these simulators is well described, but large-scale trials producing high-level evidence for their efficacy are absent as of yet. Our view is that these tools will become increasingly integral parts of daily practice in this field in the coming decade.</p

Congenital TrainHeart: development of a fully 3D printed simulator for hands-on surgical training

RIASSUNTO Introduzione: Con la crescente aspettativa di un perfetto outcome per i pazienti sottoposti a interventi di cardiochirurgia, risulta fondamentale sviluppare e utilizzare nuove modalità per la formazione dei giovani chirurghi. Tuttavia, ad oggi, l’organizzazione di corsi di simulazione risulta dispendioso sia in termini di risorse economiche che di personale. Proprio per questo, il crescente interesse collettivo verso la stampa 3D ha permesso di sviluppare nuove tecnologie che possono essere efficacemente utilizzate nell’ambito della simulazione cardiochirurgica. Obiettivo dello studio: Questa tesi descrive lo sviluppo di un simulatore a basso costo stampato in 3D che può essere utilizzato sia nell’ambito delle cardiopatie congenite che di quelle acquisite Materiali e metodi: Il simulatore è stato sviluppato in modo tale da simulare posizione, visuale ed esposizione del cuore all’interno del torace in diversi approcci chirurgici. Tutte le componenti del simulatore sono state progettate tramite un software di modellazione 3D e stampati con stampante 3D a stereolitografia. I modellini da inserire all’interno dello stesso simulatore sono stati a loro volta sviluppati o tramite l’utilizzo dello stesso software o sfruttando tecniche di ricostruzione 3D a partenza da immagini TC o RMN. Risultati: Il simulatore si compone di una struttura che simula la cavità toracica con una apertura ellittica nella parte superiore atta a simulare una sternotomia mediana. Il simulatore può essere fissato ad un treppiede permettendo aggiustamenti per quanto concerne l’altezza, nonché movimenti di inclinazione e rotazione. In aggiunta, sono state realizzate quattro cover che permettono di modificare l’apertura sulla parte superiore del simulatore, al fine di simulare accessi di tipo mininvasivo. I modellini sono stati invece stampati con una resina elastica che, date le sue caratteristiche, può essere tagliata e suturata. Conclusioni: Il nuovo simulatore stampabile in 3D che è stato sviluppato potrebbe rappresentare uno strumento estremamente valido per le simulazioni cardiochirurgiche ad alta fedeltà e per il planning personalizzato di una procedura.Background: With the growing expectation of a perfect outcome for patients undergoing cardiac surgery, it is now imperative to find alternative surgical training methods for residents and fellows. However, surgical simulation usually requires a fair amount of funds and manpower to establish a reliable program. For this reason, the increasing interest in the 3D printing field allowed the development of new technologies that found immediate application in surgical simulation. Aim of the study: This thesis illustrates the development of a low-cost 3D printed simulator for congenital and acquired cardiac surgery. Materials and methods: A simulator was designed to replicate position, view, and exposure of the heart within the chest wall using different approaches (median sternotomy, mini-sternotomy, subaxillary, and posterior mini-thoracotomy). All components were designed using a 3D modeling software and printed using a stereolithography 3D printer. All models that come with the simulator were designed using the same CAD software used for the chest simulator or using 3D reconstruction software for CT or MRI scans. Results: The simulator consists of a chest wall cavity with an oval opening on the top simulating a median sternotomy. The simulator can be attached to a tripod, allowing for height adjustments and pitch-and-roll movements. In addition, five different covers were designed to modify the opening, thus allowing to replicate minimally-invasive surgical approaches. The fully printed design made it possible to significantly reduce the cost of the entire product. All models are printed with a special elastic resin which makes it possible to cut and suture all structures. Conclusion: A novel low-cost fully 3D printed simulator was developed. This may represent a valid tool for high fidelity simulation programs in congenital and acquired cardiac surgery in addition to a patient-specific surgical planning

Use of extended realities in cardiology

Recent miniaturization of electronic components and advances in image processing software have facilitated the entry of extended reality technology into clinical practice. In the last several years, the number of applications in cardiology has multiplied, with many promising to become standard of care. We review many of these applications in the areas of patient and physician education, cardiac rehabilitation, pre-procedural planning and intraprocedural use. The rapid integration of these approaches into the many facets of cardiology suggests that they will one day become an every-day part of physician practice

CathSim: An Open-source Simulator for Autonomous Cannulation

Autonomous robots in endovascular operations have the potential to navigate circulatory systems safely and reliably while decreasing the susceptibility to human errors. However, there are numerous challenges involved with the process of training such robots such as long training duration due to sample inefficiency of machine learning algorithms and safety issues arising from the interaction between the catheter and the endovascular phantom. Physics simulators have been used in the context of endovascular procedures, but they are typically employed for staff training and generally do not conform to the autonomous cannulation goal. Furthermore, most current simulators are closed-source which hinders the collaborative development of safe and reliable autonomous systems. In this work, we introduce CathSim, an open-source simulation environment that accelerates the development of machine learning algorithms for autonomous endovascular navigation. We first simulate the high-fidelity catheter and aorta with the state-of-the-art endovascular robot. We then provide the capability of real-time force sensing between the catheter and the aorta in the simulation environment. We validate our simulator by conducting two different catheterisation tasks within two primary arteries using two popular reinforcement learning algorithms, Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC). The experimental results show that using our open-source simulator, we can successfully train the reinforcement learning agents to perform different autonomous cannulation tasks

Clinical Application of Three-dimensional Printing and Extended Reality in Congenital Heart Disease

This PhD study investigates the clinical role of the two emerging techniques, which are 3D printing and virtual reality, to improve the visualisation and surgical planning of congenital heart disease. This research findings show that both of these technologies can enhance the users’ perception on the spatial relationship of the heart structures and defects, and therefore improving the management of congenital heart disease

Translating computational modelling tools for clinical practice in congenital heart disease

Increasingly large numbers of medical centres worldwide are equipped with the means to acquire 3D images of patients by utilising magnetic resonance (MR) or computed tomography (CT) scanners. The interpretation of patient 3D image data has significant implications on clinical decision-making and treatment planning. In their raw form, MR and CT images have become critical in routine practice. However, in congenital heart disease (CHD), lesions are often anatomically and physiologically complex. In many cases, 3D imaging alone can fail to provide conclusive information for the clinical team. In the past 20-30 years, several image-derived modelling applications have shown major advancements. Tools such as computational fluid dynamics (CFD) and virtual reality (VR) have successfully demonstrated valuable uses in the management of CHD. However, due to current software limitations, these applications have remained largely isolated to research settings, and have yet to become part of clinical practice. The overall aim of this project was to explore new routes for making conventional computational modelling software more accessible for CHD clinics. The first objective was to create an automatic and fast pipeline for performing vascular CFD simulations. By leveraging machine learning, a solution was built using synthetically generated aortic anatomies, and was seen to be able to predict 3D aortic pressure and velocity flow fields with comparable accuracy to conventional CFD. The second objective was to design a virtual reality (VR) application tailored for supporting the surgical planning and teaching of CHD. The solution was a Unity-based application which included numerous specialised tools, such as mesh-editing features and online networking for group learning. Overall, the outcomes of this ongoing project showed strong indications that the integration of VR and CFD into clinical settings is possible, and has potential for extending 3D imaging and supporting the diagnosis, management and teaching of CHD

Recent Applications of Three Dimensional Printing in Cardiovascular Medicine

Three dimensional (3D) printing, which consists in the conversion of digital images into a 3D physical model, is a promising and versatile field that, over the last decade, has experienced a rapid development in medicine. Cardiovascular medicine, in particular, is one of the fastest growing area for medical 3D printing. In this review, we firstly describe the major steps and the most common technologies used in the 3D printing process, then we present current applications of 3D printing with relevance to the cardiovascular field. The technology is more frequently used for the creation of anatomical 3D models useful for teaching, training, and procedural planning of complex surgical cases, as well as for facilitating communication with patients and their families. However, the most attractive and novel application of 3D printing in the last years is bioprinting, which holds the great potential to solve the ever-increasing crisis of organ shortage. In this review, we then present some of the 3D bioprinting strategies used for fabricating fully functional cardiovascular tissues, including myocardium, heart tissue patches, and heart valves. The implications of 3D bioprinting in drug discovery, development, and delivery systems are also briefly discussed, in terms of in vitro cardiovascular drug toxicity. Finally, we describe some applications of 3D printing in the development and testing of cardiovascular medical devices, and the current regulatory frameworks that apply to manufacturing and commercialization of 3D printed products

Virtual Reality Based Environment for Orthopedic Surgery (Veos)

The traditional way of teaching surgery involves students observing a �live� surgery and then gradually assisting experienced surgeons. The creation of a Virtual Reality environment for orthopedic surgery (VEOS) can be beneficial in improving the quality of training while decreasing the time needed for training. Developing such virtual environments for educational and training purposes can supplement existing approaches. In this research, the design and development of a virtual reality based environment for orthopedic surgery is described. The scope of the simulation environment is restricted to an orthopedic surgery process known as Less Invasive Stabilization System (LISS) surgery. The primary knowledge source for the LISS surgical process was Miguel A. Pirela-Cruz (Head of Orthopedic Surgery and Rehabilitation, Texas Tech University Health Sciences Center (TTHSC)). The VEOS was designed and developed on a PC based platform. The developed VEOS was validated through interactions with surgical residents at TTHSC. Feedback from residents and our collaborator Miguel A. Pirela-Cruz was used to make necessary modifications to the surgical environment.Industrial Engineering & Managemen