Deep Reinforcement Learning in Surgical Robotics: Enhancing the Automation Level

Surgical robotics is a rapidly evolving field that is transforming the landscape of surgeries. Surgical robots have been shown to enhance precision, minimize invasiveness, and alleviate surgeon fatigue. One promising area of research in surgical robotics is the use of reinforcement learning to enhance the automation level. Reinforcement learning is a type of machine learning that involves training an agent to make decisions based on rewards and punishments. This literature review aims to comprehensively analyze existing research on reinforcement learning in surgical robotics. The review identified various applications of reinforcement learning in surgical robotics, including pre-operative, intra-body, and percutaneous procedures, listed the typical studies, and compared their methodologies and results. The findings show that reinforcement learning has great potential to improve the autonomy of surgical robots. Reinforcement learning can teach robots to perform complex surgical tasks, such as suturing and tissue manipulation. It can also improve the accuracy and precision of surgical robots, making them more effective at performing surgeries

A Novel FEM-Based Numerical Solver for Interactive Catheter Simulation in Virtual Catheterization

Virtual reality-based simulators are very helpful for trainees to acquire the skills of manipulating catheters and guidewires during the vascular interventional surgeries. In the development of such a simulator, however, it is a great challenge to realistically model and simulate deformable catheters and guidewires in an interactive manner. We propose a novel method to simulate the motion of catheters or guidewires and their interactions with patients' vascular system. Our method is based on the principle of minimal total potential energy. We formulate the total potential energy in the vascular interventional circumstance by summing up the elastic energy deriving from the bending of the catheters or guidewires, the potential energy due to the deformation of vessel walls, and the work by the external forces. We propose a novel FEM-based approach to simulate the deformation of catheters and guidewires. The motion of catheters or the guidewires and their responses to every input from the interventionalist can be calculated globally. Experiments have been conducted to validate the feasibility of the proposed method, and the results demonstrate that our method can realistically simulate the complex behaviors of catheters and guidewires in an interactive manner

Context-aware learning for robot-assisted endovascular catheterization

Endovascular intervention has become a mainstream treatment of cardiovascular diseases. However, multiple challenges remain such as unwanted radiation exposures, limited two-dimensional image guidance, insufficient force perception and haptic cues. Fast evolving robot-assisted platforms improve the stability and accuracy of instrument manipulation. The master-slave system also removes radiation to the operator. However, the integration of robotic systems into the current surgical workflow is still debatable since repetitive, easy tasks have little value to be executed by the robotic teleoperation. Current systems offer very low autonomy, potential autonomous features could bring more benefits such as reduced cognitive workloads and human error, safer and more consistent instrument manipulation, ability to incorporate various medical imaging and sensing modalities. This research proposes frameworks for automated catheterisation with different machine learning-based algorithms, includes Learning-from-Demonstration, Reinforcement Learning, and Imitation Learning. Those frameworks focused on integrating context for tasks in the process of skill learning, hence achieving better adaptation to different situations and safer tool-tissue interactions. Furthermore, the autonomous feature was applied to next-generation, MR-safe robotic catheterisation platform. The results provide important insights into improving catheter navigation in the form of autonomous task planning, self-optimization with clinical relevant factors, and motivate the design of intelligent, intuitive, and collaborative robots under non-ionizing image modalities.Open Acces

Surgical Subtask Automation for Intraluminal Procedures using Deep Reinforcement Learning

Intraluminal procedures have opened up a new sub-field of minimally invasive surgery that use flexible instruments to navigate through complex luminal structures of the body, resulting in reduced invasiveness and improved patient benefits. One of the major challenges in this field is the accurate and precise control of the instrument inside the human body. Robotics has emerged as a promising solution to this problem. However, to achieve successful robotic intraluminal interventions, the control of the instrument needs to be automated to a large extent. The thesis first examines the state-of-the-art in intraluminal surgical robotics and identifies the key challenges in this field, which include the need for safe and effective tool manipulation, and the ability to adapt to unexpected changes in the luminal environment. To address these challenges, the thesis proposes several levels of autonomy that enable the robotic system to perform individual subtasks autonomously, while still allowing the surgeon to retain overall control of the procedure. The approach facilitates the development of specialized algorithms such as Deep Reinforcement Learning (DRL) for subtasks like navigation and tissue manipulation to produce robust surgical gestures. Additionally, the thesis proposes a safety framework that provides formal guarantees to prevent risky actions. The presented approaches are evaluated through a series of experiments using simulation and robotic platforms. The experiments demonstrate that subtask automation can improve the accuracy and efficiency of tool positioning and tissue manipulation, while also reducing the cognitive load on the surgeon. The results of this research have the potential to improve the reliability and safety of intraluminal surgical interventions, ultimately leading to better outcomes for patients and surgeons

Using the Fringe Field of MRI Scanner for the Navigation of Microguidewires in the Vascular System

Le traitement du cancer, la prévention des accidents vasculaires cérébraux et le diagnostic ou le traitement des maladies vasculaires périphériques sont tous des cas d'application d'interventions à base de cathéter par le biais d'un traitement invasif minimal. Cependant, la pratique du cathétérisme est généralement pratiquée manuellement et dépend fortement de l'expérience et des compétences de l'interventionniste. La robotisation du cathétérisme a été étudiée pour faciliter la procédure en augmentant les niveaux d’autonomie par rapport à cette pratique clinique. En ce qui concerne ce problème, un des problèmes concerne le placement super sélectif du cathéter dans les artères plus étroites nécessitant une miniaturisation de l'instrument cathéter / fil de guidage attaché. Un microguide qui fonctionne dans des vaisseaux sanguins étroits et tortueux subit différentes forces mécaniques telles que le frottement avec la paroi du vaisseau. Ces forces peuvent empêcher la progression de la pointe du fil de guidage dans les vaisseaux. Une méthode proposée consiste à appliquer une force de traction à la pointe du microguide pour diriger et insérer le dispositif tout en poussant l’instrument attaché à partir de l’autre extrémité n’est plus pratique, et à exploiter le gradient du champ de franges IRM surnommé Fringe Field Navigation (FFN ) est proposée comme solution pour assurer cet actionnement. Le concept de FFN repose sur le positionnement d'un patient sur six DOF dans le champ périphérique du scanner IRM afin de permettre un actionnement directionnel pour la navigation du fil-guide. Ce travail rend compte des développements requis pour la mise en oeuvre de la FFN et l’étude du potentiel et des possibilités qu’elle offre au cathétérisme, en veillant au renforcement de l’autonomie. La cartographie du champ de franges d'un scanner IRM 3T est effectuée et la structure du champ de franges en ce qui concerne son uniformité locale est examinée. Une méthode pour la navigation d'un fil de guidage le long d'un chemin vasculaire souhaité basée sur le positionnement robotique du patient à six DOF est développée. Des expériences de FFN guidées par rayons X in vitro et in vivo sur un modèle porcin sont effectuées pour naviguer dans un fil de guidage dans la multibifurcation et les vaisseaux étroits. Une caractéristique unique de FFN est le haut gradient du champ magnétique. Il est démontré in vitro et in vivo que cette force surmonte le problème de l'insertion d'un fil microguide dans des vaisseaux tortueux et étroits pour permettre de faire avancer le fil-guide avec une distale douce au-delà de la limite d'insertion manuelle. La robustesse de FFN contre les erreurs de positionnement du patient est étudiée en relation avec l'uniformité locale dans le champ périphérique. La force élevée du champ magnétique disponible dans le champ de franges IRM peut amener les matériaux magnétiques doux à son état de saturation. Ici, le concept d'utilisation d'un ressort est présenté comme une alternative vi déformable aux aimants permanents solides pour la pointe du fil-guide. La navigation d'un microguide avec une pointe de ressort en structure vasculaire complexe est également réalisée in vitro. L'autonomie de FFN en ce qui concerne la planification d'une procédure avec autonomie de tâche obtenue dans ce travail augmente le potentiel de FFN en automatisant certaines étapes d'une procédure. En conclusion, FFN pour naviguer dans les microguides dans la structure vasculaire complexe avec autonomie pour effectuer le positionnement du patient et contrôler l'insertion du fil de guidage - avec démonstration in vivo dans un modèle porcin - peut être considéré comme un nouvel outil robotique facilitant le cathétérisme vasculaire. tout en aidant à cibler les vaisseaux lointains dans le système vasculaire.----------ABSTRACT Treatment of cancer, prevention of stroke, and diagnosis or treatment of peripheral vascular diseases are all the cases of application of catheter-based interventions through a minimal-invasive treatment. However, performing catheterization is generally practiced manually, and it highly depends on the experience and the skills of the interventionist. Robotization of catheterization has been investigated to facilitate the procedure by increasing the levels of autonomy to this clinical practice. Regarding it, one issue is the super selective placement of the catheter in the narrower arteries that require miniaturization of the tethered catheter/guidewire instrument. A microguidewire that operates in narrow and tortuous blood vessels experiences different mechanical forces like friction with the vessel wall. These forces can prevent the advancement of the tip of the guidewire in the vessels. A proposed method is applying a pulling force at the tip of the microguidewire to steer and insert the device while pushing the tethered instrument from the other end is no longer practical, and exploiting the gradient of the MRI fringe field dubbed as Fringe Field Navigation (FFN) is proposed as a solution to provide this actuation. The concept of FFN is based on six DOF positioning of a patient in the fringe field of the MRI scanner to enable directional actuation for the navigation of the guidewire. This work reports on the required developments for implementing FFN and investigating the potential and the possibilities that FFN introduces to the catheterization, with attention to enhancing the autonomy. Mapping the fringe field of a 3T MRI scanner is performed, and the structure of the fringe field regarding its local uniformity is investigated. A method for the navigation of a guidewire along a desired vascular path based on six DOF robotic patient positioning is developed. In vitro and in vivo x-ray Guided FFN experiments on a swine model of are performed to navigate a guidewire in the multibifurcation and narrow vessels. A unique feature of FFN is the high gradient of the magnetic field. It is demonstrated in vitro and in vivo that this force overcomes the issue of insertion of a microguidewire in tortuous and narrow vessels to enable advancing the guidewire with a soft distal beyond the limit of manual insertion. Robustness of FFN against the error in the positioning of the patient is investigated in relation to the local uniformity in the fringe field. The high strength of the magnetic field available in MRI fringe field can bring soft magnetic materials to its saturation state. Here, the concept of using a spring is introduced as a deformable alternative to solid permanent magnets for the tip of the guidewire. Navigation of a microguidewire with a viii spring tip in complex vascular structure is also performed in vitro. The autonomy of FFN regarding planning a procedure with Task Autonomy achieved in this work enhances the potential of FFN by automatization of certain steps of a procedure. As a conclusion, FFN to navigate microguidewires in the complex vascular structure with autonomy in performing tasks of patient positioning and controlling the insertion of the guidewire – with in vivo demonstration in swine model – can be considered as a novel robotic tool for facilitating the vascular catheterization while helping to target remote vessels in the vascular system

Impact of the COVID-19 pandemic on interventional cardiology fellowship training in the New York metropolitan area: A perspective from the United States epicenter

© 2020 Wiley Periodicals, Inc. Background: The healthcare burden posed by the coronavirus disease 2019 (COVID-19) pandemic in the New York Metropolitan area has necessitated the postponement of elective procedures resulting in a marked reduction in cardiac catheterization laboratory (CCL) volumes with a potential to impact interventional cardiology (IC) fellowship training. Methods: We conducted a web-based survey sent electronically to 21 Accreditation Council for Graduate Medical Education accredited IC fellowship program directors (PDs) and their respective fellows. Results: Fourteen programs (67%) responded to the survey and all acknowledged a significant decrease in CCL procedural volumes. More than half of the PDs reported part of their CCL being converted to inpatient units and IC fellows being redeployed to COVID-19 related duties. More than two-thirds of PDs believed that the COVID-19 pandemic would have a moderate (57%) or severe (14%) adverse impact on IC fellowship training, and 21% of the PDs expected their current fellows\u27 average percutaneous coronary intervention (PCI) volume to be below 250. Of 25 IC fellow respondents, 95% expressed concern that the pandemic would have a moderate (72%) or severe (24%) adverse impact on their fellowship training, and nearly one-fourth of fellows reported performing fewer than 250 PCIs as of March 1st. Finally, roughly one-third of PDs and IC fellows felt that there should be consideration of an extension of fellowship training or a period of early career mentorship after fellowship. Conclusions: The COVID-19 pandemic has caused a significant reduction in CCL procedural volumes that is impacting IC fellowship training in the NY metropolitan area. These results should inform professional societies and accreditation bodies to offer tailored opportunities for remediation of affected trainees

3D Imaging for Planning of Minimally Invasive Surgical Procedures

Novel minimally invasive surgeries are used for treating cardiovascular diseases and are performed under 2D fluoroscopic guidance with a C-arm system. 3D multidetector row computed tomography (MDCT) images are routinely used for preprocedural planning and postprocedural follow-up. For preprocedural planning, the ability to integrate the MDCT with fluoroscopic images for intraprocedural guidance is of clinical interest. Registration may be facilitated by rotating the C-arm to acquire 3D C-arm CT images. This dissertation describes the development of optimal scan and contrast parameters for C-arm CT in 6 swine. A 5-s ungated C-arm CT acquisition during rapid ventricular pacing with aortic root injection using minimal contrast (36 mL), producing high attenuation (1226), few artifacts (2.0), and measurements similar to those from MDCT (p\u3e0.05) was determined optimal. 3D MDCT and C-arm CT images were registered to overlay the aortic structures from MDCT onto fluoroscopic images for guidance in placing the prosthesis. This work also describes the development of a methodology to develop power equation (R2\u3e0.998) for estimating dose with C-arm CT based on applied tube voltage. Application in 10 patients yielded 5.48┬▒177 2.02 mGy indicating minimal radiation burden. For postprocedural follow-up, combinations of non-contrast, arterial, venous single energy CT (SECT) scans are used to monitor patients at multiple time intervals resulting in high cumulative radiation dose. Employing a single dual-energy CT (DECT) scan to replace two SECT scans can reduce dose. This work focuses on evaluating the feasibility of DECT imaging in the arterial phase. The replacement of non-contrast and arterial SECT acquisitions with one arterial DECT acquisition in 30 patients allowed generation of virtual non-contrast (VNC) images with 31 dose savings. Aortic luminal attenuation in VNC (32┬▒177 2 HU) was similar to true non-contrast images (35┬▒177 4 HU) indicating presence of unattenuated blood. To improve discrimination between c

3D Imaging for Planning of Minimally Invasive Surgical Procedures

Novel minimally invasive surgeries are used for treating cardiovascular diseases and are performed under 2D fluoroscopic guidance with a C-arm system. 3D multidetector row computed tomography (MDCT) images are routinely used for preprocedural planning and postprocedural follow-up. For preprocedural planning, the ability to integrate the MDCT with fluoroscopic images for intraprocedural guidance is of clinical interest. Registration may be facilitated by rotating the C-arm to acquire 3D C-arm CT images. This dissertation describes the development of optimal scan and contrast parameters for C-arm CT in 6 swine. A 5-s ungated C-arm CT acquisition during rapid ventricular pacing with aortic root injection using minimal contrast (36 mL), producing high attenuation (1226), few artifacts (2.0), and measurements similar to those from MDCT (p\u3e0.05) was determined optimal. 3D MDCT and C-arm CT images were registered to overlay the aortic structures from MDCT onto fluoroscopic images for guidance in placing the prosthesis. This work also describes the development of a methodology to develop power equation (R2\u3e0.998) for estimating dose with C-arm CT based on applied tube voltage. Application in 10 patients yielded 5.48┬▒177 2.02 mGy indicating minimal radiation burden. For postprocedural follow-up, combinations of non-contrast, arterial, venous single energy CT (SECT) scans are used to monitor patients at multiple time intervals resulting in high cumulative radiation dose. Employing a single dual-energy CT (DECT) scan to replace two SECT scans can reduce dose. This work focuses on evaluating the feasibility of DECT imaging in the arterial phase. The replacement of non-contrast and arterial SECT acquisitions with one arterial DECT acquisition in 30 patients allowed generation of virtual non-contrast (VNC) images with 31 dose savings. Aortic luminal attenuation in VNC (32┬▒177 2 HU) was similar to true non-contrast images (35┬▒177 4 HU) indicating presence of unattenuated blood. To improve discrimination between c

MR fluoroscopy in vascular and cardiac interventions (review)

Vascular and cardiac disease remains a leading cause of morbidity and mortality in developed and emerging countries. Vascular and cardiac interventions require extensive fluoroscopic guidance to navigate endovascular catheters. X-ray fluoroscopy is considered the current modality for real time imaging. It provides excellent spatial and temporal resolution, but is limited by exposure of patients and staff to ionizing radiation, poor soft tissue characterization and lack of quantitative physiologic information. MR fluoroscopy has been introduced with substantial progress during the last decade. Clinical and experimental studies performed under MR fluoroscopy have indicated the suitability of this modality for: delivery of ASD closure, aortic valves, and endovascular stents (aortic, carotid, iliac, renal arteries, inferior vena cava). It aids in performing ablation, creation of hepatic shunts and local delivery of therapies. Development of more MR compatible equipment and devices will widen the applications of MR-guided procedures. At post-intervention, MR imaging aids in assessing the efficacy of therapies, success of interventions. It also provides information on vascular flow and cardiac morphology, function, perfusion and viability. MR fluoroscopy has the potential to form the basis for minimally invasive image–guided surgeries that offer improved patient management and cost effectiveness