Robot Autonomy for Surgery

Autonomous surgery involves having surgical tasks performed by a robot operating under its own will, with partial or no human involvement. There are several important advantages of automation in surgery, which include increasing precision of care due to sub-millimeter robot control, real-time utilization of biosignals for interventional care, improvements to surgical efficiency and execution, and computer-aided guidance under various medical imaging and sensing modalities. While these methods may displace some tasks of surgical teams and individual surgeons, they also present new capabilities in interventions that are too difficult or go beyond the skills of a human. In this chapter, we provide an overview of robot autonomy in commercial use and in research, and present some of the challenges faced in developing autonomous surgical robots

Surgical Subtask Automation for Intraluminal Procedures using Deep Reinforcement Learning

Intraluminal procedures have opened up a new sub-field of minimally invasive surgery that use flexible instruments to navigate through complex luminal structures of the body, resulting in reduced invasiveness and improved patient benefits. One of the major challenges in this field is the accurate and precise control of the instrument inside the human body. Robotics has emerged as a promising solution to this problem. However, to achieve successful robotic intraluminal interventions, the control of the instrument needs to be automated to a large extent. The thesis first examines the state-of-the-art in intraluminal surgical robotics and identifies the key challenges in this field, which include the need for safe and effective tool manipulation, and the ability to adapt to unexpected changes in the luminal environment. To address these challenges, the thesis proposes several levels of autonomy that enable the robotic system to perform individual subtasks autonomously, while still allowing the surgeon to retain overall control of the procedure. The approach facilitates the development of specialized algorithms such as Deep Reinforcement Learning (DRL) for subtasks like navigation and tissue manipulation to produce robust surgical gestures. Additionally, the thesis proposes a safety framework that provides formal guarantees to prevent risky actions. The presented approaches are evaluated through a series of experiments using simulation and robotic platforms. The experiments demonstrate that subtask automation can improve the accuracy and efficiency of tool positioning and tissue manipulation, while also reducing the cognitive load on the surgeon. The results of this research have the potential to improve the reliability and safety of intraluminal surgical interventions, ultimately leading to better outcomes for patients and surgeons

Context-aware learning for robot-assisted endovascular catheterization

Endovascular intervention has become a mainstream treatment of cardiovascular diseases. However, multiple challenges remain such as unwanted radiation exposures, limited two-dimensional image guidance, insufficient force perception and haptic cues. Fast evolving robot-assisted platforms improve the stability and accuracy of instrument manipulation. The master-slave system also removes radiation to the operator. However, the integration of robotic systems into the current surgical workflow is still debatable since repetitive, easy tasks have little value to be executed by the robotic teleoperation. Current systems offer very low autonomy, potential autonomous features could bring more benefits such as reduced cognitive workloads and human error, safer and more consistent instrument manipulation, ability to incorporate various medical imaging and sensing modalities. This research proposes frameworks for automated catheterisation with different machine learning-based algorithms, includes Learning-from-Demonstration, Reinforcement Learning, and Imitation Learning. Those frameworks focused on integrating context for tasks in the process of skill learning, hence achieving better adaptation to different situations and safer tool-tissue interactions. Furthermore, the autonomous feature was applied to next-generation, MR-safe robotic catheterisation platform. The results provide important insights into improving catheter navigation in the form of autonomous task planning, self-optimization with clinical relevant factors, and motivate the design of intelligent, intuitive, and collaborative robots under non-ionizing image modalities.Open Acces

A Multi-Robot Cooperation Framework for Sewing Personalized Stent Grafts

This paper presents a multi-robot system for manufacturing personalized medical stent grafts. The proposed system adopts a modular design, which includes: a (personalized) mandrel module, a bimanual sewing module, and a vision module. The mandrel module incorporates the personalized geometry of patients, while the bimanual sewing module adopts a learning-by-demonstration approach to transfer human hand-sewing skills to the robots. The human demonstrations were firstly observed by the vision module and then encoded using a statistical model to generate the reference motion trajectories. During autonomous robot sewing, the vision module plays the role of coordinating multi-robot collaboration. Experiment results show that the robots can adapt to generalized stent designs. The proposed system can also be used for other manipulation tasks, especially for flexible production of customized products and where bimanual or multi-robot cooperation is required.Comment: 10 pages, 12 figures, accepted by IEEE Transactions on Industrial Informatics, Key words: modularity, medical device customization, multi-robot system, robot learning, visual servoing, robot sewin

A Multi-Robot Cooperation Framework for Sewing Personalized Stent Grafts

This paper presents a multi-robot system for manufacturing personalized medical stent grafts. The proposed system adopts a modular design, which includes: a (personalized) mandrel module, a bimanual sewing module, and a vision module. The mandrel module incorporates the personalized geometry of patients, while the bimanual sewing module adopts a learning-by-demonstration approach to transfer human hand-sewing skills to the robots. The human demonstrations were firstly observed by the vision module and then encoded using a statistical model to generate the reference motion trajectories. During autonomous robot sewing, the vision module plays the role of coordinating multi-robot collaboration. Experiment results show that the robots can adapt to generalized stent designs. The proposed system can also be used for other manipulation tasks, especially for flexible production of customized products and where bimanual or multi-robot cooperation is required.Comment: 10 pages, 12 figures, accepted by IEEE Transactions on Industrial Informatics, Key words: modularity, medical device customization, multi-robot system, robot learning, visual servoing, robot sewin

Artificial intelligence and automation in valvular heart diseases

Artificial intelligence (AI) is gradually changing every aspect of social life, and healthcare is no exception. The clinical procedures that were supposed to, and could previously only be handled by human experts can now be carried out by machines in a more accurate and efficient way. The coming era of big data and the advent of supercomputers provides great opportunities to the development of AI technology for the enhancement of diagnosis and clinical decision-making. This review provides an introduction to AI and highlights its applications in the clinical flow of diagnosing and treating valvular heart diseases (VHDs). More specifically, this review first introduces some key concepts and subareas in AI. Secondly, it discusses the application of AI in heart sound auscultation and medical image analysis for assistance in diagnosing VHDs. Thirdly, it introduces using AI algorithms to identify risk factors and predict mortality of cardiac surgery. This review also describes the state-of-the-art autonomous surgical robots and their roles in cardiac surgery and intervention

Automated NDT inspection for large and complex geometries of composite materials

Large components with complex geometries, made of composite materials, have become very common in modern structures. To cope with future demand projections, it is necessary to overcome the current non-destructive testing (NDT) bottlenecks encountered during the inspection phase of manufacture. This thesis investigates several aspects of the introduction of automation within the inspection process of complex parts. The use of six-axis robots for product inspection and non-destructive testing systems is the central investigation of this thesis. The challenges embraced by the research include the development of a novel controlling approach for robotic manipulators and of novel path-planning strategies. The integration of robot manipulators and NDT data acquisition instruments is optimized. An effective and reliable way to encode the NDT data through the interpolated robot feedback positions is implemented. The viability of the new external control method is evaluated experimentally. The observed maximum position and orientation errors are respectively within 2mm and within 1 degree, over an operating envelope of 3m³. A new software toolbox (RoboNDT), aimed at NDT technicians, has been developed during this work. RoboNDT is intended to transform the robot path-planning problem into an easy step of the inspection process. The software incorporates the novel path-planning algorithms developed during this research and is shaped to overcome practical limitations of current OLP software. The software has been experimentally validated using scans on real high value aerospace components. RoboNDT delivers tool-path errors that are lower than the errors given by commercial off-line path-planning software. For example the variability of the standoff is within 10 mm for the tool-paths created with the commercial software and within 4.5 mm for the RoboNDT tool-paths, over a scanned area of 1.6m². The output of this research was used to support a 3-year industrial project, called IntACom and led by TWI on behalf of major aerospace sponsors. The result is a demonstrator system, currently in use at TWI Technology Centre, which is capable of inspecting complex geometries with high throughput. The IntACom system can scan real components 2.8 times faster than traditional 3-DoF scanners deploying phased-array inspection and 6.7 times faster than commercial gantry systems deploying traditional single-element inspection.Large components with complex geometries, made of composite materials, have become very common in modern structures. To cope with future demand projections, it is necessary to overcome the current non-destructive testing (NDT) bottlenecks encountered during the inspection phase of manufacture. This thesis investigates several aspects of the introduction of automation within the inspection process of complex parts. The use of six-axis robots for product inspection and non-destructive testing systems is the central investigation of this thesis. The challenges embraced by the research include the development of a novel controlling approach for robotic manipulators and of novel path-planning strategies. The integration of robot manipulators and NDT data acquisition instruments is optimized. An effective and reliable way to encode the NDT data through the interpolated robot feedback positions is implemented. The viability of the new external control method is evaluated experimentally. The observed maximum position and orientation errors are respectively within 2mm and within 1 degree, over an operating envelope of 3m³. A new software toolbox (RoboNDT), aimed at NDT technicians, has been developed during this work. RoboNDT is intended to transform the robot path-planning problem into an easy step of the inspection process. The software incorporates the novel path-planning algorithms developed during this research and is shaped to overcome practical limitations of current OLP software. The software has been experimentally validated using scans on real high value aerospace components. RoboNDT delivers tool-path errors that are lower than the errors given by commercial off-line path-planning software. For example the variability of the standoff is within 10 mm for the tool-paths created with the commercial software and within 4.5 mm for the RoboNDT tool-paths, over a scanned area of 1.6m². The output of this research was used to support a 3-year industrial project, called IntACom and led by TWI on behalf of major aerospace sponsors. The result is a demonstrator system, currently in use at TWI Technology Centre, which is capable of inspecting complex geometries with high throughput. The IntACom system can scan real components 2.8 times faster than traditional 3-DoF scanners deploying phased-array inspection and 6.7 times faster than commercial gantry systems deploying traditional single-element inspection

Recent Advancements in Augmented Reality for Robotic Applications: A Survey

Robots are expanding from industrial applications to daily life, in areas such as medical robotics, rehabilitative robotics, social robotics, and mobile/aerial robotics systems. In recent years, augmented reality (AR) has been integrated into many robotic applications, including medical, industrial, human–robot interactions, and collaboration scenarios. In this work, AR for both medical and industrial robot applications is reviewed and summarized. For medical robot applications, we investigated the integration of AR in (1) preoperative and surgical task planning; (2) image-guided robotic surgery; (3) surgical training and simulation; and (4) telesurgery. AR for industrial scenarios is reviewed in (1) human–robot interactions and collaborations; (2) path planning and task allocation; (3) training and simulation; and (4) teleoperation control/assistance. In addition, the limitations and challenges are discussed. Overall, this article serves as a valuable resource for working in the field of AR and robotic research, offering insights into the recent state of the art and prospects for improvement

Aerospace medicine and biology: A continuing bibliography with indexes (supplement 359)

This bibliography lists 164 reports, articles and other documents introduced into the NASA Scientific and Technical Information System during Jan. 1992. Subject coverage includes: aerospace medicine and physiology, life support systems and man/system technology, protective clothing, exobiology and extraterrestrial life, planetary biology, and flight crew behavior and performance