Robot Autonomy for Surgery

Autonomous surgery involves having surgical tasks performed by a robot operating under its own will, with partial or no human involvement. There are several important advantages of automation in surgery, which include increasing precision of care due to sub-millimeter robot control, real-time utilization of biosignals for interventional care, improvements to surgical efficiency and execution, and computer-aided guidance under various medical imaging and sensing modalities. While these methods may displace some tasks of surgical teams and individual surgeons, they also present new capabilities in interventions that are too difficult or go beyond the skills of a human. In this chapter, we provide an overview of robot autonomy in commercial use and in research, and present some of the challenges faced in developing autonomous surgical robots

Automated sequence and motion planning for robotic spatial extrusion of 3D trusses

While robotic spatial extrusion has demonstrated a new and efficient means to fabricate 3D truss structures in architectural scale, a major challenge remains in automatically planning extrusion sequence and robotic motion for trusses with unconstrained topologies. This paper presents the first attempt in the field to rigorously formulate the extrusion sequence and motion planning (SAMP) problem, using a CSP encoding. Furthermore, this research proposes a new hierarchical planning framework to solve the extrusion SAMP problems that usually have a long planning horizon and 3D configuration complexity. By decoupling sequence and motion planning, the planning framework is able to efficiently solve the extrusion sequence, end-effector poses, joint configurations, and transition trajectories for spatial trusses with nonstandard topologies. This paper also presents the first detailed computation data to reveal the runtime bottleneck on solving SAMP problems, which provides insight and comparing baseline for future algorithmic development. Together with the algorithmic results, this paper also presents an open-source and modularized software implementation called Choreo that is machine-agnostic. To demonstrate the power of this algorithmic framework, three case studies, including real fabrication and simulation results, are presented.Comment: 24 pages, 16 figure

Modeling, Sensorization and Control of Concentric-Tube Robots

Since the concept of the Concentric-Tube Robot (CTR) was proposed in 2006, CTRs have been a popular research topic in the field of surgical robotics. The unique mechanical design of this robot allows it to navigate through narrow channels in the human anatomy and operate in highly constrained environments. It is therefore likely to become the next generation of surgical robots to overcome the challenges that cannot be addressed by current technologies. In CSTAR, we have had ongoing work over the past several years aimed at developing novel techniques and technologies for CTRs. This thesis describes the contributions made in this context, focusing primarily on topics such as modeling, sensorization, and control of CTRs. Prior to this work, one of the main challenges in CTRs was to develop a kinematic model that achieves a balance between the numerical accuracy and computational efficiency for surgical applications. In this thesis, a fast kinematic model of CTRs is proposed, which can be solved at a comparatively fast rate (0.2 ms) with minimal loss of accuracy (0.1 mm) for a 3-tube CTR. A Jacobian matrix is derived based on this model, leading to the development of a real-time trajectory tracking controller for CTRs. For tissue-robot interactions, a force-rejection controller is proposed for position control of CTRs under time-varying force disturbances. In contrast to rigid-link robots, instability of position control could be caused by non-unique solutions to the forward kinematics of CTRs. This phenomenon is modeled and analyzed, resulting in design criteria that can ensure kinematic stability of a CTR in its entire workspace. Force sensing is another major difficulty for CTRs. To address this issue, commercial force/torque sensors (Nano43, ATI Industrial Automation, United States) are integrated into one of our CTR prototypes. These force/torque sensors are replaced by Fiber-Bragg Grating (FBG) sensors that are helically-wrapped and embedded in CTRs. A strain-force calculation algorithm is proposed, to convert the reflected wavelength of FBGs into force measurements with 0.1 N force resolution at 100 Hz sampling rate. In addition, this thesis reports on our innovations in prototyping drive units for CTRs. Three designs of CTR prototypes are proposed, the latest one being significantly more compact and cost efficient in comparison with most designs in the literature. All of these contributions have brought this technology a few steps closer to being used in operating rooms. Some of the techniques and technologies mentioned above are not merely limited to CTRs, but are also suitable for problems arising in other types of surgical robots, for example, for sensorizing da Vinci surgical instruments for force sensing (see Appendix A)

Surgical Subtask Automation for Intraluminal Procedures using Deep Reinforcement Learning

Intraluminal procedures have opened up a new sub-field of minimally invasive surgery that use flexible instruments to navigate through complex luminal structures of the body, resulting in reduced invasiveness and improved patient benefits. One of the major challenges in this field is the accurate and precise control of the instrument inside the human body. Robotics has emerged as a promising solution to this problem. However, to achieve successful robotic intraluminal interventions, the control of the instrument needs to be automated to a large extent. The thesis first examines the state-of-the-art in intraluminal surgical robotics and identifies the key challenges in this field, which include the need for safe and effective tool manipulation, and the ability to adapt to unexpected changes in the luminal environment. To address these challenges, the thesis proposes several levels of autonomy that enable the robotic system to perform individual subtasks autonomously, while still allowing the surgeon to retain overall control of the procedure. The approach facilitates the development of specialized algorithms such as Deep Reinforcement Learning (DRL) for subtasks like navigation and tissue manipulation to produce robust surgical gestures. Additionally, the thesis proposes a safety framework that provides formal guarantees to prevent risky actions. The presented approaches are evaluated through a series of experiments using simulation and robotic platforms. The experiments demonstrate that subtask automation can improve the accuracy and efficiency of tool positioning and tissue manipulation, while also reducing the cognitive load on the surgeon. The results of this research have the potential to improve the reliability and safety of intraluminal surgical interventions, ultimately leading to better outcomes for patients and surgeons

Data-Driven Methods Applied to Soft Robot Modeling and Control: A Review

Soft robots show compliance and have infinite degrees of freedom. Thanks to these properties, such robots can be leveraged for surgery, rehabilitation, biomimetics, unstructured environment exploring, and industrial grippers. In this case, they attract scholars from a variety of areas. However, nonlinearity and hysteresis effects also bring a burden to robot modeling. Moreover, following their flexibility and adaptation, soft robot control is more challenging than rigid robot control. In order to model and control soft robots, a large number of data-driven methods are utilized in pairs or separately. This review first briefly introduces two foundations for data-driven approaches, which are physical models and the Jacobian matrix, then summarizes three kinds of data-driven approaches, which are statistical method, neural network, and reinforcement learning. This review compares the modeling and controller features, e.g., model dynamics, data requirement, and target task, within and among these categories. Finally, we summarize the features of each method. A discussion about the advantages and limitations of the existing modeling and control approaches is presented, and we forecast the future of data-driven approaches in soft robots. A website (https://sites.google.com/view/23zcb) is built for this review and will be updated frequently. Note to Practitioners —This work is motivated by the need for a review introducing soft robot modeling and control methods in parallel. Modeling and control play significant roles in robot research, and they are challenging especially for soft robots. The nonlinear and complex deformation of such robots necessitates specific modeling and control approaches. We introduce the state-of-the-art data-driven methods and survey three approaches widely utilized. This review also compares the performance of these methods, considering some important features like data amount requirement, control frequency, and target task. The features of each approach are summarized, and we discuss the possible future of this area

Control strategies for cleaning robots in domestic applications: A comprehensive review:

Service robots are built and developed for various applications to support humans as companion, caretaker, or domestic support. As the number of elderly people grows, service robots will be in increasing demand. Particularly, one of the main tasks performed by elderly people, and others, is the complex task of cleaning. Therefore, cleaning tasks, such as sweeping floors, washing dishes, and wiping windows, have been developed for the domestic environment using service robots or robot manipulators with several control approaches. This article is primarily focused on control methodology used for cleaning tasks. Specifically, this work mainly discusses classical control and learning-based controlled methods. The classical control approaches, which consist of position control, force control, and impedance control , are commonly used for cleaning purposes in a highly controlled environment. However, classical control methods cannot be generalized for cluttered environment so that learning-based control methods could be an alternative solution. Learning-based control methods for cleaning tasks can encompass three approaches: learning from demonstration (LfD), supervised learning (SL), and reinforcement learning (RL). These control approaches have their own capabilities to generalize the cleaning tasks in the new environment. For example, LfD, which many research groups have used for cleaning tasks, can generate complex cleaning trajectories based on human demonstration. Also, SL can support the prediction of dirt areas and cleaning motion using large number of data set. Finally, RL can learn cleaning actions and interact with the new environment by the robot itself. In this context, this article aims to provide a general overview of robotic cleaning tasks based on different types of control methods using manipulator. It also suggest a description of the future directions of cleaning tasks based on the evaluation of the control approaches

Improving Strength and Stability in Continuum Robots

Continuum robots, which are bio-inspired ’trunk-like’ robots, are characterized for their inherent compliance and range of motion. One of the key challenges in continuum robotics research is developing robots with sufficient strength and stability without adding additional weight or complexity to the design. The research conducted in this dissertation encompasses design and modeling strategies that address these challenges in strength and stability. This work improves three continuum robot actuation paradigms: (1) tendon-driven continuum robots (TDCR), (2) concentric tube robots (CTR), and (3) concentric push-pull robots (CPPR). The first chapter of contribution covers strategies for improving strength in TDCRs. The payload capacity and torsional stiffness of the robot can be improved by leveraging the geometry of the backbone design and tendon routing, with design choices experimentally validated on a robot prototype. The second chapter covers a new bending actuator, concentric precurved bellows (CPB), that are based upon CTR actuation. The high torsional stiffness of bellows geometry virtually eliminates the torsional compliance instability found in CTRs. Two bellows designs are developed for 3D printing and the mechanical properties of these designs are characterized through experiments on prototypes and in static finite element analysis. A torsionally rigid kinematic model is derived and validated on 3D printed prototypes. The third chapter of contribution covers the development and validation of a mechanics-based CPPR kinematics model. CPPRs are constructed from concentrically nested, asymmetrically patterned tubes that are fixed together at their distal tips. Relative translations between the tubes induces bending shapes from the robot. The model expands the possible design space of CPPRs by enabling the modeling of external loads, non-planar bending shapes, and CPPRs with more than two tubes. The model is validated on prototypes in loaded and unloaded experiments

Soft Robot-Assisted Minimally Invasive Surgery and Interventions: Advances and Outlook

Since the emergence of soft robotics around two decades ago, research interest in the field has escalated at a pace. It is fuelled by the industry's appreciation of the wide range of soft materials available that can be used to create highly dexterous robots with adaptability characteristics far beyond that which can be achieved with rigid component devices. The ability, inherent in soft robots, to compliantly adapt to the environment, has significantly sparked interest from the surgical robotics community. This article provides an in-depth overview of recent progress and outlines the remaining challenges in the development of soft robotics for minimally invasive surgery

Towards edge robotics: the progress from cloud-based robotic systems to intelligent and context-aware robotic services

Current robotic systems handle a different range of applications such as video surveillance, delivery of goods, cleaning, material handling, assembly, painting, or pick and place services. These systems have been embraced not only by the general population but also by the vertical industries to help them in performing daily activities. Traditionally, the robotic systems have been deployed in standalone robots that were exclusively dedicated to performing a specific task such as cleaning the floor in indoor environments. In recent years, cloud providers started to offer their infrastructures to robotic systems for offloading some of the robot’s functions. This ultimate form of the distributed robotic system was first introduced 10 years ago as cloud robotics and nowadays a lot of robotic solutions are appearing in this form. As a result, standalone robots became software-enhanced objects with increased reconfigurability as well as decreased complexity and cost. Moreover, by offloading the heavy processing from the robot to the cloud, it is easier to share services and information from various robots or agents to achieve better cooperation and coordination. Cloud robotics is suitable for human-scale responsive and delay-tolerant robotic functionalities (e.g., monitoring, predictive maintenance). However, there is a whole set of real-time robotic applications (e.g., remote control, motion planning, autonomous navigation) that can not be executed with cloud robotics solutions, mainly because cloud facilities traditionally reside far away from the robots. While the cloud providers can ensure certain performance in their infrastructure, very little can be ensured in the network between the robots and the cloud, especially in the last hop where wireless radio access networks are involved. Over the last years advances in edge computing, fog computing, 5G NR, network slicing, Network Function Virtualization (NFV), and network orchestration are stimulating the interest of the industrial sector to satisfy the stringent and real-time requirements of their applications. Robotic systems are a key piece in the industrial digital transformation and their benefits are very well studied in the literature. However, designing and implementing a robotic system that integrates all the emerging technologies and meets the connectivity requirements (e.g., latency, reliability) is an ambitious task. This thesis studies the integration of modern Information andCommunication Technologies (ICTs) in robotic systems and proposes some robotic enhancements that tackle the real-time constraints of robotic services. To evaluate the performance of the proposed enhancements, this thesis departs from the design and prototype implementation of an edge native robotic system that embodies the concepts of edge computing, fog computing, orchestration, and virtualization. The proposed edge robotics system serves to represent two exemplary robotic applications. In particular, autonomous navigation of mobile robots and remote-control of robot manipulator where the end-to-end robotic system is distributed between the robots and the edge server. The open-source prototype implementation of the designed edge native robotic system resulted in the creation of two real-world testbeds that are used in this thesis as a baseline scenario for the evaluation of new innovative solutions in robotic systems. After detailing the design and prototype implementation of the end-to-end edge native robotic system, this thesis proposes several enhancements that can be offered to robotic systems by adapting the concept of edge computing via the Multi-Access Edge Computing (MEC) framework. First, it proposes exemplary network context-aware enhancements in which the real-time information about robot connectivity and location can be used to dynamically adapt the end-to-end system behavior to the actual status of the communication (e.g., radio channel). Three different exemplary context-aware enhancements are proposed that aim to optimize the end-to-end edge native robotic system. Later, the thesis studies the capability of the edge native robotic system to offer potential savings by means of computation offloading for robot manipulators in different deployment configurations. Further, the impact of different wireless channels (e.g., 5G, 4G andWi-Fi) to support the data exchange between a robot manipulator and its remote controller are assessed. In the following part of the thesis, the focus is set on how orchestration solutions can support mobile robot systems to make high quality decisions. The application of OKpi as an orchestration algorithm and DLT-based federation are studied to meet the KPIs that autonomously controlledmobile robots have in order to provide uninterrupted connectivity over the radio access network. The elaborated solutions present high compatibility with the designed edge robotics system where the robot driving range is extended without any interruption of the end-to-end edge robotics service. While the DLT-based federation extends the robot driving range by deploying access point extension on top of external domain infrastructure, OKpi selects the most suitable access point and computing resource in the cloud-to-thing continuum in order to fulfill the latency requirements of autonomously controlled mobile robots. To conclude the thesis the focus is set on how robotic systems can improve their performance by leveraging Artificial Intelligence (AI) and Machine Learning (ML) algorithms to generate smart decisions. To do so, the edge native robotic system is presented as a true embodiment of a Cyber-Physical System (CPS) in Industry 4.0, showing the mission of AI in such concept. It presents the key enabling technologies of the edge robotic system such as edge, fog, and 5G, where the physical processes are integrated with computing and network domains. The role of AI in each technology domain is identified by analyzing a set of AI agents at the application and infrastructure level. In the last part of the thesis, the movement prediction is selected to study the feasibility of applying a forecast-based recovery mechanism for real-time remote control of robotic manipulators (FoReCo) that uses ML to infer lost commands caused by interference in the wireless channel. The obtained results are showcasing the its potential in simulation and real-world experimentation.Programa de Doctorado en Ingeniería Telemática por la Universidad Carlos III de MadridPresidente: Karl Holger.- Secretario: Joerg Widmer.- Vocal: Claudio Cicconett