SofaGym: An open platform for Reinforcement Learning based on Soft Robot simulations

International audienceOpenAI Gym is one of the standard interfaces used to train Reinforcement Learning (RL) Algorithms. The Simulation Open Framework Architecture (SOFA) is a physics based engine that is used for soft robotics simulation and control based on real-time models of deformation. The aim of this paper is to present SofaGym, an open source software to create OpenAI Gym interfaces, called environments, out of soft robot digital twins.The link between soft robotics and RL offers new challenges for both fields: representation of the soft robot in a RL context, complex interactions with the environment, use of specific mechanical tools to control soft robots, transfer of policies learned in simulation to the real world, etc. The article presents the large possible uses of SofaGym to tackle these challenges by using RL and planning algorithms. This publication contains neither new algorithms nor new models but proposes a new platform, open to the community, that offers non existing possibilities of coupling RL to physics based simulation of soft robots. We present 11 environments, representing a wide variety of soft robots and applications, we highlight the challenges showcased by each environment. We propose methods of solving the task using traditional control, RL and planning and point out research perspectives using the platform

Automated planning for robotic guidewire navigation in the coronary arteries

International audienceSoft continuum robots, and comparable instruments allow to perform some surgical procedures noninvasively. While safer, less morbid and more cost-effective, these medical interventions increase the complexity for the practitioners: the manipulation of anatomical structures is indirect through telescopic and flexible devices and the visual feedback is indirect through monitors. Interventional cardiology is an example of complex procedures where catheters and guidewires are manipulated to reach and treat remote areas of the vascular network. Such interventions may be assisted with a robot that will operate the tools but the planning (choice of tools and trajectories) remains a complex task. In this paper we use a simulation framework for flexible devices inside the vasculature and we propose a method to automatically control these devices to reach specific locations. Experiments performed on 15 patient geometries exhibit good performance. Automatic manipulation reaches the goal in more than 90% of the cases

Navigation autonome de guides et cathéters pour une opération en cardiologie interventionnelle robotisée

Cette thèse porte sur la navigation automatique de guides et cathéters dans les artères coronaires. Dans un premier temps, nous introduisons le contexte de la cardiologie interventionnelle et le cas particulier des interventions coronaires percutanées. Nous justifions aussi l'utilité de la robotisation de ces procédures. Nous faisons ensuite le pont entre le problème de la navigation de guides et le contrôle de robots déformables, puis examinons les travaux existants sur le même sujet.Notre étude commence par la conception de simulations dynamiques réalistes des artères, du coeur et de leur interaction avec les instruments chirurgicaux. Cela a nécessité la collecte de données patient, la compréhension de l'influence du battement cardiaque sur la déformation des artères et comment cette déformation impacte la navigation de guides. Les interventions coronaires percutanées considérées reposent également fortement sur les contacts entre les instruments et les parois arterielles, ce qui est une difficulté supplémentaire au regard du temps de simulation ainsi que de leur précision.Nous avons ensuite utilisé ces simulations pour formuler le problème de la navigation de guides dans les artères coronaires comme un problème de prise de décision séquentielle. Nous avons défini les espaces d'état et d'action correspondants et conçu une fonction de récompense adéquate. Nous avons testé plusieurs stratégies de contrôle, y compris des mèthodes de contrôle basées sur un modèle inverse et des méthodes de recherche arborescente. Nous avons également proposé une nouvelle combinaison de ces méthodes qui atteint de bonnes performances dans les cas de figure testés.Finalement nous avons conçu un algorithme de vision et un schéma de recallage permettant de transférer les algorithmes de commande sur un robot physique. L'algorithme de contrôle en boucle fermé ainsi constitué a été testé sur deux fantômes en silicone différents sur lesquels l'algorithme atteint un taux de succès comparable à une navigation manuelle et à une navigation en télé-opérant le robot.This thesis focuses on automatic navigation of guidewires and catheters in the coronary arteries. We first introduce the context of interventional cardiology and percutaneous coronary interventions. We also justify the use of a robot for those procedures. We then link the guidewire navigation problem to the control of soft robots and review the existing literature.Our study starts with building realistic, dynamic simulations of the arteries, heart and their interaction with the instruments. This required the gathering of patient data, understanding how the heart beat motion deforms the heart and how these deformations impact guidewire navigation. Percutaneous coronary interventions are also very contact dense which is a challenge for both simulation speed and accuracy.We then used these simulations to frame the problem of navigating the coronaries as a sequential decision making problem. We determined the corresponding state and action spaces and devised a reward. We tested a variety of control schemes on the navigation problem, including inverse model based control and tree-search based control and proposed a novel combination of both which achieves good performance in navigating the coronaries.Finally, we design a vision algorithm and registration scheme to transfer the algorithms to a physical robot. The closed loop control scheme was tested on two different silicone phantoms and achieves the same success rate as navigating by hand or by tele-operating the robot

Improving the accessibility of touristic destinations with an assistive technology for hiking:applying universal design principles through service design

Accessible Tourism focus on the logistical attributes being accessible to all and on the process to develop accessible products and services with all stakeholders of the touristic destination. Assistive technologies can be used to improve the accessibility of touristic destination and attraction. Some assistive technologies are designed for hiking. However, their integration on the customer journey has to be designed as a service. To this end, universal design principles and guidelines can be used to design products and services accessible to all. Universal design and accessible tourism are both rooted in the social model of disability, which states that it is the society who is disabling. The potential and the conceptualization of applying universal design principles for tourism has been widely discussed. However, little has been done to operationalize this idea. In this article, we demonstrate how to co-create with users an accessible touristic service based on an assistive technology who enables hiking for people using wheelchairs. Our main findings illustrate the pros and the cons of using and assistive technologies and the importance of considering the whole customer journey to improve the accessibility of touristic destinations

Automated planning for robotic guidewire navigation in the coronary arteries

International audienceSoft continuum robots, and comparable instruments allow to perform some surgical procedures noninvasively. While safer, less morbid and more cost-effective, these medical interventions increase the complexity for the practitioners: the manipulation of anatomical structures is indirect through telescopic and flexible devices and the visual feedback is indirect through monitors. Interventional cardiology is an example of complex procedures where catheters and guidewires are manipulated to reach and treat remote areas of the vascular network. Such interventions may be assisted with a robot that will operate the tools but the planning (choice of tools and trajectories) remains a complex task. In this paper we use a simulation framework for flexible devices inside the vasculature and we propose a method to automatically control these devices to reach specific locations. Experiments performed on 15 patient geometries exhibit good performance. Automatic manipulation reaches the goal in more than 90% of the cases

Automated planning for robotic guidewire navigation in the coronary arteries

International audienceSoft continuum robots, and comparable instruments allow to perform some surgical procedures noninvasively. While safer, less morbid and more cost-effective, these medical interventions increase the complexity for the practitioners: the manipulation of anatomical structures is indirect through telescopic and flexible devices and the visual feedback is indirect through monitors. Interventional cardiology is an example of complex procedures where catheters and guidewires are manipulated to reach and treat remote areas of the vascular network. Such interventions may be assisted with a robot that will operate the tools but the planning (choice of tools and trajectories) remains a complex task. In this paper we use a simulation framework for flexible devices inside the vasculature and we propose a method to automatically control these devices to reach specific locations. Experiments performed on 15 patient geometries exhibit good performance. Automatic manipulation reaches the goal in more than 90% of the cases

SofaGym: An open platform for Reinforcement Learning based on Soft Robot simulations

International audienceOpenAI Gym is one of the standard interfaces used to train Reinforcement Learning (RL) Algorithms. The Simulation Open Framework Architecture (SOFA) is a physics based engine that is used for soft robotics simulation and control based on real-time models of deformation. The aim of this paper is to present SofaGym, an open source software to create OpenAI Gym interfaces, called environments, out of soft robot digital twins.The link between soft robotics and RL offers new challenges for both fields: representation of the soft robot in a RL context, complex interactions with the environment, use of specific mechanical tools to control soft robots, transfer of policies learned in simulation to the real world, etc. The article presents the large possible uses of SofaGym to tackle these challenges by using RL and planning algorithms. This publication contains neither new algorithms nor new models but proposes a new platform, open to the community, that offers non existing possibilities of coupling RL to physics based simulation of soft robots. We present 11 environments, representing a wide variety of soft robots and applications, we highlight the challenges showcased by each environment. We propose methods of solving the task using traditional control, RL and planning and point out research perspectives using the platform

SofaGym: An open platform for Reinforcement Learning based on Soft Robot simulations

International audienceOpenAI Gym is one of the standard interfaces used to train Reinforcement Learning (RL) Algorithms. The Simulation Open Framework Architecture (SOFA) is a physics based engine that is used for soft robotics simulation and control based on real-time models of deformation. The aim of this paper is to present SofaGym, an open source software to create OpenAI Gym interfaces, called environments, out of soft robot digital twins.The link between soft robotics and RL offers new challenges for both fields: representation of the soft robot in a RL context, complex interactions with the environment, use of specific mechanical tools to control soft robots, transfer of policies learned in simulation to the real world, etc. The article presents the large possible uses of SofaGym to tackle these challenges by using RL and planning algorithms. This publication contains neither new algorithms nor new models but proposes a new platform, open to the community, that offers non existing possibilities of coupling RL to physics based simulation of soft robots. We present 11 environments, representing a wide variety of soft robots and applications, we highlight the challenges showcased by each environment. We propose methods of solving the task using traditional control, RL and planning and point out research perspectives using the platform

Automated planning for robotic guidewire navigation in the coronary arteries

International audienceSoft continuum robots, and comparable instruments allow to perform some surgical procedures noninvasively. While safer, less morbid and more cost-effective, these medical interventions increase the complexity for the practitioners: the manipulation of anatomical structures is indirect through telescopic and flexible devices and the visual feedback is indirect through monitors. Interventional cardiology is an example of complex procedures where catheters and guidewires are manipulated to reach and treat remote areas of the vascular network. Such interventions may be assisted with a robot that will operate the tools but the planning (choice of tools and trajectories) remains a complex task. In this paper we use a simulation framework for flexible devices inside the vasculature and we propose a method to automatically control these devices to reach specific locations. Experiments performed on 15 patient geometries exhibit good performance. Automatic manipulation reaches the goal in more than 90% of the cases