Implementing Deep Reinforcement Learning for Robotic Control in CoppeliaSim at MANULAB

Dette prosjektet består av tre hovedkomponenter: implementasjon av ROS2 ved MANULAB, bruk av CoppeliaSim, og utføring av Reinforcement Learning ved hjelp av PyRep på IDUN høy-ytelses databehandlingsklynge (HPC). I forbindelse med ROS2 har det blitt etablert en kommunikasjonslink mellom Kuka LBR iiwa 14 R820 robotarmen og to grensesnitt: RVIZ2 og CoppeliaSim. For å legge til rette for denne kommunikasjonen, er det nøye konstruert et ROS2- arbeidsområde. Når det gjelder CoppeliaSim, er det laget en scene som etterligner den virkelige MANULAB-oppsettet. Denne scenen danner grunnlaget for både invers kinematikk og banestyring, som er implementert via egendefinert scripting. Innen området Reinforcement Learning er det utviklet et Python-skript for å legge til rette for anvendelsen av Proximal Policy Optimization (PPO)-algoritmen innenfor den nevnte CoppeliaSim-scenen, ved hjelp av PyRep. Dette skriptet tilbyr et bredt spekter av funksjonaliteter relatert til og nødvendige for RL-trening. For å videre utvide prosjektets kapabiliteter er det laget en container og databehandlingsmiljø for å utnytte databehandlingskraften til IDUN HPC-klynge for utførelse av RL-simuleringer i CoppeliaSim. Effektiviteten og funksjonaliteten til dette oppsettet er bekreftet gjennom en serie tester som involverer mange kjøringer, hvor 13 av disse spenner mellom 13 000 og 20 000 episoder. Prosjektets kodebase og ROS2-arbeidsområdet er åpent tilgjengelige på følgende GitHub-repositorium: https://github.com/EdvardBM/Edvard3. Detaljert overvåking av RL-kjøringene kan fås via følgende WandB-link: https://wandb.ai/mastersthesis/ End_game?workspace=user-ortnevik.The present project encompasses three core components: the implementation of ROS2 at MANULAB, the utilization of CoppeliaSim, and the execution of Reinforcement Learning using PyRep on the IDUN High-Performance Computing (HPC) cluster. In the context of ROS2, a communication link has been established between the Kuka LBR iiwa 14 R820 robot arm and two interfaces: RVIZ2 and CoppeliaSim. To facilitate this communication, a ROS2 workspace has been meticulously constructed. With respect to CoppeliaSim, a scene mirroring the real-world MANULAB setup has been created. This scene forms the basis for both inverse kinematics and path planning, which have been implemented via custom scripting. In the domain of Reinforcement Learning, a Python script has been developed to facilitate the application of the Proximal Policy Optimization (PPO) algorithm within the aforementioned CoppeliaSim scene, using PyRep. This script offers a wide array of functionalities related to and necessary for RL training. To further extend the capabilities of the project, a dedicated container and computing environment have been established to leverage the computational power of the IDUN HPC cluster for performing RL simulations in CoppeliaSim. The efficiency and functionality of this setup have been verified through a series of tests involving numerous runs, with 13 of these spanning between 13,000 and 20,000 episodes. The project’s codebase and the ROS2 workspace are openly available on the following GitHub repository: https://github.com/EdvardBM/Edvard3. Detailed monitoring of the RL runs can be accessed via the following WandB link: https: //wandb.ai/mastersthesis/End_game?workspace=user-ortnevik

A CoppeliaSim Dynamic Simulator for the Da Vinci Research Kit

The design of a physics-based dynamic simulator of a robot requires to properly integrate the robot kinematic and dynamic properties in a virtual environment. Naturally, the closer is the integrated information to the real robot properties, the more accurate the simulator predicts the real robot behaviour. A reliable robot simulator is a valuable asset for developing new research ideas; its use dramatically reduces the costs and it is available to all researchers. This letter presents a dynamic simulator of the da Vinci Research Kit (dVRK) patient-side manipulator (PSM). The kinematic and dynamic properties of the simulator rely on the parameters identified in Wang et al. With respect to the kinematic simulator previously developed by some of the authors, this work: (i) redefines the kinematic architecture and the actuation model by modeling the double parallelogram and the counterweight mechanism, to reflect the structure of the real robot; (ii) integrates the identified dynamic parameters in the simulation model. The obtained simulator enables the design and validation of control strategies relying on the robot dynamic model, including interaction force estimation and control, that are fundamental to guarantee safety in many surgical tasks

Combining Grasping with Adaptive Path Following and Locomotion for Modular Snake Robots

In this paper, a framework architecture that combines grasping with adaptive locomotion for modular snake robots is presented. The proposed framework allows for simulating a snake robot model with locomotion and prehensile capabilities in a virtual environment. The simulated robot can be equipped with different sensors. Tactile perception can be achieved by using contact sensors to retrieve forces, torques, contact positions and contact normals. A camera can be attached to the snake robot head for visual perception purposes. To demonstrate the potential of the proposed framework, a case study is outlined concerning the execution of operations that combine locomotion and grasping. Related simulation results are presented.publishedVersio

HoRoSim, a Holistic Robot Simulator: Arduino Code, Electronic Circuits and Physics

Online teaching, which has been the only way to keep teaching during the pandemic, imposes severe restrictions on hand-on robotic courses. Robot simulators can help to reduce the impact, but most of them use high-level commands to control the robots. In this paper, a new holistic simulator, HoRoSim, is presented. The new simulator uses abstractions of electronic circuits, Arduino code and a robot simulator with a physics engine to simulate microcontroller-based robots. To the best of our knowledge, it is the only tool that simulates Arduino code and multi-body physics. In order to illustrate its possibilities, two use cases are analysed: a line-following robot, which was used by our students to finish their mandatory activities, and a PID controller testbed. Preliminary tests with master's students indicate that HoRoSim can help to increase student engagement during online courses

Robust Tracking Control of Heterogeneous Robots With Uncertainty: A Super-Exponential Convergence Neurodynamic Approach

Abstract The immediate feedback tracking control system design of heterogeneous robots with uncertainty is considered to be a significant issue in robotic research. Note that when the robot information is uncertain, the scale of computation would become increasingly large and the accuracy of tracking control would become exceptionally low. The realization of the immediate feedback control system of heterogeneous robots with uncertainty remains to be a challenging problem. Many conventional zeroing neural network (CZNN) models have been developed accordingly. However, most of them are supported by the hypothesis that the robot parameters are complete and accurate, and the associated models possess the exponential convergence property. To handle the robot uncertainty as well as to improve the convergence performance, a new zeroing neural network (ZNN) with super-exponential convergence (SEC) rate is put forward in this paper termed SEC-ZNN, to resolve the robust control issue of uncertain heterogeneous robots. The proposed SEC-ZNN takes full advantage of effector real-time information, with robust controlling and super-exponential convergence performance so far as to the robot information is uncertain. Theoretically, the super-exponential convergence properties including lower error bound and faster convergence rate are rigorously proved. Moreover, circular path-tracking example, comparisons and tests via MATLAB, Coppeliasim and experiment via robot INNFOS substantiate the efficaciousness and preponderance of the SEC-ZNN for the immediate feedback control system for heterogeneous robots with uncertainty. Note to Practitioners —This paper is motivated by the problem that most robots which need real-time tracking control in real applications come with uncertainty. It is important to note that traditional robot tracking control algorithms mostly require complete robot information or assume information complete, which does not correspond to the actual situation of robot control. Moreover, for practical applications in robotics, the real-time tracking control problem is very attractive. Therefore, an accurate, efficient and stable solution is of great significance to practitioners in this area. In this paper, the SEC-ZNN algorithm is proposed to solve the problem of real-time control of heterogeneous robots with uncertainty in real applications for practitioners. The proposed methos makes full use of the real-time feedback infromation to solve the real-time tracking control problem of heterogeneous robots with uncertainty at the velocity level. The algorithmic steps and principle explanation of the SEC-ZNN scheme are also presented for better understanding. Simulation studies and comparisons are performed on a Stewart robot to confirm the effectiveness and superiority of the proposed scheme. Furthermore, the simulation experiment in Coppeliasim platform is performed to confirm the possibility of portability of the SEC-ZNN to real robot operations. Finally, applications on a real-world robot INNFOS verify the physical relizability of the proposed SEC-ZNN for the engineering practice via heterogeneous robots.Abstract The immediate feedback tracking control system design of heterogeneous robots with uncertainty is considered to be a significant issue in robotic research. Note that when the robot information is uncertain, the scale of computation would become increasingly large and the accuracy of tracking control would become exceptionally low. The realization of the immediate feedback control system of heterogeneous robots with uncertainty remains to be a challenging problem. Many conventional zeroing neural network (CZNN) models have been developed accordingly. However, most of them are supported by the hypothesis that the robot parameters are complete and accurate, and the associated models possess the exponential convergence property. To handle the robot uncertainty as well as to improve the convergence performance, a new zeroing neural network (ZNN) with super-exponential convergence (SEC) rate is put forward in this paper termed SEC-ZNN, to resolve the robust control issue of uncertain heterogeneous robots. The proposed SEC-ZNN takes full advantage of effector real-time information, with robust controlling and super-exponential convergence performance so far as to the robot information is uncertain. Theoretically, the super-exponential convergence properties including lower error bound and faster convergence rate are rigorously proved. Moreover, circular path-tracking example, comparisons and tests via MATLAB, Coppeliasim and experiment via robot INNFOS substantiate the efficaciousness and preponderance of the SEC-ZNN for the immediate feedback control system for heterogeneous robots with uncertainty. Note to Practitioners —This paper is motivated by the problem that most robots which need real-time tracking control in real applications come with uncertainty. It is important to note that traditional robot tracking control algorithms mostly require complete robot information or assume information complete, which does not correspond to the actual situation of robot control. Moreover, for practical applications in robotics, the real-time tracking control problem is very attractive. Therefore, an accurate, efficient and stable solution is of great significance to practitioners in this area. In this paper, the SEC-ZNN algorithm is proposed to solve the problem of real-time control of heterogeneous robots with uncertainty in real applications for practitioners. The proposed methos makes full use of the real-time feedback infromation to solve the real-time tracking control problem of heterogeneous robots with uncertainty at the velocity level. The algorithmic steps and principle explanation of the SEC-ZNN scheme are also presented for better understanding. Simulation studies and comparisons are performed on a Stewart robot to confirm the effectiveness and superiority of the proposed scheme. Furthermore, the simulation experiment in Coppeliasim platform is performed to confirm the possibility of portability of the SEC-ZNN to real robot operations. Finally, applications on a real-world robot INNFOS verify the physical relizability of the proposed SEC-ZNN for the engineering practice via heterogeneous robots

Time-Series Classification for Action Detection in Imitation Learning

V této práci porovnáváme různé metody klasifikace časových řad. Metody pokrývají klasifikaci algoritmem KNN pomocí vypočítaných vzdáleností Dynamic Time Warping (DTW), Long-Short Term memory (LSTM) a metodou GRU. Porovnáváme kvalitu klasifikátorů na 4 různých datasetech, které zahrnují pohyby rukou, gest, pohyby robotického manipulátoru a audio nahrávky. Výsledky vyhodnocujeme pro různá nastavení a parametry metod. Porovnává se obtížnost jednotlivých datasetů.In this work, we compare different methods for time series classification. The methods cover KNN classification using distances computed by Dynamic Time Warping (DTW), Long-Short Term Memory (LSTM), and GRU method. We compare the quality of the classifiers on 4 different datasets, covering hand movements, gestures, movements of a robotic manipulator, and audio recordings. We evaluate the results for various settings and parameters of the methods. The difficulty of individual datasets is compared

Comprehensive simulation of cooperative robotic system for advanced composite manufacturing: A case study

Composite materials are widely used because of their light weight and high strength properties. They are typically made up of multi-directional layers of high strength fibres, connected by a resin. The manufacturing of composite parts is complex, time-consuming and prone to errors. This work investigates the use of robotics in the field of composite material manufacturing, which has not been well investigated to date (particularly in simulation). Effective autonomous material transportation, accurate localization and limited material deformation during robotic grasping are required for optimum placement and lay-up. In this paper, a simulation of a proposed cooperative robotic system, which integrates an autonomous mobile robot with a fixed-base manipulator, is presented. An approach based on machine vision is adopted to accurately track the position and orientation of the fibre plies. A simulation platform with a built-in physics engine is used to simulate material deformation under gravity and external forces. This allows realistic simulation of robotic manipulation for raw materials. The results demonstrate promising features of the proposed system. A root mean square error of 9.00 mm for the estimation of the raw material position and 0.05 degrees for the fibre orientation detection encourages further research for developing the proposed robotic manufacturing system