Reset-free Trial-and-Error Learning for Robot Damage Recovery

The high probability of hardware failures prevents many advanced robots (e.g., legged robots) from being confidently deployed in real-world situations (e.g., post-disaster rescue). Instead of attempting to diagnose the failures, robots could adapt by trial-and-error in order to be able to complete their tasks. In this situation, damage recovery can be seen as a Reinforcement Learning (RL) problem. However, the best RL algorithms for robotics require the robot and the environment to be reset to an initial state after each episode, that is, the robot is not learning autonomously. In addition, most of the RL methods for robotics do not scale well with complex robots (e.g., walking robots) and either cannot be used at all or take too long to converge to a solution (e.g., hours of learning). In this paper, we introduce a novel learning algorithm called "Reset-free Trial-and-Error" (RTE) that (1) breaks the complexity by pre-generating hundreds of possible behaviors with a dynamics simulator of the intact robot, and (2) allows complex robots to quickly recover from damage while completing their tasks and taking the environment into account. We evaluate our algorithm on a simulated wheeled robot, a simulated six-legged robot, and a real six-legged walking robot that are damaged in several ways (e.g., a missing leg, a shortened leg, faulty motor, etc.) and whose objective is to reach a sequence of targets in an arena. Our experiments show that the robots can recover most of their locomotion abilities in an environment with obstacles, and without any human intervention.Comment: 18 pages, 16 figures, 3 tables, 6 pseudocodes/algorithms, video at https://youtu.be/IqtyHFrb3BU, code at https://github.com/resibots/chatzilygeroudis_2018_rt

Real-time trajectory generation for dynamic systems with nonholonomic constraints using Player/Stage and NTG.

This thesis will present various methods of trajectory generation for various types of mobile robots. Then it will progress to evaluating Robot Operating Systems (ROS’s) that can be used to control and simulate mobile robots, and it will explain why Player/Stage was chosen as the ROS for this thesis. It will then discuss Nonlinear Trajectory Generation as the main method for producing a path for mobile robots with dynamic and kinematic constraints. Finally, it will combine Player, Stage, and NTG into a system that produces a trajectory in real-time for a mobile robot and simulates a differential drive robot being driven from the initial state to the goal state in the presence of obstacles. Experiments will include the following: Blobfinding for physical and simulated camera systems, position control of physical and simulated differential drive robots, wall following using simulated range sensors, trajectory generation for omnidirectional and differential drive robots, and a combination of blobfinding, position control, and trajectory generation. Each experiment was a success, to varying degrees. The culmination of the thesis will present a real-time trajectory generation and position control method for a differential drive robot in the presence of obstacles

The Pyro toolkit for AI and robotics

This article introduces Pyro, an open-source Python robotics toolkit for exploring topics in AI and robotics. We present key abstractions that al- low Pyro controllers to run unchanged on a variety of real and simulated robots. We demonstrate Py- ro’s use in a set of curricular modules. We then de- scribe how Pyro can provide a smooth transition for the student from symbolic agents to real-world robots, which significantly reduces the cost of learning to use robots. Finally we show how Pyro has been successfully integrated into existing AI and robotics courses

The Pyro toolkit for AI and robotics

This article introduces Pyro, an open-source Python robotics toolkit for exploring topics in AI and robotics. We present key abstractions that al- low Pyro controllers to run unchanged on a variety of real and simulated robots. We demonstrate Py- ro’s use in a set of curricular modules. We then de- scribe how Pyro can provide a smooth transition for the student from symbolic agents to real-world robots, which significantly reduces the cost of learning to use robots. Finally we show how Pyro has been successfully integrated into existing AI and robotics courses