Reset-free Trial-and-Error Learning for Robot Damage Recovery

The high probability of hardware failures prevents many advanced robots (e.g., legged robots) from being confidently deployed in real-world situations (e.g., post-disaster rescue). Instead of attempting to diagnose the failures, robots could adapt by trial-and-error in order to be able to complete their tasks. In this situation, damage recovery can be seen as a Reinforcement Learning (RL) problem. However, the best RL algorithms for robotics require the robot and the environment to be reset to an initial state after each episode, that is, the robot is not learning autonomously. In addition, most of the RL methods for robotics do not scale well with complex robots (e.g., walking robots) and either cannot be used at all or take too long to converge to a solution (e.g., hours of learning). In this paper, we introduce a novel learning algorithm called "Reset-free Trial-and-Error" (RTE) that (1) breaks the complexity by pre-generating hundreds of possible behaviors with a dynamics simulator of the intact robot, and (2) allows complex robots to quickly recover from damage while completing their tasks and taking the environment into account. We evaluate our algorithm on a simulated wheeled robot, a simulated six-legged robot, and a real six-legged walking robot that are damaged in several ways (e.g., a missing leg, a shortened leg, faulty motor, etc.) and whose objective is to reach a sequence of targets in an arena. Our experiments show that the robots can recover most of their locomotion abilities in an environment with obstacles, and without any human intervention.Comment: 18 pages, 16 figures, 3 tables, 6 pseudocodes/algorithms, video at https://youtu.be/IqtyHFrb3BU, code at https://github.com/resibots/chatzilygeroudis_2018_rt

Trends in the control of hexapod robots: a survey

The static stability of hexapods motivates their design for tasks in which stable locomotion is required, such as navigation across complex environments. This task is of high interest due to the possibility of replacing human beings in exploration, surveillance and rescue missions. For this application, the control system must adapt the actuation of the limbs according to their surroundings to ensure that the hexapod does not tumble during locomotion. The most traditional approach considers their limbs as robotic manipulators and relies on mechanical models to actuate them. However, the increasing interest in model-free models for the control of these systems has led to the design of novel solutions. Through a systematic literature review, this paper intends to overview the trends in this field of research and determine in which stage the design of autonomous and adaptable controllers for hexapods is.The first author received funding through a doctoral scholarship from the Portuguese Foundation for Science and Technology (FCT) (Grant No. SFRH/BD/145818/2019), with funds from the Portuguese Ministry of Science, Technology and Higher Education and the European Social Fund through the Programa Operacional Regional Norte. This work has been supported by the FCT national funds, under the national support to R&D units grant, through the reference project UIDB/04436/2020 and UIDP/04436/2020

Real-time Digital Double Framework to Predict Collapsible Terrains for Legged Robots

Inspired by the digital twinning systems, a novel real-time digital double framework is developed to enhance robot perception of the terrain conditions. Based on the very same physical model and motion control, this work exploits the use of such simulated digital double synchronized with a real robot to capture and extract discrepancy information between the two systems, which provides high dimensional cues in multiple physical quantities to represent differences between the modelled and the real world. Soft, non-rigid terrains cause common failures in legged locomotion, whereby visual perception solely is insufficient in estimating such physical properties of terrains. We used digital double to develop the estimation of the collapsibility, which addressed this issue through physical interactions during dynamic walking. The discrepancy in sensory measurements between the real robot and its digital double are used as input of a learning-based algorithm for terrain collapsibility analysis. Although trained only in simulation, the learned model can perform collapsibility estimation successfully in both simulation and real world. Our evaluation of results showed the generalization to different scenarios and the advantages of the digital double to reliably detect nuances in ground conditions.Comment: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Preprint version. Accepted June 202

Locomotion Control of Hexapod Walking Robot with Four Degrees of Freedom per Leg

V této práci představujeme nového šestinohého robota jménem HAntR, kterého jsme vytvořili dle potřeb Laboratoře výpočetní robotiky Centra umělé inteligence fakulty Elektrotechnické Českého vysokého učení technického v Praze. Jeho hlavním účelem jest vylepšit schopnosti pohybu v těžkém terénu původního robotu přidáním čtvrtého stupně volnosti každé noze. Na základě nově navržené nohy jsme také přepracovali celé tělo robotu tak, aby splnilo i další požadavky, jako například menší rozměry, či možnost osazení alespoň šesti Lithium-Iontovými monočlánky. V práci pečlivě popisujeme motivace a úvahy, které nás k výslednému návrhu vedly. Uvádíme řešení přímé i inverzní kinematické úlohy řešené pomocí podmínky na ideální orientaci konce nohy a uvažující i důležité kinematické singularity. Navržený robot byl vyzkoušen v několika experimentech, při kterých byl použit námi navržený řídicí systém napsaný v jazyce C++. Ukázalo se, že HAntR vydrží díky zvýšené energetické hustotě a lepšímu rozkladu sil v končetinách autonomně fungovat přes hodinu. Robot je také schopen jít rychlostí až 0.42m/s, což předčí mnohé srovnatelné roboty. Při experimentu, kdy robot stál na nakloněné rovině, bylo prokázáno zlepšení oproti předchozímu robotu. A také jsme dle pokynů této práce potvrdili, že i HAntR je schopen adaptivní chůze spoléhající pouze na poziční zpětnou vazbu.In this thesis a novel six-legged robot called HAntR is presented. The robot was developed according to needs of the Robotics Laboratory, at the Artificial Intelligent Center, Faculty of Electrical Engineering, Czech Technical University in Prague. Its main purpose is enhancing rough-terrain movement capabilities by upgrading a former design by adding fourth degree of freedom to each leg. We also revised robot torso to fit new leg design and incorporate other requirements such as smaller dimensions with space for at least six Lithium-Ion cells. We thoroughly describe motivations and considerations that led us to the presented particular solution. Further, the solutions of forward and inverse kinematic tasks with partial orientation constraint and important singularities avoidance are presented. The proposed design has been evaluated in several experimental deployments, which utilised developed software controller written in C++. Endurance tests showed, that HAntR is able to remotely operate for over an hour thanks to increased energy density. Maximal speed test resulted to 0.42m/s during tripod gait, which outpaces most of the comparable robotic platforms. Experiment where HAntR stood on platform with varying inclination showed qualitative improvement against former robot. Finally, in accord with the thesis assignment, we proved that HAntR is able to perform walking with adaptive gait using positional feedback only

Development and Field Testing of the FootFall Planning System for the ATHLETE Robots

The FootFall Planning System is a ground-based planning and decision support system designed to facilitate the control of walking activities for the ATHLETE (All-Terrain Hex-Limbed Extra-Terrestrial Explorer) family of robots. ATHLETE was developed at NASA's Jet Propulsion Laboratory (JPL) and is a large six-legged robot designed to serve multiple roles during manned and unmanned missions to the Moon; its roles include transportation, construction and exploration. Over the four years from 2006 through 2010 the FootFall Planning System was developed and adapted to two generations of the ATHLETE robots and tested at two analog field sites (the Human Robotic Systems Project's Integrated Field Test at Moses Lake, Washington, June 2008, and the Desert Research and Technology Studies (D-RATS), held at Black Point Lava Flow in Arizona, September 2010). Having 42 degrees of kinematic freedom, standing to a maximum height of just over 4 meters, and having a payload capacity of 450 kg in Earth gravity, the current version of the ATHLETE robot is a uniquely complex system. A central challenge to this work was the compliance of the high-DOF (Degree Of Freedom) robot, especially the compliance of the wheels, which affected many aspects of statically-stable walking. This paper will review the history of the development of the FootFall system, sharing design decisions, field test experiences, and the lessons learned concerning compliance and self-awareness

Learning physical characteristics like animals for legged robots

Physical characteristics of terrains, such as softness and friction, provide essential information for legged robots to avoid non-geometric obstacles, like mires and slippery stones, in the wild. The perception of such characteristics often relies on tactile perception and vision prediction. Although tactile perception is more accurate, it is limited to close-range use; by contrast, establishing a supervised or self-supervised contactless prediction system using computer vision requires adequate labeled data and lacks the ability to adapt to the dynamic environment. In this paper, we simulate the behavior of animals and propose an unsupervised learning framework for legged robots to learn the physical characteristics of terrains, which is the first report to manage it online, incrementally and with the ability to solve cognitive conflicts. The proposed scheme allows robots to interact with the environment and adjust their cognition in real time, therefore endowing robots with the adaptation ability. Indoor and outdoor experiments on a hexapod robot are carried out to show that the robot can extract tactile and visual features of terrains to create cognitive networks independently; an associative layer between visual and tactile features is created during the robot’s exploration; with the layer, the robot can autonomously generate a physical segmentation model of terrains and solve cognitive conflicts in an ever-changing environment, facilitating its safe navigation

Virtual Reality

At present, the virtual reality has impact on information organization and management and even changes design principle of information systems, which will make it adapt to application requirements. The book aims to provide a broader perspective of virtual reality on development and application. First part of the book is named as "virtual reality visualization and vision" and includes new developments in virtual reality visualization of 3D scenarios, virtual reality and vision, high fidelity immersive virtual reality included tracking, rendering and display subsystems. The second part named as "virtual reality in robot technology" brings forth applications of virtual reality in remote rehabilitation robot-based rehabilitation evaluation method and multi-legged robot adaptive walking in unstructured terrains. The third part, named as "industrial and construction applications" is about the product design, space industry, building information modeling, construction and maintenance by virtual reality, and so on. And the last part, which is named as "culture and life of human" describes applications of culture life and multimedia-technology

Design, Manufacturing, and Locomotion Studies of Ambulatory Micro-Robots

Biological research over the past several decades has elucidated some of the mechanisms behind highly mobile, efficient, and robust locomotion in insects such as the cockroach. Roboticists have used this information to create biologically-inspired machines capable of running, jumping, and climbing robustly over a variety of terrains. To date, little work has been done to develop an at-scale insect-inspired robot capable of similar feats, due to limitations in fabrication, actuation, and electronics integration at small scales. This thesis addresses these challenges, focusing on the mechanical design and fabrication of a sub-2g walking robot, the Harvard Ambulatory MicroRobot (HAMR). The development of HAMR includes modeling and parameter selection for a two degree of freedom leg powertrain that enables locomotion. In addition, a design inspired by pop-up books that enables fast and repeatable assembly of the miniature walking robot is presented. Finally, a method to drive HAMR resulting in speeds up to 37cm/s is presented, along with simple control schemes.Engineering and Applied Science