RSG: Fast Learning Adaptive Skills for Quadruped Robots by Skill Graph

Developing robotic intelligent systems that can adapt quickly to unseen wild situations is one of the critical challenges in pursuing autonomous robotics. Although some impressive progress has been made in walking stability and skill learning in the field of legged robots, their ability to fast adaptation is still inferior to that of animals in nature. Animals are born with massive skills needed to survive, and can quickly acquire new ones, by composing fundamental skills with limited experience. Inspired by this, we propose a novel framework, named Robot Skill Graph (RSG) for organizing massive fundamental skills of robots and dexterously reusing them for fast adaptation. Bearing a structure similar to the Knowledge Graph (KG), RSG is composed of massive dynamic behavioral skills instead of static knowledge in KG and enables discovering implicit relations that exist in be-tween of learning context and acquired skills of robots, serving as a starting point for understanding subtle patterns existing in robots' skill learning. Extensive experimental results demonstrate that RSG can provide rational skill inference upon new tasks and environments and enable quadruped robots to adapt to new scenarios and learn new skills rapidly

RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control

We present a unified model-based and data-driven approach for quadrupedal planning and control to achieve dynamic locomotion over uneven terrain. We utilize on-board proprioceptive and exteroceptive feedback to map sensory information and desired base velocity commands into footstep plans using a reinforcement learning (RL) policy trained in simulation over a wide range of procedurally generated terrains. When ran online, the system tracks the generated footstep plans using a model-based controller. We evaluate the robustness of our method over a wide variety of complex terrains. It exhibits behaviors which prioritize stability over aggressive locomotion. Additionally, we introduce two ancillary RL policies for corrective whole-body motion tracking and recovery control. These policies account for changes in physical parameters and external perturbations. We train and evaluate our framework on a complex quadrupedal system, ANYmal version B, and demonstrate transferability to a larger and heavier robot, ANYmal C, without requiring retraining.Comment: 19 pages, 15 figures, 6 tables, 1 algorithm, submitted to T-RO; under revie

Master of Science

thesisAdvances in the field of robotics have laid a solid foundation for human-robot-interaction research; this research values demonstrations of emotional competence from robotic systems and herein lie opportunities for progress within the therapeutic industry, creation of companion robots, and integration of robotics among everyday households. The development of emotive expression within robotics is progressing at a fair pace; however, there is next to no research on this form of expression as it pertains to a robot's manner of walking. The work presented here proves that it is possible for robots to walk with the capability of expressing emotions that are identifiable by their human counterparts. This hypothesis is explored utilizing a four-legged robot in simulation and reality, and the details necessary for this application are presented in this work. This quadruped is comprised of four manipulators each consisting of seven degrees of freedom. The inverse kinematics and dynamics are solved for each leg with closed form solutions that incorporate the inverse of Euler's finite rotation formula. With the kinematics solved, the robot utilizes a central pattern generator to create a neutral gait and balances with an augmented center of pressure that closely resembles the zero moment point algorithm. Independent of the kinematics, a method of generating poses that represent the emotions: happy, sad, angry, and fearful, is presented. This work also details how to overlay poses atop a gait to transform the neutral gait into an emotive walking style. In addition to laying the framework for developing the emotive walking styles, an evaluation of the presented gaits is detailed. Two IRB approved studies were performed independently of each other. The first study took feedback from subjects regarding ways to make the emotive gaits more compelling and applied them to the initial poses. The second study evaluated the effectiveness of the final gaits, with improved poses, and proves that emotive walking patterns were created; walking patterns that will be suitable for emotional acuity

Design and development of a hominid robot with local control in its adaptable feet to enhance locomotion capabilities

With increasing mechanization of our daily lives, the expectations and demands in robotic systems increase in the general public and in scientists alike. In recent events such as the Deepwater Horizon''-accident or the nuclear disaster at Fukushima, mobile robotic systems were used, e.g., to support local task forces by gaining visual material to allow an analysis of the situation. Especially the Fukushima example shows that the robotic systems not only have to face a variety of different tasks during operation but also have to deal with different demands regarding the robot's mobility characteristics. To be able to cope with future requirements, it seems necessary to develop kinematically complex systems that feature several different operating modes. That is where this thesis comes in: A robotic system is developed, whose morphology is oriented on chimpanzees and which has the possibility due to its electro-mechanical structure and the degrees of freedom in its arms and legs to walk with different gaits in different postures. For the proposed robot, the chimpanzee was chosen as a model, since these animals show a multitude of different gaits in nature. A quadrupedal gait like crawl allows the robot to traverse safely and stable over rough terrain. A change into the humanoid, bipedal posture enables the robot to move in man-made environments. The structures, which are necessary to ensure an effective and stable locomotion in these two poses, e.g., the feet, are presented in more detail within the thesis. This includes the biological model and an abstraction to allow a technical implementation. In addition, biological spines are analyzed and the development of an active, artificial spine for the robotic system is described. These additional degrees of freedom can increase the robot's locomotion and manipulation capabilities and even allow to show movements, which are not possible without a spine. Unfortunately, the benefits of using an artificial spine in robotic systems are nowadays still neglected, due to the increased complexity of system design and control. To be able to control such a kinematically complex system, a multitude of sensors is installed within the robot's structures. By placing evaluation electronics close by, a local and decentralized preprocessing is realized. Due to this preprocessing is it possible to realize behaviors on the lowest level of robot control: in this thesis it is exemplarily demonstrated by a local controller in the robot's lower leg. In addition to the development and evaluation of robot's structures, the functionality of the overall system is analyzed in different environments. This includes the presentation of detailed data to show the advantages and disadvantages of the local controller. The robot can change its posture independently from a quadrupedal into a bipedal stance and the other way around without external assistance. Once the robot stands upright, it is to investigate to what extent the quadrupedal walking pattern and control structures (like the local controller) have to be modified to contribute to the bipedal walking as well

Towards understanding of climbing, tip-over prevention and self-righting behaviors in Hexapoda

Die vorliegende Dissertation mit dem Titel “Towards understanding of climbing, tip-over prevention and self-righting behaviors in Hexapoda” untersucht in drei Studien exemplarisch, wie (i) Wüstenameisen ihre Beine einsetzen um An- und Abstiege zu überwinden, wie (ii) Wüsten- und Waldameisen ein Umkippen an steilen Anstiegen vermeiden, und wie sich (iii) Madagaskar-Fauchschaben, Amerikanische Großschaben und Blaberus discoidalis Audinet-Servill, 1839 aus Rückenlagen drehen und aufrichten. Neuartige biomechanischen Beschreibungen umfassen unter anderem: Impuls- und Kraftwirkungen einzelner Ameisenbeine auf den Untergrund beim Bergauf- und Bergabklettern, Kippmomente bei kletternden Ameisen, Energiegebirge-Modelle (energy landscapes) zur Quantifizierung der Körperform für die funktionelle Beschreibung des Umdrehens aus der Rückenlage

Actuation-Aware Simplified Dynamic Models for Robotic Legged Locomotion

In recent years, we witnessed an ever increasing number of successful hardware implementations of motion planners for legged robots. If one common property is to be identified among these real-world applications, that is the ability of online planning. Online planning is forgiving, in the sense that it allows to relentlessly compensate for external disturbances of whatever form they might be, ranging from unmodeled dynamics to external pushes or unexpected obstacles and, at the same time, follow user commands. Initially replanning was restricted only to heuristic-based planners that exploit the low computational effort of simplified dynamic models. Such models deliberately only capture the main dynamics of the system, thus leaving to the controllers the issue of anchoring the desired trajectory to the whole body model of the robot. In recent years, however, we have seen a number of new approaches attempting to increase the accuracy of the dynamic formulation without trading-off the computational efficiency of simplified models. In this dissertation, as an example of successful hardware implementation of heuristics and simplified model-based locomotion, I describe the framework that I developed for the generation of an omni-directional bounding gait for the HyQ quadruped robot. By analyzing the stable limit cycles for the sagittal dynamics and the Center of Pressure (CoP) for the lateral stabilization, the described locomotion framework is able to achieve a stable bounding while adapting to terrains of mild roughness and to sudden changes of the user desired linear and angular velocities. The next topic reported and second contribution of this dissertation is my effort to formulate more descriptive simplified dynamic models, without trading off their computational efficiency, in order to extend the navigation capabilities of legged robots to complex geometry environments. With this in mind, I investigated the possibility of incorporating feasibility constraints in these template models and, in particular, I focused on the joint torques limits which are usually neglected at the planning stage. In this direction, the third contribution discussed in this thesis is the formulation of the so called actuation wrench polytope (AWP), defined as the set of feasible wrenches that an articulated robot can perform given its actuation limits. Interesected with the contact wrench cone (CWC), this yields a new 6D polytope that we name feasible wrench polytope (FWP), defined as the set of all wrenches that a legged robot can realize given its actuation capabilities and the friction constraints. Results are reported where, thanks to efficient computational geometry algorithms and to appropriate approximations, the FWP is employed for a one-step receding horizon optimization of center of mass trajectory and phase durations given a predefined step sequence on rough terrains. For the sake of reachable workspace augmentation, I then decided to trade off the generality of the FWP formulation for a suboptimal scenario in which a quasi-static motion is assumed. This led to the definition of the, so called, local/instantaneous actuation region and of the global actuation/feasible region. They both can be seen as different variants of 2D linear subspaces orthogonal to gravity where the robot is guaranteed to place its own center of mass while being able to carry its own body weight given its actuation capabilities. These areas can be intersected with the well known frictional support region, resulting in a 2D linear feasible region, thus providing an intuitive tool that enables the concurrent online optimization of actuation consistent CoM trajectories and target foothold locations on rough terrains

Automatic fine motor control behaviours for autonomous mobile agents operating on uneven terrains

A novel mechanism able to produce increasingly stable paths for mobile robotic agents travelling over uneven terrain is proposed in this paper. In doing so, cognitive agents can focus on higher-level goal planning, with the increased confidence the resulting tasks will be automatically accomplished via safe and reliable paths within the lower-level skills of the platform. The strategy proposes the extension of the Fast Marching level-set method of propagating interfaces in 3D lattices with a metric to reduce robot body instability. This is particularly relevant for kinematically reconfigurable platforms which significantly modify their mass distribution through posture adaptation, such as humanoids or mobile robots equipped with manipulator arms or varying traction arrangements. Simulation results of an existing reconfigurable mobile rescue robot operating on real scenarios illustrate the validity of the proposed strategy. Copyright 2010 ACM

Streamlined sim-to-real transfer for deep-reinforcement learning in robotics locomotion

Legged robots possess superior mobility compared to other machines, yet designing controllers for them can be challenging. Classic control methods require engineers to distill their knowledge into controllers, which is time-consuming and limiting when approaching dynamic tasks in unknown environments. Conversely, learning- based methods that gather knowledge from data can potentially unlock the versatility of legged systems. In this thesis, we propose a novel approach called CPG-Actor, which incor- porates feedback into a fully differentiable Central Pattern Generator (CPG) formulation using neural networks and Deep-Reinforcement Learning (RL). This approach achieves approximately twenty times better training performance compared to previous methods and provides insights into the impact of training on the distribution of parameters in both the CPGs and MLP feedback network. Adopting Deep-RL to design controllers comes at the expense of gathering extensive data, typically done in simulation to reduce time. However, controllers trained with data collected in simulation often lose performance when deployed in the real world, referred to as the sim-to-real gap. To address this, we propose a new method called Extended Random Force Injection (ERFI), which randomizes only two parameters to allow for sim-to-real transfer of locomotion controllers. ERFI demonstrated high robustness when varying masses of the base, or attaching a manipulator arm to the robot during testing, and achieved competitive performance comparable to standard randomization techniques. Furthermore, we propose a new method called Roll-Drop to enhance the robustness of Deep-RL policies to observation noise. Roll-Drop introduces dropout during rollout, achieving an 80% success rate when tested with up to 25% noise injected in the observations. Finally, we adopted model-free controllers to enable omni-directional bipedal lo- comotion on point feet with a quadruped robot without any hardware modification or external support. Despite the limitations posed by the quadruped’s hardware, the study considers this a perfect benchmark task to assess the shortcomings of sim- to-real techniques and unlock future avenues for the legged robotics community. Overall, this thesis demonstrates the potential of learning-based methods to design dynamic and robust controllers for legged robots while limiting the effort needed for sim-to-real transfer