CAJun: Continuous Adaptive Jumping using a Learned Centroidal Controller

We present CAJun, a novel hierarchical learning and control framework that enables legged robots to jump continuously with adaptive jumping distances. CAJun consists of a high-level centroidal policy and a low-level leg controller. In particular, we use reinforcement learning (RL) to train the centroidal policy, which specifies the gait timing, base velocity, and swing foot position for the leg controller. The leg controller optimizes motor commands for the swing and stance legs according to the gait timing to track the swing foot target and base velocity commands using optimal control. Additionally, we reformulate the stance leg optimizer in the leg controller to speed up policy training by an order of magnitude. Our system combines the versatility of learning with the robustness of optimal control. By combining RL with optimal control methods, our system achieves the versatility of learning while enjoys the robustness from control methods, making it easily transferable to real robots. We show that after 20 minutes of training on a single GPU, CAJun can achieve continuous, long jumps with adaptive distances on a Go1 robot with small sim-to-real gaps. Moreover, the robot can jump across gaps with a maximum width of 70cm, which is over 40% wider than existing methods.Comment: Please visit https://yxyang.github.io/cajun/ for additional result

In silico case studies of compliant robots: AMARSI deliverable 3.3

In the deliverable 3.2 we presented how the morphological computing ap- proach can significantly facilitate the control strategy in several scenarios, e.g. quadruped locomotion, bipedal locomotion and reaching. In particular, the Kitty experimental platform is an example of the use of morphological computation to allow quadruped locomotion. In this deliverable we continue with the simulation studies on the application of the different morphological computation strategies to control a robotic system

Learning dynamic motor skills for terrestrial locomotion

The use of Deep Reinforcement Learning (DRL) has received significantly increased attention from researchers within the robotics field following the success of AlphaGo, which demonstrated the superhuman capabilities of deep reinforcement algorithms in terms of solving complex tasks by beating professional GO players. Since then, an increasing number of researchers have investigated the potential of using DRL to solve complex high-dimensional robotic tasks, such as legged locomotion, arm manipulation, and grasping, which are difficult tasks to solve using conventional optimization approaches. Understanding and recreating various modes of terrestrial locomotion has been of long-standing interest to roboticists. A large variety of applications, such as rescue missions, disaster responses and science expeditions, strongly demand mobility and versatility in legged locomotion to enable task completion. In order to create useful physical robots, it is necessary to design controllers to synthesize the complex locomotion behaviours observed in humans and other animals. In the past, legged locomotion was mainly achieved via analytical engineering approaches. However, conventional analytical approaches have their limitations, as they require relatively large amounts of human effort and knowledge. Machine learning approaches, such as DRL, require less human effort compared to analytical approaches. The project conducted for this thesis explores the feasibility of using DRL to acquire control policies comparable to, or better than, those acquired through analytical approaches while requiring less human effort. In this doctoral thesis, we developed a Multi-Expert Learning Architecture (MELA) that uses DRL to learn multi-skill control policies capable of synthesizing a diverse set of dynamic locomotion behaviours for legged robots. We first proposed a novel DRL framework for the locomotion of humanoid robots. The proposed learning framework is capable of acquiring robust and dynamic motor skills for humanoids, including balancing, walking, standing-up fall recovery. We subsequently improved upon the learning framework and design a novel multi-expert learning architecture that is capable of fusing multiple motor skills together in a seamless fashion and ultimately deploy this framework on a real quadrupedal robot. The successful deployment of learned control policies on a real quadrupedal robot demonstrates the feasibility of using an Artificial Intelligence (AI) based approach for real robot motion control

Optimization-based multi-contact motion planning for legged robots

For legged robots, generating dynamic and versatile motions is essential for interacting with complex and ever-changing environments. So far, robots that routinely operate reliably over rough terrains remains an elusive goal. Yet the primary promise of legged locomotion is to replace humans and animals in performing tedious and menial tasks, without requiring changes in the environment as wheeled robots do. A necessary step towards this goal is to endow robots with capabilities to reason about contacts but this vital skill is currently missing. An important justification for this is that contact phenomena are inherently non-smooth and non-convex. As a result, posing and solving problems involving contacts is non-trivial. Optimization-based motion planning constitutes a powerful paradigm to this end. Consequently, this thesis considers the problem of generating motions in contact-rich situations. Specifically, we introduce several methods that compute dynamic and versatile motion plans from a holistic optimization perspective based on trajectory optimization techniques. The advantage is that the user needs to provide a high-level task description in the form of an objective function only. Subsequently, the methods output a detailed motion plan—that includes contact locations, timings, gait patterns—that optimally achieves the high-level task. Initially, we assume that such a motion plan is available, and we investigate the relevant control problem. The problem is to track a nominal motion plan as close as possible given external disturbances by computing inputs for the robot. Thus, this stage typically follows the motion planning stage. Additionally, this thesis presents methods that do not necessarily require a separate control stage by computing the controller structure automatically. Afterwards, we proceed to the main parts of this thesis. First, assuming a pre-specified contact sequence, we formulate a trajectory optimization method reminiscent of hybrid approaches. Its backbone is a high-accuracy integrator, enabling reliable long-term motion planning while satisfying both translational and rotational dynamics. We utilize it to compute motion plans for a hopper traversing rough terrains—with gaps and obstacles—and performing explosive motions, like a somersault. Subsequently, we provide a discussion on how to extend the method when the contact sequence is unspecified. In the next chapter, we increase the complexity of the problem in many aspects. First, we formulate the problem in joint-level utilizing full dynamics and kinematics models. Second, we assume a contact-implicit perspective, i.e. decisions about contacts are implicitly defined in the problem’s formulation rather than defined as explicit contact modes. As a result, pre-specification of the contact interactions is not required, like the order by which the feet contact the ground for a quadruped robot model and the respective timings. Finally, we extend the classical rigid contact model to surfaces with soft and slippery properties. We quantitatively evaluate our proposed framework by performing comparisons against the rigid model and an alternative contact-implicit framework. Furthermore, we compute motion plans for a high-dimensional quadruped robot in a variety of terrains exhibiting the enhanced properties. In the final study, we extend the classical Differential Dynamic Programming algorithm to handle systems defined by implicit dynamics. While this can be of interest in its own right, our particular application is computing motion plans in contact-rich settings. Compared to the method presented in the previous chapter, this formulation enables experiencing contacts with all body parts in a receding horizon fashion, albeit with limited contact discovery capabilities. We demonstrate the properties of our proposed extension by comparing implicit and explicit models and generating motion plans for a single-legged robot with multiple contacts both for trajectory optimization and receding horizon settings. We conclude this thesis by providing insights and limitations of the proposed methods, and possible future directions that can improve and extend aspects of the presented work

Legged Robots for Object Manipulation: A Review

Legged robots can have a unique role in manipulating objects in dynamic, human-centric, or otherwise inaccessible environments. Although most legged robotics research to date typically focuses on traversing these challenging environments, many legged platform demonstrations have also included "moving an object" as a way of doing tangible work. Legged robots can be designed to manipulate a particular type of object (e.g., a cardboard box, a soccer ball, or a larger piece of furniture), by themselves or collaboratively. The objective of this review is to collect and learn from these examples, to both organize the work done so far in the community and highlight interesting open avenues for future work. This review categorizes existing works into four main manipulation methods: object interactions without grasping, manipulation with walking legs, dedicated non-locomotive arms, and legged teams. Each method has different design and autonomy features, which are illustrated by available examples in the literature. Based on a few simplifying assumptions, we further provide quantitative comparisons for the range of possible relative sizes of the manipulated object with respect to the robot. Taken together, these examples suggest new directions for research in legged robot manipulation, such as multifunctional limbs, terrain modeling, or learning-based control, to support a number of new deployments in challenging indoor/outdoor scenarios in warehouses/construction sites, preserved natural areas, and especially for home robotics.Comment: Preprint of the paper submitted to Frontiers in Mechanical Engineerin

Sabertooth: A High Mobility Quadrupedal Robot Platform

Team Sabertooth aimed to design and realize an innovative high mobility, quadrupedal robot platform capable of delivering a payload over terrain otherwise impassable by wheeled vehicles at a speed of 5 feet per second. The robot uses a spring system in each of its legs for energy efficient locomotion. The 4ft x 3ft x 3ft freestanding four legged robot weighs approximately 300 pounds with an additional payload capacity of 30 pounds. An important feature of the robot is the passive, two degree of freedom body joint which allows flexibility in terms of robot motions for going around tight corners and ascending stairs. A distributed control and software architecture is used for world mapping, path planning and motion control

Sabertooth: A High Mobility Quadrupedal Robot Platform

Team Sabertooth aimed to design and realize an innovative high mobility, quadrupedal robot capable of delivering a payload over terrain impassable by wheeled vehicles at a speed of 5fps. The robot is designed to ascend and descend stairs. The robot uses a spring system in each of its legs for energy efficient locomotion. The 4\u27x3\u27x3\u27 freestanding four legged robot weighs approximately 300lbs with an additional payload capacity of 30lbs. The passive two degree of freedom body joint allows flexibility in terms of robot motion for going around tight corners and ascending stairs. The system integrates sensors for staircase recognition, obstacle avoidance, and distance calculation. A distributed control and software architecture is used for world mapping, path planning and motion control

Fast Sensing and Adaptive Actuation for Robust Legged Locomotion

Robust legged locomotion in complex terrain demands fast perturbation detection and reaction. In animals, due to the neural transmission delays, the high-level control loop involving the brain is absent from mitigating the initial disturbance. Instead, the low-level compliant behavior embedded in mechanics and the mid-level controllers in the spinal cord are believed to provide quick response during fast locomotion. Still, it remains unclear how these low- and mid-level components facilitate robust locomotion. This thesis aims to identify and characterize the underlining elements responsible for fast sensing and actuation. To test individual elements and their interplay, several robotic systems were implemented. The implementations include active and passive mechanisms as a combination of elasticities and dampers in multi-segment robot legs, central pattern generators inspired by intraspinal controllers, and a synthetic robotic version of an intraspinal sensor. The first contribution establishes the notion of effective damping. Effective damping is defined as the total energy dissipation during one step, which allows quantifying how much ground perturbation is mitigated. Using this framework, the optimal damper is identified as viscous and tunable. This study paves the way for integrating effective dampers to legged designs for robust locomotion. The second contribution introduces a novel series elastic actuation system. The proposed system tackles the issue of power transmission over multiple joints, while featuring intrinsic series elasticity. The design is tested on a hopper with two more elastic elements, demonstrating energy recuperation and enhanced dynamic performance. The third contribution proposes a novel tunable damper and reveals its influence on legged hopping. A bio-inspired slack tendon mechanism is implemented in parallel with a spring. The tunable damping is rigorously quantified on a central-pattern-generator-driven hopping robot, which reveals the trade-off between locomotion robustness and efficiency. The last contribution explores the intraspinal sensing hypothesis of birds. We speculate that the observed intraspinal structure functions as an accelerometer. This accelerometer could provide fast state feedback directly to the adjacent central pattern generator circuits, contributing to birds’ running robustness. A biophysical simulation framework is established, which provides new perspectives on the sensing mechanics of the system, including the influence of morphologies and material properties. Giving an overview of the hierarchical control architecture, this thesis investigates the fast sensing and actuation mechanisms in several control layers, including the low-level mechanical response and the mid-level intraspinal controllers. The contributions of this work provide new insight into animal loco-motion robustness and lays the foundation for future legged robot design

Quadrupedal Robotics Platform

The purpose of this project is to take lessons learned from past MQPs, current industry products, and current research to create a quadrupedal platform capable of attaining unsupported walking. The team designed the platform to utilize series elastic actuation, force-sensing feet, and custom hardware to create a modular and easily expandable platform for future project use. CNC milling and water-jetting were used to manufacture the complete platform which was then vigorously tested under its own weight to determine its capabilities

Mechanism and Behaviour Co-optimisation of High Performance Mobile Robots

Mobile robots do not display the level of physical performance one would expect, given the specifications of their hardware. This research is based on the idea that their poor performance is at least partly due to their design, and proposes an optimisation approach for the design of high-performance mobile robots. The aim is to facilitate the design process, and produce versatile and robust robots that can exploit the maximum potential of today's technology. This can be achieved by a systematic optimisation study that is based on careful modelling of the robot's dynamics and its limitations, and takes into consideration the performance requirements that the robot is designed to meet. The approach is divided into two parts: (1) an optimisation framework, and (2) an optimisation methodology. In the framework, designs that can perform a large set of tasks are sought, by simultaneously optimising the design and the behaviours to perform them. The optimisation methodology consists of several stages, where various techniques are used for determining the design's most important parameters, and for maximising the chances of finding the best possible design based on the designer's evaluation criteria. The effectiveness of the optimisation approach is proved via a specific case-study of a high-performance balancing and hopping monopedal robot. The outcome is a robot design and a set of optimal behaviours that can meet several performance requirements of conflicting nature, by pushing the hardware to its limits in a safe way. The findings of this research demonstrate the importance of using realistic models, and taking into consideration the tasks that the robot is meant to perform in the design process