SafeSteps: Learning Safer Footstep Planning Policies for Legged Robots via Model-Based Priors

We present a footstep planning policy for quadrupedal locomotion that is able to directly take into consideration a-priori safety information in its decisions. At its core, a learning process analyzes terrain patches, classifying each landing location by its kinematic feasibility, shin collision, and terrain roughness. This information is then encoded into a small vector representation and passed as an additional state to the footstep planning policy, which furthermore proposes only safe footstep location by applying a masked variant of the Proximal Policy Optimization (PPO) algorithm. The performance of the proposed approach is shown by comparative simulations on an electric quadruped robot walking in different rough terrain scenarios. We show that violations of the above safety conditions are greatly reduced both during training and the successive deployment of the policy, resulting in an inherently safer footstep planner. Furthermore, we show how, as a byproduct, fewer reward terms are needed to shape the behavior of the policy, which in return is able to achieve both better final performances and sample efficienc

Quadrupedal Footstep Planning using Learned Motion Models of a Black-Box Controller

Legged robots are increasingly entering new domains and applications, including search and rescue, inspection, and logistics. However, for such systems to be valuable in real-world scenarios, they must be able to autonomously and robustly navigate irregular terrains. In many cases, robots that are sold on the market do not provide such abilities, being able to perform only blind locomotion. Furthermore, their controller cannot be easily modified by the end-user, requiring a new and time-consuming control synthesis. In this work, we present a fast local motion planning pipeline that extends the capabilities of a black-box walking controller that is only able to track high-level reference velocities. More precisely, we learn a set of motion models for such a controller that maps high-level velocity commands to Center of Mass (CoM) and footstep motions. We then integrate these models with a variant of the A star algorithm to plan the CoM trajectory, footstep sequences, and corresponding high-level velocity commands based on visual information, allowing the quadruped to safely traverse irregular terrains at demand

Enforcing Constraints over Learned Policies via Nonlinear MPC: Application to the Pendubot

In recent years Reinforcement Learning (RL) has achieved remarkable results. Nonetheless RL algorithms prove to be unsuccessful in robotics applications where constraints satisfaction is involved, e.g. for safety. In this work we propose a control algorithm that allows to enforce constraints over a learned control policy. Hence we combine Nonlinear Model Predictive Control (NMPC) with control-state trajectories generated from the learned policy at each time step. We prove the effectiveness of our method on the Pendubot, a challenging underactuated robot

Leveraging Symmetry in RL-based Legged Locomotion Control

Model-free reinforcement learning is a promising approach for autonomously solving challenging robotics control problems, but faces exploration difficulty without information of the robot's kinematics and dynamics morphology. The under-exploration of multiple modalities with symmetric states leads to behaviors that are often unnatural and sub-optimal. This issue becomes particularly pronounced in the context of robotic systems with morphological symmetries, such as legged robots for which the resulting asymmetric and aperiodic behaviors compromise performance, robustness, and transferability to real hardware. To mitigate this challenge, we can leverage symmetry to guide and improve the exploration in policy learning via equivariance/invariance constraints. In this paper, we investigate the efficacy of two approaches to incorporate symmetry: modifying the network architectures to be strictly equivariant/invariant, and leveraging data augmentation to approximate equivariant/invariant actor-critics. We implement the methods on challenging loco-manipulation and bipedal locomotion tasks and compare with an unconstrained baseline. We find that the strictly equivariant policy consistently outperforms other methods in sample efficiency and task performance in simulation. In addition, symmetry-incorporated approaches exhibit better gait quality, higher robustness and can be deployed zero-shot in real-world experiments

Production of He-4 and (4) in Pb-Pb collisions at root(NN)-N-S=2.76 TeV at the LHC

Results on the production of He-4 and (4) nuclei in Pb-Pb collisions at root(NN)-N-S = 2.76 TeV in the rapidity range vertical bar y vertical bar <1, using the ALICE detector, are presented in this paper. The rapidity densities corresponding to 0-10% central events are found to be dN/dy4(He) = (0.8 +/- 0.4 (stat) +/- 0.3 (syst)) x 10(-6) and dN/dy4 = (1.1 +/- 0.4 (stat) +/- 0.2 (syst)) x 10(-6), respectively. This is in agreement with the statistical thermal model expectation assuming the same chemical freeze-out temperature (T-chem = 156 MeV) as for light hadrons. The measured ratio of (4)/He-4 is 1.4 +/- 0.8 (stat) +/- 0.5 (syst). (C) 2018 Published by Elsevier B.V.Peer reviewe

Learning-based methods for Robotic control

Robots nowadays are being employed in increasingly complex scenarios, where the number of possible assumptions that can be made to ease the control synthesis is getting considerably smaller compared to the past. In fact, back in the day control engineers could heavily rely on a static world assumption and on a perfect knowledge of the system dynamics, since robots were practically only confined in controlled assembly lines where everything was predetermined beforehand. Given these premises, it was fairly easy to synthesize control laws able to solve with high precision the programmed task. Recently, task complexity started to grow considerably with respect to the past, requiring a new type of controller able to adapt continuously to the unknown scenarios to be faced. Among all the new methods, learning-based control can be considered one of the most promising approaches in literature today. This thesis investigates the use of this new control technique in robotics. We start by giving some background materials on Machine Learning, discussing how we can learn a better dynamical model for the robot just from sensor data, or even directly synthesize a control law from experiences. Then, after a small excursus on Optimal Control we present our contributions in this novel field. Specifically, a learning-based feedback linearization controller is proposed to deal with model uncertainties in fully actuated robots. This novel technique is then extended to underactuated systems, where control is tremendously complicated by the impossibility in these robots to follow arbitrary trajectories which are not dynamically feasible, i.e. not generated by an exact knowledge of their models. Finally, we present a contribution in the field of Reinforcement Learning, an approach that is able to learn directly a controller for a given task just by a trial and error mechanism. As detailed in the first chapters, Reinforcement Learning does not assure arbitrary constraints satisfaction in the final learned controller, which limits tremendously its applicability on real platforms. For this aspect, we propose an online mechanism where Optimal Control is used to enhance the safety of the final control law

On-line learning for planning and control of underactuated robots with uncertain dynamics

We present an iterative approach for planning and controlling motions of underactuated robots with uncertain dynamics. At its core, there is a learning process which estimates the perturbations induced by the model uncertainty on the active and passive degrees of freedom. The generic iteration of the algorithm makes use of the learned data in both the planning phase, which is based on optimization, and the control phase, where partial feedback linearization of the active dofs is performed on the model updated on-line. The performance of the proposed approach is shown by comparative simulations and experiments on a Pendubot executing various types of swing-up maneuvers. Very few iterations are typically needed to generate dynamically feasible trajectories and the tracking control that guarantees their accurate execution, even in the presence of large model uncertainties

Learning feedback linearization control without torque measurements

Feedback Linearization (FL) allows the best control performance in executing a desired motion task when an accurate dynamic model of a fully actuated robot is available. However, due to residual parametric uncertainties and unmodeled dynamic effects, a complete cancellation of the nonlinear dynamics by feedback is hardly achieved in practice. In this paper, we summarize a novel learning framework aimed at improving online the torque correction necessary for obtaining perfect cancellation with a FL controller, using only joint position measurements. We extend then this framework to the class of underactuated robots controlled by Partial Feedback Linearization (PFL), where we simultaneously learn a feasible trajectory satisfying the boundary conditions on the desired motion while improving the associated tracking performance

An online learning procedure for feedback linearization control without torque measurements

By exploiting an a priori estimate of the dynamic model of a manipulator, it is possible to command joint torques which ideally realize a Feedback Linearization (FL) controller. The exact cancellation may nevertheless not be achieved due to model uncertainties and possible errors in the estimation of the dynamic coefficients. In this work, an online learning scheme for control based on FL is presented. By reading joint positions and joint velocities information only (without the use of any torque measurement), we are able to learn those model uncertain- ties and thus achieve perfect FL control. Simulations results on the popular KUKA LWR iiwa robot are reported to show the quality of the proposed approach

Dynamics Harmonic Analysis of Robotic Systems:Application in Data-Driven Koopman Modelling

We introduce the use of harmonic analysis to decompose the state space of symmetric robotic systems into orthogonal isotypic subspaces. These are lower-dimensional spaces that capture distinct, symmetric, and synergistic motions. For linear dynamics, we characterize how this decomposition leads to a subdivision of the dynamics into independent linear systems on each subspace, a property we term dynamics harmonic analysis (DHA). To exploit this property, we use Koopman operator theory to propose an equivariant deep-learning architecture that leverages the properties of DHA to learn a global linear model of the system dynamics. Our architecture, validated on synthetic systems and the dynamics of locomotion of a quadrupedal robot, exhibits enhanced generalization, sample efficiency, and interpretability, with fewer trainable parameters and computational costs.</p