Hierarchical Reinforcement Learning for Precise Soccer Shooting Skills using a Quadrupedal Robot

We address the problem of enabling quadrupedal robots to perform precise shooting skills in the real world using reinforcement learning. Developing algorithms to enable a legged robot to shoot a soccer ball to a given target is a challenging problem that combines robot motion control and planning into one task. To solve this problem, we need to consider the dynamics limitation and motion stability during the control of a dynamic legged robot. Moreover, we need to consider motion planning to shoot the hard-to-model deformable ball rolling on the ground with uncertain friction to a desired location. In this paper, we propose a hierarchical framework that leverages deep reinforcement learning to train (a) a robust motion control policy that can track arbitrary motions and (b) a planning policy to decide the desired kicking motion to shoot a soccer ball to a target. We deploy the proposed framework on an A1 quadrupedal robot and enable it to accurately shoot the ball to random targets in the real world.Comment: Accepted to 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2022

Legged Robots for Object Manipulation: A Review

Legged robots can have a unique role in manipulating objects in dynamic, human-centric, or otherwise inaccessible environments. Although most legged robotics research to date typically focuses on traversing these challenging environments, many legged platform demonstrations have also included "moving an object" as a way of doing tangible work. Legged robots can be designed to manipulate a particular type of object (e.g., a cardboard box, a soccer ball, or a larger piece of furniture), by themselves or collaboratively. The objective of this review is to collect and learn from these examples, to both organize the work done so far in the community and highlight interesting open avenues for future work. This review categorizes existing works into four main manipulation methods: object interactions without grasping, manipulation with walking legs, dedicated non-locomotive arms, and legged teams. Each method has different design and autonomy features, which are illustrated by available examples in the literature. Based on a few simplifying assumptions, we further provide quantitative comparisons for the range of possible relative sizes of the manipulated object with respect to the robot. Taken together, these examples suggest new directions for research in legged robot manipulation, such as multifunctional limbs, terrain modeling, or learning-based control, to support a number of new deployments in challenging indoor/outdoor scenarios in warehouses/construction sites, preserved natural areas, and especially for home robotics.Comment: Preprint of the paper submitted to Frontiers in Mechanical Engineerin

Optimization of Parametrised Kicking Motion for Humanoid Soccer Player

International audienceThis paper presents a detailed description of the optimization process used to increase the kicking skills of humanoid players. The kicking movement consists of making a step forward to put the support foot next to the ball to kick, then to execute the kicking motion by rocking the leg that must strike the ball. The rocking motion of the leg passes through three positions; the first position is the leg being raised backward, the second is the leg in position of strike, and the third is the leg forward to sweep the ball. The optimization process is an evolution of skills parameters guided by a evaluation function that aims to send the ball as far as possible. To optimize the kick a rocking motion of the torso is also introduced. The parameters used for each one of the three kicking positions are the angle of the torso, the angle of the foot sole, the longitudinal and the vertical coordinates of the foot toe. Thanks to this technique it is possible to increase the distance of the kicked ball by 34%