Toward Real-Time Decentralized Reinforcement Learning using Finite Support Basis Functions

This paper addresses the design and implementation of complex Reinforcement Learning (RL) behaviors where multi-dimensional action spaces are involved, as well as the need to execute the behaviors in real-time using robotic platforms with limited computational resources and training times. For this purpose, we propose the use of decentralized RL, in combination with finite support basis functions as alternatives to Gaussian RBF, in order to alleviate the effects of the curse of dimensionality on the action and state spaces respectively, and to reduce the computation time. As testbed, a RL based controller for the in-walk kick in NAO robots, a challenging and critical problem for soccer robotics, is used. The reported experiments show empirically that our solution saves up to 99.94% of execution time and 98.82% of memory consumption during execution, without diminishing performance compared to classical approaches.Comment: Accepted in the RoboCup Symposium 2017. Final version will be published at Springe

Walking Patterns for Real Time Path Planning Simulation of Humanoids

International audienceWe present here a detailed description of the walking algorithm that was designed for 3D simulation of locomotion and path planning of humanoid robots. The walking patterns described were implemented on NAO humanoid models that are used in the 3D simulation league of RoboCup to play soccer. The locomotion algorithm is based on the well known 3D-LIP model that consists of defining walking primitives of the center of mass, keeping its height constant and assuming no torque at the support foot. This paper proposes to detail how to connect the walking primitives, especially at the start of the walk. The second added value of this work resides in the rotation walking primitives that are generated differently from the linear translation walking primitives. This enables the robot to achieve fast rotation on the spot or about a center located on the longitudinal axis. The paper also addresses the issue of re-entrance, i.e. how to take into account a new walking request in real time without waiting for the end of the current walk

Estimation of the transverse sway motion of a biped robot during the march

Debido al balanceo generado por la dinámica natural y perturbaciones durante la marcha de un robot humanoide, es difícil predecir su postura en determinado instante a lo largo de la misma, complicando así el desarrollo de tareas de manipulación, cooperación, evasión de obstáculos, retroalimentación servo-visual, entre otras. En este documento se describe una metodología para predecir el movimiento de balanceo en el plano transversal del robot, a partir de la trayectoria de un punto fijo en su estructura mecánica. Se considera el estudio de dos modelos matemáticos para aproximar el movimiento de balanceo del humanoide: aproximación mediante una función sinusoidal y aproximación por series de Fourier. En ambos casos, es necesario el conocimiento de los parámetros involucrados del modelo, por lo que se implementan tres técnicas de aproximación paramétrica: mínimos cuadrados e identificación algebraica para el caso de la aproximación sinusoidal, y el cálculo de coeficientes para el caso de las series de Fourier. Para validar la metodología, se lleva a cabo el seguimiento del robot en tiempo real puesto que la trayectoria del punto de interés es afectada por diversos factores como la fricción, inclinación e imperfecciones de la superficie, el estado de conservación del robot, entre otros. A partir de diversos experimentos, se desarrolla una comparación cuantitativa entre las diferentes aproximaciones para verificar aquel que mejor reproduce a la dinámica del balanceo del robot.Peer Reviewe