SYNLOCO: Synthesizing Central Pattern Generator and Reinforcement Learning for Quadruped Locomotion

The Central Pattern Generator (CPG) is adept at generating rhythmic gait patterns characterized by consistent timing and adequate foot clearance. Yet, its open-loop configuration often compromises the system's control performance in response to environmental variations. On the other hand, Reinforcement Learning (RL), celebrated for its model-free properties, has gained significant traction in robotics due to its inherent adaptability and robustness. However, initiating traditional RL approaches from the ground up presents computational challenges and a heightened risk of converging to suboptimal local minima. In this paper, we propose an innovative quadruped locomotion framework, SYNLOCO, by synthesizing CPG and RL that can ingeniously integrate the strengths of both methods, enabling the development of a locomotion controller that is both stable and natural. Furthermore, we introduce a set of performance-driven reward metrics that augment the learning of locomotion control. To optimize the learning trajectory of SYNLOCO, a two-phased training strategy is presented. Our empirical evaluation, conducted on a Unitree GO1 robot under varied conditions--including distinct velocities, terrains, and payload capacities--showcases SYNLOCO's ability to produce consistent and clear-footed gaits across diverse scenarios. The developed controller exhibits resilience against substantial parameter variations, underscoring its potential for robust real-world applications.Comment: 7 Page

Design, implementation and integration of an autonomy payload for a quadruped-legged robot

Aquesta tesi final de màster versa sobre la descripció, disseny i implantació d’un conjunt de millores que fan possible l’autonomia d’un robot quadrúpede. El conjunt de millores descrites es troben separades en dos grups per millorar la comprensió el lector. Aquests són: Hardware i Software. Durant la primera part, es descriu al complet: el dimensionament, el disseny i la implementació dels elements mecànics i electrònics (bateries, ordinador a bord, càmera, i elements passius) que formen part de la millora en el hardware. Aquesta inclou el disseny i impressió en 3D, així com el procés d’acoblament al robot original. Pel que fa a la millora en "Software"; es planteja, descriu, i implementa un observador d’estat amb la intenció futura d’implementar-lo al mateix robot. L’observador es descriu mitjançant l’ús d’un conegut algoritme de fusió de dades anomenat Filtre de Kalman (on, en aquest cas, se n’utilitza la versió estesa). Per provar l’observador dissenyat, es realitza una simulació amb dades generades artificialment a partir d’un model cinemàtic. A continuació, tot i que la futura implantació es pretén que sigui al robot quadrúpede, per poder prova el filtre amb dades reals, es registren les dades del vol d’un dron fent ús de la comunicació ROS1 2 Galactic. De les mateixes dades, es torna a experimentar amb el filtre, però, aquest cop, fent ús del registre de dades del controlador del dron (model PX4). El treball conclou amb quatre estudis addicionals sobre la tasca: econòmic, temporal, social i mediambiental; així com amb unes conclusions que inclouen propostes de futures tasques a portar a capThis master thesis is based on the description, design and implementation of a set of improve- ments that make possible the autonomy of a quadruped robot. The set of improvements described are separated into two large groups that are disassociated in this memory as Hardware and Software, in order to improve the reader’s comprehension. During the first part, the dimensioning, design and implementation of the mechanical and elec- tronic elements (batteries, on-board computer, camera, and passive elements) that are part of the improvement in the hardware is described in full. This part, in turn, includes the design and 3D printing, as well as the assembly process to the original robot. As for the improvement in software, a state observer is proposed, described and implemented with the future intention of being implemented in the robot itself. This is described through the use of a well-known data fusion algorithm called Kalman Filter (which, in this case, is used through its extended version). To test the designed observer, three experiments are performed. The first one is based on a simulation with artificially generated data from a kinematic model. The second is based on real data collected from the flight of a drone. It also uses the ROS2 2 Galactic operating system to establish communication between data. Finally, the third uses data extracted from the drone’s own controller (PX4 model). The thesis ends with four additional studies: economic, temporal, social, and environmental, as well as conclusions that incorporate proposals for future lines of workEsta tesis final de máster se basa en la descripción, diseño e implementación de un conjunto de mejoras que hacen posible la autonomía de un robot cuadrúpedo. El conjunto de mejoras descritas se encuentran separadas en dos grandes grupos que se desasocian en esta memoria como Hardware y Software, a fin de mejorar la compresión del lector. Durante la primera parte, se describe al completo: el dimensionamiento, el diseño y la imple- mentación de los elementos mecánicos y electrónicos (baterías, ordenador a bordo, cámara, y elementos pasivos) que forman parte de la mejora en el hardware. Esta parte, a su vez, incluye el diseño e impresión en 3D, así como el proceso de ensamblaje al robot original. En cuanto a la mejora en software; se plantea, describe e implementa, un observador de estado con la intención futura de ser implementado en el mismo robot. Este se describe mediante el uso de un conocido algoritmo de fusión de datos llamado Filtro de Kalman (que, en este caso, es usado mediante su versión extendida). Para probar el observador diseñado, se realizan tres experimentos. El primero se basa en una simulación con datos generados artificialmente a partir de un modelo cinemático. El segundo, aunque la futura implementación está pensada sobre el robot cuadrúpedo, se realiza utilizando datos reales recogidos del vuelo de un dron, haciendo uso de la comunicación ROS3 2 Galactic. Por último, el tercero utiliza datos extraídos del propio controlador del dron (modelo PX4). La tesis finaliza con cuatro estudios adicionales sobre la misma: económico, temporal, social y medioambiental; así como, con unas conclusiones que incorporan propuestas para futuras líneas de trabaj

Contact Optimization for Non-Prehensile Loco-Manipulation via Hierarchical Model Predictive Control

Recent studies on quadruped robots have focused on either locomotion or mobile manipulation using a robotic arm. Legged robots can manipulate heavier and larger objects using non-prehensile manipulation primitives, such as planar pushing, to drive the object to the desired location. In this paper, we present a novel hierarchical model predictive control (MPC) for contact optimization of the manipulation task. Using two cascading MPCs, we split the loco-manipulation problem into two parts: the first to optimize both contact force and contact location between the robot and the object, and the second to regulate the desired interaction force through the robot locomotion. Our method is successfully validated in both simulation and hardware experiments. While the baseline locomotion MPC fails to follow the desired trajectory of the object, our proposed approach can effectively control both object's position and orientation with minimal tracking error. This capability also allows us to perform obstacle avoidance for both the robot and the object during the loco-manipulation task.Comment: 7 pages, 9 figure

CAJun: Continuous Adaptive Jumping using a Learned Centroidal Controller

We present CAJun, a novel hierarchical learning and control framework that enables legged robots to jump continuously with adaptive jumping distances. CAJun consists of a high-level centroidal policy and a low-level leg controller. In particular, we use reinforcement learning (RL) to train the centroidal policy, which specifies the gait timing, base velocity, and swing foot position for the leg controller. The leg controller optimizes motor commands for the swing and stance legs according to the gait timing to track the swing foot target and base velocity commands using optimal control. Additionally, we reformulate the stance leg optimizer in the leg controller to speed up policy training by an order of magnitude. Our system combines the versatility of learning with the robustness of optimal control. By combining RL with optimal control methods, our system achieves the versatility of learning while enjoys the robustness from control methods, making it easily transferable to real robots. We show that after 20 minutes of training on a single GPU, CAJun can achieve continuous, long jumps with adaptive distances on a Go1 robot with small sim-to-real gaps. Moreover, the robot can jump across gaps with a maximum width of 70cm, which is over 40% wider than existing methods.Comment: Please visit https://yxyang.github.io/cajun/ for additional result

Learning to See Physical Properties with Active Sensing Motor Policies

Knowledge of terrain's physical properties inferred from color images can aid in making efficient robotic locomotion plans. However, unlike image classification, it is unintuitive for humans to label image patches with physical properties. Without labeled data, building a vision system that takes as input the observed terrain and predicts physical properties remains challenging. We present a method that overcomes this challenge by self-supervised labeling of images captured by robots during real-world traversal with physical property estimators trained in simulation. To ensure accurate labeling, we introduce Active Sensing Motor Policies (ASMP), which are trained to explore locomotion behaviors that increase the accuracy of estimating physical parameters. For instance, the quadruped robot learns to swipe its foot against the ground to estimate the friction coefficient accurately. We show that the visual system trained with a small amount of real-world traversal data accurately predicts physical parameters. The trained system is robust and works even with overhead images captured by a drone despite being trained on data collected by cameras attached to a quadruped robot walking on the ground.Comment: In CoRL 2023. Website: https://gmargo11.github.io/active-sensing-loco

Coupling Vision and Proprioception for Navigation of Legged Robots

We exploit the complementary strengths of vision and proprioception to develop a point-goal navigation system for legged robots, called VP-Nav. Legged systems are capable of traversing more complex terrain than wheeled robots, but to fully utilize this capability, we need a high-level path planner in the navigation system to be aware of the walking capabilities of the low-level locomotion policy in varying environments. We achieve this by using proprioceptive feedback to ensure the safety of the planned path by sensing unexpected obstacles like glass walls, terrain properties like slipperiness or softness of the ground and robot properties like extra payload that are likely missed by vision. The navigation system uses onboard cameras to generate an occupancy map and a corresponding cost map to reach the goal. A fast marching planner then generates a target path. A velocity command generator takes this as input to generate the desired velocity for the walking policy. A safety advisor module adds sensed unexpected obstacles to the occupancy map and environment-determined speed limits to the velocity command generator. We show superior performance compared to wheeled robot baselines, and ablation studies which have disjoint high-level planning and low-level control. We also show the real-world deployment of VP-Nav on a quadruped robot with onboard sensors and computation. Videos at https://navigation-locomotion.github.ioComment: CVPR 2022 final version. Website at https://navigation-locomotion.github.i

Design Principles for a Family of Direct-Drive Legged Robots

This letter introduces Minitaur, a dynamically running and leaping quadruped, which represents a novel class of direct-drive (DD) legged robots. We present a methodology that achieves the well-known benefits of DD robot design (transparency, mechanical robustness/efficiency, high-actuation bandwidth, and increased specific power), affording highly energetic behaviors across our family of machines despite severe limitations in specific force. We quantify DD drivetrain benefits using a variety of metrics, compare our machines\u27 performance to previously reported legged platforms, and speculate on the potential broad-reaching value of “transparency” for legged locomotion. For more information: Kod*lab