Robust High-speed Running for Quadruped Robots via Deep Reinforcement Learning

Deep reinforcement learning has emerged as a popular and powerful way to develop locomotion controllers for quadruped robots. Common approaches have largely focused on learning actions directly in joint space, or learning to modify and offset foot positions produced by trajectory generators. Both approaches typically require careful reward shaping and training for millions of time steps, and with trajectory generators introduce human bias into the resulting control policies. In this paper, we instead explore learning foot positions in Cartesian space, which we track with impedance control, for a task of running as fast as possible subject to environmental disturbances. Compared with other action spaces, we observe less needed reward shaping, much improved sample efficiency, the emergence of natural gaits such as galloping and bounding, and ease of sim-to-sim transfer. Policies can be learned in only a few million time steps, even for challenging tasks of running over rough terrain with loads of over 100% of the nominal quadruped mass. Training occurs in PyBullet, and we perform a sim-to-sim transfer to Gazebo, where our quadruped is able to run at over 4 m/s without a load, and 3.5 m/s with a 10 kg load, which is over 83% of the nominal quadruped mass. Video results can be found at https://youtu.be/roE1vxpEWfw.Comment: arXiv admin note: text overlap with arXiv:2011.0708

CPG-RL: Learning Central Pattern Generators for Quadruped Locomotion

In this letter, we present a method for integrating central pattern generators (CPGs), i.e. systems of coupled oscillators, into the deep reinforcement learning (DRL) framework to produce robust and omnidirectional quadruped locomotion. The agent learns to directly modulate the intrinsic oscillator setpoints (amplitude and frequency) and coordinate rhythmic behavior among different oscillators. This approach also allows the use of DRL to explore questions related to neuroscience, namely the role of descending pathways, interoscillator couplings, and sensory feedback in gait generation. We train our policies in simulation and perform a sim-to-real transfer to the Unitree A1 quadruped, where we observe robust behavior to disturbances unseen during training, most notably to a dynamically added 13.75 kg load representing 115% of the nominal quadruped mass. We test several different observation spaces based on proprioceptive sensing and show that our framework is deployable with no domain randomization and very little feedback, where along with the oscillator states, it is possible to provide only contact booleans in the observation space. Video results can be found at https://youtu.be/xqXHLzLsEV4.Comment: Accepted for IEEE Robotics and Automation Letters, September 202

Cognitive Control and Bilingualism: The Bilingual Advantage Through the Lens of Dimensional Overlap

Past research shows that the bilingual experience may enhance cognitive executive function. In this experiment, we evaluated cognitive control in bilinguals relative to monolinguals by using a dimensional overlap model to predict performance in a task composed of Stroop and Simon stimuli. A group of 24 Spanish monolinguals and 24 bilinguals with differing first languages and all having Spanish as a second language (L2) did a picture naming task and a task composed of Stroop and Simon stimuli, where the effect of different overlap conditions (spatial/color) between stimuli and responses were examined. The tasks were performed in Spanish for both groups and performance was indexed with behavioral and electrophysiological measures. We hypothesized that the bilinguals’ daily language practice in L2 reflected overlap conditions similar to the Simon task. Both naming a picture in L2 and the Simon task would involve conflict at the response level. L2 picture naming entails interference between two potential oral responses, to name in L2 vs. L1 (correct vs. incorrect responses, respectively). Similarly, incongruent stimuli in the Simon task produce interference because the irrelevant dimension (spatial location) overlap with an incorrect response. In contrast, the manual Stroop task involves a different type of conflict between two overlapping stimulus dimensions (the ink color and the color meaning). We predicted for these reasons a superior performance in Simon tasks over Stroop tasks for bilinguals, while monolinguals were expected to have a similar performance in both tasks. We also expected to see a correlation between the performance on the picture naming task and the Simon task in bilinguals. However, the behavioral results did not confirm these hypotheses. In fact, both groups had similar congruency effects as measured by reaction times and error rates, and there was no correlation between the picture naming and Simon task in bilinguals. Despite this, the electrophysiological data suggested a relationship between the picture naming task and the P300 congruency effect in bilinguals. Our findings provide insights into the neurocognitive bases of language and serve as a research avenue for language behaviors in bilinguals.Spanish Government PID2019-111359GB-I00/SRA State Research Agency/10.13039/50110001103