471 research outputs found
Learning Image-Conditioned Dynamics Models for Control of Under-actuated Legged Millirobots
Millirobots are a promising robotic platform for many applications due to
their small size and low manufacturing costs. Legged millirobots, in
particular, can provide increased mobility in complex environments and improved
scaling of obstacles. However, controlling these small, highly dynamic, and
underactuated legged systems is difficult. Hand-engineered controllers can
sometimes control these legged millirobots, but they have difficulties with
dynamic maneuvers and complex terrains. We present an approach for controlling
a real-world legged millirobot that is based on learned neural network models.
Using less than 17 minutes of data, our method can learn a predictive model of
the robot's dynamics that can enable effective gaits to be synthesized on the
fly for following user-specified waypoints on a given terrain. Furthermore, by
leveraging expressive, high-capacity neural network models, our approach allows
for these predictions to be directly conditioned on camera images, endowing the
robot with the ability to predict how different terrains might affect its
dynamics. This enables sample-efficient and effective learning for locomotion
of a dynamic legged millirobot on various terrains, including gravel, turf,
carpet, and styrofoam. Experiment videos can be found at
https://sites.google.com/view/imageconddy
Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers
We propose to address quadrupedal locomotion tasks using Reinforcement
Learning (RL) with a Transformer-based model that learns to combine
proprioceptive information and high-dimensional depth sensor inputs. While
learning-based locomotion has made great advances using RL, most methods still
rely on domain randomization for training blind agents that generalize to
challenging terrains. Our key insight is that proprioceptive states only offer
contact measurements for immediate reaction, whereas an agent equipped with
visual sensory observations can learn to proactively maneuver environments with
obstacles and uneven terrain by anticipating changes in the environment many
steps ahead. In this paper, we introduce LocoTransformer, an end-to-end RL
method for quadrupedal locomotion that leverages a Transformer-based model for
fusing proprioceptive states and visual observations. We evaluate our method in
challenging simulated environments with different obstacles and uneven terrain.
We show that our method obtains significant improvements over policies with
only proprioceptive state inputs, and that Transformer-based models further
improve generalization across environments. Our project page with videos is at
https://RchalYang.github.io/LocoTransformer .Comment: Our project page with videos is at
https://RchalYang.github.io/LocoTransforme
Subwords as Skills: Tokenization for Sparse-Reward Reinforcement Learning
Exploration in sparse-reward reinforcement learning is difficult due to the
requirement of long, coordinated sequences of actions in order to achieve any
reward. Moreover, in continuous action spaces there are an infinite number of
possible actions, which only increases the difficulty of exploration. One class
of methods designed to address these issues forms temporally extended actions,
often called skills, from interaction data collected in the same domain, and
optimizes a policy on top of this new action space. Typically such methods
require a lengthy pretraining phase, especially in continuous action spaces, in
order to form the skills before reinforcement learning can begin. Given prior
evidence that the full range of the continuous action space is not required in
such tasks, we propose a novel approach to skill-generation with two
components. First we discretize the action space through clustering, and second
we leverage a tokenization technique borrowed from natural language processing
to generate temporally extended actions. Such a method outperforms baselines
for skill-generation in several challenging sparse-reward domains, and requires
orders-of-magnitude less computation in skill-generation and online rollouts
Towards Autonomous Pipeline Inspection with Hierarchical Reinforcement Learning
Inspection and maintenance are two crucial aspects of industrial pipeline
plants. While robotics has made tremendous progress in the mechanic design of
in-pipe inspection robots, the autonomous control of such robots is still a big
open challenge due to the high number of actuators and the complex manoeuvres
required. To address this problem, we investigate the usage of Deep
Reinforcement Learning for achieving autonomous navigation of in-pipe robots in
pipeline networks with complex topologies. Moreover, we introduce a
hierarchical policy decomposition based on Hierarchical Reinforcement Learning
to learn robust high-level navigation skills. We show that the hierarchical
structure introduced in the policy is fundamental for solving the navigation
task through pipes and necessary for achieving navigation performances superior
to human-level control
- …