39 research outputs found
Meta Adaptation using Importance Weighted Demonstrations
Imitation learning has gained immense popularity because of its high
sample-efficiency. However, in real-world scenarios, where the trajectory
distribution of most of the tasks dynamically shifts, model fitting on
continuously aggregated data alone would be futile. In some cases, the
distribution shifts, so much, that it is difficult for an agent to infer the
new task. We propose a novel algorithm to generalize on any related task by
leveraging prior knowledge on a set of specific tasks, which involves assigning
importance weights to each past demonstration. We show experiments where the
robot is trained from a diversity of environmental tasks and is also able to
adapt to an unseen environment, using few-shot learning. We also developed a
prototype robot system to test our approach on the task of visual navigation,
and experimental results obtained were able to confirm these suppositions
Simulation-based reinforcement learning for real-world autonomous driving
We use reinforcement learning in simulation to obtain a driving system
controlling a full-size real-world vehicle. The driving policy takes RGB images
from a single camera and their semantic segmentation as input. We use mostly
synthetic data, with labelled real-world data appearing only in the training of
the segmentation network.
Using reinforcement learning in simulation and synthetic data is motivated by
lowering costs and engineering effort.
In real-world experiments we confirm that we achieved successful sim-to-real
policy transfer. Based on the extensive evaluation, we analyze how design
decisions about perception, control, and training impact the real-world
performance
To Stir or Not to Stir:Online Estimation of Liquid Properties for Pouring Actions
Our brains are able to exploit coarse physical models of fluids to solve
everyday manipulation tasks. There has been considerable interest in developing
such a capability in robots so that they can autonomously manipulate fluids
adapting to different conditions. In this paper, we investigate the problem of
adaptation to liquids with different characteristics. We develop a simple
calibration task (stirring with a stick) that enables rapid inference of the
parameters of the liquid from RBG data. We perform the inference in the space
of simulation parameters rather than on physically accurate parameters. This
facilitates prediction and optimization tasks since the inferred parameters may
be fed directly to the simulator. We demonstrate that our "stirring" learner
performs better than when the robot is calibrated with pouring actions. We show
that our method is able to infer properties of three different liquids --
water, glycerin and gel -- and present experimental results by executing
stirring and pouring actions on a UR10. We believe that decoupling of the
training actions from the goal task is an important step towards simple,
autonomous learning of the behavior of different fluids in unstructured
environments.Comment: Presented at the Modeling the Physical World: Perception, Learning,
and Control Workshop (NeurIPS) 201
Distributional Actor-Critic Ensemble for Uncertainty-Aware Continuous Control
Uncertainty quantification is one of the central challenges for machine
learning in real-world applications. In reinforcement learning, an agent
confronts two kinds of uncertainty, called epistemic uncertainty and aleatoric
uncertainty. Disentangling and evaluating these uncertainties simultaneously
stands a chance of improving the agent's final performance, accelerating
training, and facilitating quality assurance after deployment. In this work, we
propose an uncertainty-aware reinforcement learning algorithm for continuous
control tasks that extends the Deep Deterministic Policy Gradient algorithm
(DDPG). It exploits epistemic uncertainty to accelerate exploration and
aleatoric uncertainty to learn a risk-sensitive policy. We conduct numerical
experiments showing that our variant of DDPG outperforms vanilla DDPG without
uncertainty estimation in benchmark tasks on robotic control and power-grid
optimization.Comment: 10 pages, 6 figures. Accepted to International Joint Conference on
Neural Networks (IJCNN 2022), July 18-23, Padua, Ital
Bridging Action Space Mismatch in Learning from Demonstrations
Learning from demonstrations (LfD) methods guide learning agents to a desired
solution using demonstrations from a teacher. While some LfD methods can handle
small mismatches in the action spaces of the teacher and student, here we
address the case where the teacher demonstrates the task in an action space
that can be substantially different from that of the student -- thereby
inducing a large action space mismatch. We bridge this gap with a framework,
Morphological Adaptation in Imitation Learning (MAIL), that allows training an
agent from demonstrations by other agents with significantly different
morphologies (from the student or each other). MAIL is able to learn from
suboptimal demonstrations, so long as they provide some guidance towards a
desired solution. We demonstrate MAIL on challenging household cloth
manipulation tasks and introduce a new DRY CLOTH task -- cloth manipulation in
3D task with obstacles. In these tasks, we train a visual control policy for a
robot with one end-effector using demonstrations from a simulated agent with
two end-effectors. MAIL shows up to 27% improvement over LfD and non-LfD
baselines. It is deployed to a real Franka Panda robot, and can handle multiple
variations in cloth properties (color, thickness, size, material) and pose
(rotation and translation). We further show generalizability to transfers from
n-to-m end-effectors, in the context of a simple rearrangement task
World Model Based Sim2Real Transfer for Visual Navigation
Sim2Real transfer has gained popularity because it helps transfer from
inexpensive simulators to real world. This paper presents a novel system that
fuses components in a traditional \textit{World Model} into a robust system,
trained entirely within a simulator, that \textit{Zero-Shot} transfers to the
real world. To facilitate transfer, we use an intermediary representation that
are based on \textit{Bird's Eye View (BEV)} images. Thus, our robot learns to
navigate in a simulator by first learning to translate from complex
\textit{First-Person View (FPV)} based RGB images to BEV representations, then
learning to navigate using those representations. Later, when tested in the
real world, the robot uses the perception model that translates FPV-based RGB
images to embeddings that are used by the downstream policy. The incorporation
of state-checking modules using \textit{Anchor images} and \textit{Mixture
Density LSTM} not only interpolates uncertain and missing observations but also
enhances the robustness of the model when exposed to the real-world
environment. We trained the model using data collected using a
\textit{Differential drive} robot in the CARLA simulator. Our methodology's
effectiveness is shown through the deployment of trained models onto a
\textit{Real world Differential drive} robot. Lastly we release a comprehensive
codebase, dataset and models for training and deployment that are available to
the public.Comment: Under Review at the International Conference on Robotics and
Automation 2024; Accepted at NeurIPS 2023, Robot Learning Worksho