113 research outputs found
Stable Motion Primitives via Imitation and Contrastive Learning
Learning from humans allows non-experts to program robots with ease, lowering
the resources required to build complex robotic solutions. Nevertheless, such
data-driven approaches often lack the ability to provide guarantees regarding
their learned behaviors, which is critical for avoiding failures and/or
accidents. In this work, we focus on reaching/point-to-point motions, where
robots must always reach their goal, independently of their initial state. This
can be achieved by modeling motions as dynamical systems and ensuring that they
are globally asymptotically stable. Hence, we introduce a novel Contrastive
Learning loss for training Deep Neural Networks (DNN) that, when used together
with an Imitation Learning loss, enforces the aforementioned stability in the
learned motions. Differently from previous work, our method does not restrict
the structure of its function approximator, enabling its use with arbitrary
DNNs and allowing it to learn complex motions with high accuracy. We validate
it using datasets and a real robot. In the former case, motions are 2 and 4
dimensional, modeled as first- and second-order dynamical systems. In the
latter, motions are 3, 4, and 6 dimensional, of first and second order, and are
used to control a 7DoF robot manipulator in its end effector space and joint
space. More details regarding the real-world experiments are presented in:
\url{https://youtu.be/OM-2edHBRfc}
Smooth Exploration for Robotic Reinforcement Learning
Reinforcement learning (RL) enables robots to learn skills from interactions
with the real world. In practice, the unstructured step-based exploration used
in Deep RL -- often very successful in simulation -- leads to jerky motion
patterns on real robots. Consequences of the resulting shaky behavior are poor
exploration, or even damage to the robot. We address these issues by adapting
state-dependent exploration (SDE) to current Deep RL algorithms. To enable this
adaptation, we propose two extensions to the original SDE, using more general
features and re-sampling the noise periodically, which leads to a new
exploration method generalized state-dependent exploration (gSDE). We evaluate
gSDE both in simulation, on PyBullet continuous control tasks, and directly on
three different real robots: a tendon-driven elastic robot, a quadruped and an
RC car. The noise sampling interval of gSDE permits to have a compromise
between performance and smoothness, which allows training directly on the real
robots without loss of performance. The code is available at
https://github.com/DLR-RM/stable-baselines3.Comment: Code: https://github.com/DLR-RM/stable-baselines3/ Training scripts:
https://github.com/DLR-RM/rl-baselines3-zoo
Interactive Imitation Learning in State-Space
Imitation Learning techniques enable programming the behavior of agents
through demonstrations rather than manual engineering. However, they are
limited by the quality of available demonstration data. Interactive Imitation
Learning techniques can improve the efficacy of learning since they involve
teachers providing feedback while the agent executes its task. In this work, we
propose a novel Interactive Learning technique that uses human feedback in
state-space to train and improve agent behavior (as opposed to alternative
methods that use feedback in action-space). Our method titled Teaching
Imitative Policies in State-space~(TIPS) enables providing guidance to the
agent in terms of `changing its state' which is often more intuitive for a
human demonstrator. Through continuous improvement via corrective feedback,
agents trained by non-expert demonstrators using TIPS outperformed the
demonstrator and conventional Imitation Learning agents.Comment: Presented at the 4th Conference on Robot Learning (CoRL) 2020, 11
pages, 4 figure
TrajFlow: Learning the Distribution over Trajectories
Predicting the future behaviour of people remains an open challenge for the
development of risk-aware autonomous vehicles. An important aspect of this
challenge is effectively capturing the uncertainty which is inherent to human
behaviour. This paper studies an approach for probabilistic motion forecasting
with improved accuracy in the predicted sample likelihoods. We are able to
learn multi-modal distributions over the motions of an agent solely from data,
while also being able to provide predictions in real-time. Our approach
achieves state-of-the-art results on the inD dataset when evaluated with the
standard metrics employed for motion forecasting. Furthermore, our approach
also achieves state-of-the-art results when evaluated with respect to the
likelihoods it assigns to its generated trajectories. Evaluations on artificial
datasets indicate that the distributions learned by our model closely
correspond to the true distributions observed in data and are not as prone
towards being over-confident in a single outcome in the face of uncertainty
Learning from Few Demonstrations with Frame-Weighted Motion Generation
Learning from Demonstration (LfD) enables robots to acquire versatile skills
by learning motion policies from human demonstrations. It endows users with an
intuitive interface to transfer new skills to robots without the need for
time-consuming robot programming and inefficient solution exploration. During
task executions, the robot motion is usually influenced by constraints imposed
by environments. In light of this, task-parameterized LfD (TP-LfD) encodes
relevant contextual information into reference frames, enabling better skill
generalization to new situations. However, most TP-LfD algorithms typically
require multiple demonstrations across various environmental conditions to
ensure sufficient statistics for a meaningful model. It is not a trivial task
for robot users to create different situations and perform demonstrations under
all of them. Therefore, this paper presents a novel algorithm to learn skills
from few demonstrations. By leveraging the reference frame weights that capture
the frame importance or relevance during task executions, our method
demonstrates excellent skill acquisition performance, which is validated in
real robotic environments.Comment: Accepted by ISER. For the experiment video, see
https://youtu.be/JpGjk4eKC3
Deep Metric Imitation Learning for Stable Motion Primitives
Imitation Learning (IL) is a powerful technique for intuitive robotic
programming. However, ensuring the reliability of learned behaviors remains a
challenge. In the context of reaching motions, a robot should consistently
reach its goal, regardless of its initial conditions. To meet this requirement,
IL methods often employ specialized function approximators that guarantee this
property by construction. Although effective, these approaches come with a set
of limitations: 1) they are unable to fully exploit the capabilities of modern
Deep Neural Network (DNN) architectures, 2) some are restricted in the family
of motions they can model, resulting in suboptimal IL capabilities, and 3) they
require explicit extensions to account for the geometry of motions that
consider orientations. To address these challenges, we introduce a novel
stability loss function, drawing inspiration from the triplet loss used in the
deep metric learning literature. This loss does not constrain the DNN's
architecture and enables learning policies that yield accurate results.
Furthermore, it is easily adaptable to the geometry of the robot's state space.
We provide a proof of the stability properties induced by this loss and
empirically validate our method in various settings. These settings include
Euclidean and non-Euclidean state spaces, as well as first-order and
second-order motions, both in simulation and with real robots. More details
about the experimental results can be found at: https://youtu.be/ZWKLGntCI6w.Comment: 21 pages, 15 figures, 4 table
ILoSA: Interactive Learning of Stiffness and Attractors
Teaching robots how to apply forces according to our preferences is still an
open challenge that has to be tackled from multiple engineering perspectives.
This paper studies how to learn variable impedance policies where both the
Cartesian stiffness and the attractor can be learned from human demonstrations
and corrections with a user-friendly interface. The presented framework, named
ILoSA, uses Gaussian Processes for policy learning, identifying regions of
uncertainty and allowing interactive corrections, stiffness modulation and
active disturbance rejection. The experimental evaluation of the framework is
carried out on a Franka-Emika Panda in three separate cases with unique force
interaction properties: 1) pulling a plug wherein a sudden force discontinuity
occurs upon successful removal of the plug, 2) pushing a box where a sustained
force is required to keep the robot in motion, and 3) wiping a whiteboard in
which the force is applied perpendicular to the direction of movement
- …