26,610 research outputs found
An Uncertainty-Aware Minimal Intervention Control Strategy Learned from Demonstrations
Motivated by the desire to have robots physically present in human environments, in recent years we have witnessed an emergence of different approaches for learning active compliance. Some of the most compelling solutions exploit a minimal intervention control principle, correcting deviations from a goal only when necessary, and among those who follow this concept, several probabilistic techniques have stood out from the rest. However, these approaches are prone to requiring several task demonstrations for proper gain estimation and to generating unpredictable robot motions in the face of uncertainty. Here we present a Programming by Demonstration approach for uncertainty-aware impedance regulation, aimed at making the robot compliant - and safe to interact with - when the uncertainty about its predicted actions is high. Moreover, we propose a data-efficient strategy, based on the energy observed during demonstrations, to achieve minimal intervention control, when the uncertainty is low. The approach is validated in an experimental scenario, where a human collaboratively moves an object with a 7-DoF torque-controlled robot
Uncertainty-Aware Imitation Learning using Kernelized Movement Primitives
During the past few years, probabilistic approaches to imitation learning
have earned a relevant place in the literature. One of their most prominent
features, in addition to extracting a mean trajectory from task demonstrations,
is that they provide a variance estimation. The intuitive meaning of this
variance, however, changes across different techniques, indicating either
variability or uncertainty. In this paper we leverage kernelized movement
primitives (KMP) to provide a new perspective on imitation learning by
predicting variability, correlations and uncertainty about robot actions. This
rich set of information is used in combination with optimal controller fusion
to learn actions from data, with two main advantages: i) robots become safe
when uncertain about their actions and ii) they are able to leverage partial
demonstrations, given as elementary sub-tasks, to optimally perform a higher
level, more complex task. We showcase our approach in a painting task, where a
human user and a KUKA robot collaborate to paint a wooden board. The task is
divided into two sub-tasks and we show that using our approach the robot
becomes compliant (hence safe) outside the training regions and executes the
two sub-tasks with optimal gains.Comment: Published in the proceedings of IROS 201
Uncertainty-Aware Imitation Learning using Kernelized Movement Primitives
During the past few years, probabilistic approaches to imitation learning
have earned a relevant place in the literature. One of their most prominent
features, in addition to extracting a mean trajectory from task demonstrations,
is that they provide a variance estimation. The intuitive meaning of this
variance, however, changes across different techniques, indicating either
variability or uncertainty. In this paper we leverage kernelized movement
primitives (KMP) to provide a new perspective on imitation learning by
predicting variability, correlations and uncertainty about robot actions. This
rich set of information is used in combination with optimal controller fusion
to learn actions from data, with two main advantages: i) robots become safe
when uncertain about their actions and ii) they are able to leverage partial
demonstrations, given as elementary sub-tasks, to optimally perform a higher
level, more complex task. We showcase our approach in a painting task, where a
human user and a KUKA robot collaborate to paint a wooden board. The task is
divided into two sub-tasks and we show that using our approach the robot
becomes compliant (hence safe) outside the training regions and executes the
two sub-tasks with optimal gains.Comment: Submitted to IROS1
MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulation
Robotic systems that aspire to operate in uninstrumented real-world
environments must perceive the world directly via onboard sensing. Vision-based
learning systems aim to eliminate the need for environment instrumentation by
building an implicit understanding of the world based on raw pixels, but
navigating the contact-rich high-dimensional search space from solely sparse
visual reward signals significantly exacerbates the challenge of exploration.
The applicability of such systems is thus typically restricted to simulated or
heavily engineered environments since agent exploration in the real-world
without the guidance of explicit state estimation and dense rewards can lead to
unsafe behavior and safety faults that are catastrophic. In this study, we
isolate the root causes behind these limitations to develop a system, called
MoDem-V2, capable of learning contact-rich manipulation directly in the
uninstrumented real world. Building on the latest algorithmic advancements in
model-based reinforcement learning (MBRL), demo-bootstrapping, and effective
exploration, MoDem-V2 can acquire contact-rich dexterous manipulation skills
directly in the real world. We identify key ingredients for leveraging
demonstrations in model learning while respecting real-world safety
considerations -- exploration centering, agency handover, and actor-critic
ensembles. We empirically demonstrate the contribution of these ingredients in
four complex visuo-motor manipulation problems in both simulation and the real
world. To the best of our knowledge, our work presents the first successful
system for demonstration-augmented visual MBRL trained directly in the real
world. Visit https://sites.google.com/view/modem-v2 for videos and more
details.Comment: 9 pages, 8 figure
Uncertainty-Aware Shared Autonomy System with Hierarchical Conservative Skill Inference
Shared autonomy imitation learning, in which robots share workspace with
humans for learning, enables correct actions in unvisited states and the
effective resolution of compounding errors through expert's corrections.
However, it demands continuous human attention and supervision to lead the
demonstrations, without considering the risks associated with human judgment
errors and delayed interventions. This can potentially lead to high levels of
fatigue for the demonstrator and the additional errors. In this work, we
propose an uncertainty-aware shared autonomy system that enables the robot to
infer conservative task skills considering environmental uncertainties and
learning from expert demonstrations and corrections. To enhance generalization
and scalability, we introduce a hierarchical structure-based skill uncertainty
inference framework operating at more abstract levels. We apply this to robot
motion to promote a more stable interaction. Although shared autonomy systems
have demonstrated high-level results in recent research and play a critical
role, specific system design details have remained elusive. This paper provides
a detailed design proposal for a shared autonomy system considering various
robot configurations. Furthermore, we experimentally demonstrate the system's
capability to learn operational skills, even in dynamic environments with
interference, through pouring and pick-and-place tasks. Our code will be
released soon.Comment: Submitted to ICRA 2024 and currently under revie
Interactive Imitation Learning in Robotics: A Survey
Interactive Imitation Learning (IIL) is a branch of Imitation Learning (IL)
where human feedback is provided intermittently during robot execution allowing
an online improvement of the robot's behavior. In recent years, IIL has
increasingly started to carve out its own space as a promising data-driven
alternative for solving complex robotic tasks. The advantages of IIL are its
data-efficient, as the human feedback guides the robot directly towards an
improved behavior, and its robustness, as the distribution mismatch between the
teacher and learner trajectories is minimized by providing feedback directly
over the learner's trajectories. Nevertheless, despite the opportunities that
IIL presents, its terminology, structure, and applicability are not clear nor
unified in the literature, slowing down its development and, therefore, the
research of innovative formulations and discoveries. In this article, we
attempt to facilitate research in IIL and lower entry barriers for new
practitioners by providing a survey of the field that unifies and structures
it. In addition, we aim to raise awareness of its potential, what has been
accomplished and what are still open research questions. We organize the most
relevant works in IIL in terms of human-robot interaction (i.e., types of
feedback), interfaces (i.e., means of providing feedback), learning (i.e.,
models learned from feedback and function approximators), user experience
(i.e., human perception about the learning process), applications, and
benchmarks. Furthermore, we analyze similarities and differences between IIL
and RL, providing a discussion on how the concepts offline, online, off-policy
and on-policy learning should be transferred to IIL from the RL literature. We
particularly focus on robotic applications in the real world and discuss their
implications, limitations, and promising future areas of research
Towards Minimal Intervention Control with Competing Constraints
As many imitation learning algorithms focus on pure trajectory generation in either Cartesian space or joint space, the problem of considering competing trajectory constraints from both spaces still presents several challenges. In particular, when perturbations are applied to the robot, the underlying controller should take into account the importance of each space for the task execution, and compute the control effort accordingly. However, no such controller formulation exists. In this paper, we provide a minimal intervention control strategy that simultaneously addresses the problems of optimal control and competing constraints between Cartesian and joint spaces. In light of the inconsistency between Cartesian and joint constraints, we exploit the robot null space from an information-theory perspective so as to reduce the corresponding conflict. An optimal solution to the aforementioned controller is derived and furthermore a connection to the classical finite horizon linear quadratic regulator (LQR) is provided. Finally, a writing task in a simulated robot verifies the effectiveness of our approach
- …