25,461 research outputs found
Reinforcement Learning Approaches in Social Robotics
This article surveys reinforcement learning approaches in social robotics.
Reinforcement learning is a framework for decision-making problems in which an
agent interacts through trial-and-error with its environment to discover an
optimal behavior. Since interaction is a key component in both reinforcement
learning and social robotics, it can be a well-suited approach for real-world
interactions with physically embodied social robots. The scope of the paper is
focused particularly on studies that include social physical robots and
real-world human-robot interactions with users. We present a thorough analysis
of reinforcement learning approaches in social robotics. In addition to a
survey, we categorize existent reinforcement learning approaches based on the
used method and the design of the reward mechanisms. Moreover, since
communication capability is a prominent feature of social robots, we discuss
and group the papers based on the communication medium used for reward
formulation. Considering the importance of designing the reward function, we
also provide a categorization of the papers based on the nature of the reward.
This categorization includes three major themes: interactive reinforcement
learning, intrinsically motivated methods, and task performance-driven methods.
The benefits and challenges of reinforcement learning in social robotics,
evaluation methods of the papers regarding whether or not they use subjective
and algorithmic measures, a discussion in the view of real-world reinforcement
learning challenges and proposed solutions, the points that remain to be
explored, including the approaches that have thus far received less attention
is also given in the paper. Thus, this paper aims to become a starting point
for researchers interested in using and applying reinforcement learning methods
in this particular research field
Reactive Stepping for Humanoid Robots using Reinforcement Learning: Application to Standing Push Recovery on the Exoskeleton Atalante
State-of-the-art reinforcement learning is now able to learn versatile
locomotion, balancing and push-recovery capabilities for bipedal robots in
simulation. Yet, the reality gap has mostly been overlooked and the simulated
results hardly transfer to real hardware. Either it is unsuccessful in practice
because the physics is over-simplified and hardware limitations are ignored, or
regularity is not guaranteed, and unexpected hazardous motions can occur. This
paper presents a reinforcement learning framework capable of learning robust
standing push recovery for bipedal robots that smoothly transfer to reality,
providing only instantaneous proprioceptive observations. By combining original
termination conditions and policy smoothness conditioning, we achieve stable
learning, sim-to-real transfer and safety using a policy without memory nor
explicit history. Reward engineering is then used to give insights into how to
keep balance. We demonstrate its performance in reality on the lower-limb
medical exoskeleton Atalante
Fast Model Identification via Physics Engines for Data-Efficient Policy Search
This paper presents a method for identifying mechanical parameters of robots
or objects, such as their mass and friction coefficients. Key features are the
use of off-the-shelf physics engines and the adaptation of a Bayesian
optimization technique towards minimizing the number of real-world experiments
needed for model-based reinforcement learning. The proposed framework
reproduces in a physics engine experiments performed on a real robot and
optimizes the model's mechanical parameters so as to match real-world
trajectories. The optimized model is then used for learning a policy in
simulation, before real-world deployment. It is well understood, however, that
it is hard to exactly reproduce real trajectories in simulation. Moreover, a
near-optimal policy can be frequently found with an imperfect model. Therefore,
this work proposes a strategy for identifying a model that is just good enough
to approximate the value of a locally optimal policy with a certain confidence,
instead of wasting effort on identifying the most accurate model. Evaluations,
performed both in simulation and on a real robotic manipulation task, indicate
that the proposed strategy results in an overall time-efficient, integrated
model identification and learning solution, which significantly improves the
data-efficiency of existing policy search algorithms.Comment: IJCAI 1
Supervised Learning and Reinforcement Learning of Feedback Models for Reactive Behaviors: Tactile Feedback Testbed
Robots need to be able to adapt to unexpected changes in the environment such
that they can autonomously succeed in their tasks. However, hand-designing
feedback models for adaptation is tedious, if at all possible, making
data-driven methods a promising alternative. In this paper we introduce a full
framework for learning feedback models for reactive motion planning. Our
pipeline starts by segmenting demonstrations of a complete task into motion
primitives via a semi-automated segmentation algorithm. Then, given additional
demonstrations of successful adaptation behaviors, we learn initial feedback
models through learning from demonstrations. In the final phase, a
sample-efficient reinforcement learning algorithm fine-tunes these feedback
models for novel task settings through few real system interactions. We
evaluate our approach on a real anthropomorphic robot in learning a tactile
feedback task.Comment: Submitted to the International Journal of Robotics Research. Paper
length is 21 pages (including references) with 12 figures. A video overview
of the reinforcement learning experiment on the real robot can be seen at
https://www.youtube.com/watch?v=WDq1rcupVM0. arXiv admin note: text overlap
with arXiv:1710.0855
Learning to bag with a simulation-free reinforcement learning framework for robots
Bagging is an essential skill that humans perform in their daily activities.
However, deformable objects, such as bags, are complex for robots to
manipulate. This paper presents an efficient learning-based framework that
enables robots to learn bagging. The novelty of this framework is its ability
to perform bagging without relying on simulations. The learning process is
accomplished through a reinforcement learning algorithm introduced in this
work, designed to find the best grasping points of the bag based on a set of
compact state representations. The framework utilizes a set of primitive
actions and represents the task in five states. In our experiments, the
framework reaches a 60 % and 80 % of success rate after around three hours of
training in the real world when starting the bagging task from folded and
unfolded, respectively. Finally, we test the trained model with two more bags
of different sizes to evaluate its generalizability.Comment: IET Cyber-Systems and Robotic
- …