8 research outputs found
Deep Predictive Policy Training using Reinforcement Learning
Skilled robot task learning is best implemented by predictive action policies
due to the inherent latency of sensorimotor processes. However, training such
predictive policies is challenging as it involves finding a trajectory of motor
activations for the full duration of the action. We propose a data-efficient
deep predictive policy training (DPPT) framework with a deep neural network
policy architecture which maps an image observation to a sequence of motor
activations. The architecture consists of three sub-networks referred to as the
perception, policy and behavior super-layers. The perception and behavior
super-layers force an abstraction of visual and motor data trained with
synthetic and simulated training samples, respectively. The policy super-layer
is a small sub-network with fewer parameters that maps data in-between the
abstracted manifolds. It is trained for each task using methods for policy
search reinforcement learning. We demonstrate the suitability of the proposed
architecture and learning framework by training predictive policies for skilled
object grasping and ball throwing on a PR2 robot. The effectiveness of the
method is illustrated by the fact that these tasks are trained using only about
180 real robot attempts with qualitative terminal rewards.Comment: This work is submitted to IEEE/RSJ International Conference on
Intelligent Robots and Systems 2017 (IROS2017
From Human Physical Interaction To Online Motion Adaptation Using Parameterized Dynamical Systems
In this work, we present an adaptive motion planning approach for impedance-controlled robots to modify their tasks based on human physical interactions. We use a class of parameterized time-independent dynamical systems for motion generation where the modulation of such parameters allows for motion flexibility. To adapt to human interactions, we update the parameter of our dynamical system in order to reduce the tracking error (i.e., between the desired trajectory generated by the dynamical system and the real trajectory influenced by the human interaction). We provide analytical analysis and several simulations of our method. Finally, we investigate our approach through real world experiments with 7-DOF KUKA LWR 4+ robot performing tasks such as polishing and pick-and-place
Reinforcement Learning Approaches in Social Robotics
This article surveys reinforcement learning approaches in social robotics.
Reinforcement learning is a framework for decision-making problems in which an
agent interacts through trial-and-error with its environment to discover an
optimal behavior. Since interaction is a key component in both reinforcement
learning and social robotics, it can be a well-suited approach for real-world
interactions with physically embodied social robots. The scope of the paper is
focused particularly on studies that include social physical robots and
real-world human-robot interactions with users. We present a thorough analysis
of reinforcement learning approaches in social robotics. In addition to a
survey, we categorize existent reinforcement learning approaches based on the
used method and the design of the reward mechanisms. Moreover, since
communication capability is a prominent feature of social robots, we discuss
and group the papers based on the communication medium used for reward
formulation. Considering the importance of designing the reward function, we
also provide a categorization of the papers based on the nature of the reward.
This categorization includes three major themes: interactive reinforcement
learning, intrinsically motivated methods, and task performance-driven methods.
The benefits and challenges of reinforcement learning in social robotics,
evaluation methods of the papers regarding whether or not they use subjective
and algorithmic measures, a discussion in the view of real-world reinforcement
learning challenges and proposed solutions, the points that remain to be
explored, including the approaches that have thus far received less attention
is also given in the paper. Thus, this paper aims to become a starting point
for researchers interested in using and applying reinforcement learning methods
in this particular research field
Reinforcement learning for sequential decision-making: a data driven approach for finance
This work presents a variety of reinforcement learning applications to the
domain of nance. It composes of two-part. The rst one represents a technical
overview of the basic concepts in machine learning, which are required
to understand and work with the reinforcement learning paradigm and are
shared among the domains of applications. Chapter 1 outlines the fundamental
principle of machine learning reasoning before introducing the neural
network model as a central component of every algorithm presented in this
work. Chapter 2 introduces the idea of reinforcement learning from its roots,
focusing on the mathematical formalism generally employed in every application.
We focus on integrating the reinforcement learning framework with the
neural network, and we explain their critical role in the eld's development.
After the technical part, we present our original contribution, articulated
in three di erent essays. The narrative line follows the idea of introducing
the use of varying reinforcement learning algorithms through a trading application
(Brini and Tantari, 2021) in Chapter 3. Then in Chapter 4 we
focus on one of the presented reinforcement learning algorithms and aim at
improving its performance and scalability in solving the trading problem by
leveraging prior knowledge of the setting. In Chapter 5 of the second part,
we use the same reinforcement learning algorithm to solve the problem of
exchanging liquidity in a system of banks that can borrow and lend money,
highlighting the
exibility and the e ectiveness of the reinforcement learning
paradigm in the broad nancial domain. We conclude with some remarks
and ideas for further research in reinforcement learning applied to nance