3 research outputs found
Deep Reinforcement Learning with Interactive Feedback in a Human-Robot Environment
Robots are extending their presence in domestic environments every day, being
more common to see them carrying out tasks in home scenarios. In the future,
robots are expected to increasingly perform more complex tasks and, therefore,
be able to acquire experience from different sources as quickly as possible. A
plausible approach to address this issue is interactive feedback, where a
trainer advises a learner on which actions should be taken from specific states
to speed up the learning process. Moreover, deep reinforcement learning has
been recently widely utilized in robotics to learn the environment and acquire
new skills autonomously. However, an open issue when using deep reinforcement
learning is the excessive time needed to learn a task from raw input images. In
this work, we propose a deep reinforcement learning approach with interactive
feedback to learn a domestic task in a human-robot scenario. We compare three
different learning methods using a simulated robotic arm for the task of
organizing different objects; the proposed methods are (i) deep reinforcement
learning (DeepRL); (ii) interactive deep reinforcement learning using a
previously trained artificial agent as an advisor (agent-IDeepRL); and (iii)
interactive deep reinforcement learning using a human advisor (human-IDeepRL).
We demonstrate that interactive approaches provide advantages for the learning
process. The obtained results show that a learner agent, using either
agent-IDeepRL or human-IDeepRL, completes the given task earlier and has fewer
mistakes compared to the autonomous DeepRL approach.Comment: In press journal Applied Science
Human Engagement Providing Evaluative and Informative Advice for Interactive Reinforcement Learning
Reinforcement learning is an approach used by intelligent agents to
autonomously learn new skills. Although reinforcement learning has been
demonstrated to be an effective learning approach in several different
contexts, a common drawback exhibited is the time needed in order to
satisfactorily learn a task, especially in large state-action spaces. To
address this issue, interactive reinforcement learning proposes the use of
externally-sourced information in order to speed up the learning process. Up to
now, different information sources have been used to give advice to the learner
agent, among them human-sourced advice. When interacting with a learner agent,
humans may provide either evaluative or informative advice. From the agent's
perspective these styles of interaction are commonly referred to as
reward-shaping and policy-shaping respectively. Evaluation requires the human
to provide feedback on the prior action performed, while informative advice
they provide advice on the best action to select for a given situation. Prior
research has focused on the effect of human-sourced advice on the interactive
reinforcement learning process, specifically aiming to improve the learning
speed of the agent, while reducing the engagement with the human. This work
presents an experimental setup for a human-trial designed to compare the
methods people use to deliver advice in term of human engagement. Obtained
results show that users giving informative advice to the learner agents provide
more accurate advice, are willing to assist the learner agent for a longer
time, and provide more advice per episode. Additionally, self-evaluation from
participants using the informative approach has indicated that the agent's
ability to follow the advice is higher, and therefore, they feel their own
advice to be of higher accuracy when compared to people providing evaluative
advice.Comment: 33 pages, 15 figure