5 research outputs found
Learning to Switch Between Machines and Humans
Reinforcement learning agents have been mostly developed and evaluated under
the assumption that they will operate in a fully autonomous manner -- they will
take all actions. In this work, our goal is to develop algorithms that, by
learning to switch control between machine and human agents, allow existing
reinforcement learning agents to operate under different automation levels. To
this end, we first formally define the problem of learning to switch control
among agents in a team via a 2-layer Markov decision process. Then, we develop
an online learning algorithm that uses upper confidence bounds on the agents'
policies and the environment's transition probabilities to find a sequence of
switching policies. We prove that the total regret of our algorithm with
respect to the optimal switching policy is sublinear in the number of learning
steps. Moreover, we also show that our algorithm can be used to find multiple
sequences of switching policies across several independent teams of agents
operating in similar environments, where it greatly benefits from maintaining
shared confidence bounds for the environments' transition probabilities.
Simulation experiments in obstacle avoidance in a semi-autonomous driving
scenario illustrate our theoretical findings and demonstrate that, by
exploiting the specific structure of the problem, our proposed algorithm is
superior to problem-agnostic algorithms.Comment: Added support for unknown transition probabilities and multiple team
Multiagent Deep Reinforcement Learning: Challenges and Directions Towards Human-Like Approaches
This paper surveys the field of multiagent deep reinforcement learning. The
combination of deep neural networks with reinforcement learning has gained
increased traction in recent years and is slowly shifting the focus from
single-agent to multiagent environments. Dealing with multiple agents is
inherently more complex as (a) the future rewards depend on the joint actions
of multiple players and (b) the computational complexity of functions
increases. We present the most common multiagent problem representations and
their main challenges, and identify five research areas that address one or
more of these challenges: centralised training and decentralised execution,
opponent modelling, communication, efficient coordination, and reward shaping.
We find that many computational studies rely on unrealistic assumptions or are
not generalisable to other settings; they struggle to overcome the curse of
dimensionality or nonstationarity. Approaches from psychology and sociology
capture promising relevant behaviours such as communication and coordination.
We suggest that, for multiagent reinforcement learning to be successful, future
research addresses these challenges with an interdisciplinary approach to open
up new possibilities for more human-oriented solutions in multiagent
reinforcement learning.Comment: 37 pages, 6 figure