17,866 research outputs found
Automatic Curriculum Learning For Deep RL: A Short Survey
Automatic Curriculum Learning (ACL) has become a cornerstone of recent
successes in Deep Reinforcement Learning (DRL).These methods shape the learning
trajectories of agents by challenging them with tasks adapted to their
capacities. In recent years, they have been used to improve sample efficiency
and asymptotic performance, to organize exploration, to encourage
generalization or to solve sparse reward problems, among others. The ambition
of this work is dual: 1) to present a compact and accessible introduction to
the Automatic Curriculum Learning literature and 2) to draw a bigger picture of
the current state of the art in ACL to encourage the cross-breeding of existing
concepts and the emergence of new ideas.Comment: Accepted at IJCAI202
Reward Shaping with Recurrent Neural Networks for Speeding up On-Line Policy Learning in Spoken Dialogue Systems
Statistical spoken dialogue systems have the attractive property of being
able to be optimised from data via interactions with real users. However in the
reinforcement learning paradigm the dialogue manager (agent) often requires
significant time to explore the state-action space to learn to behave in a
desirable manner. This is a critical issue when the system is trained on-line
with real users where learning costs are expensive. Reward shaping is one
promising technique for addressing these concerns. Here we examine three
recurrent neural network (RNN) approaches for providing reward shaping
information in addition to the primary (task-orientated) environmental
feedback. These RNNs are trained on returns from dialogues generated by a
simulated user and attempt to diffuse the overall evaluation of the dialogue
back down to the turn level to guide the agent towards good behaviour faster.
In both simulated and real user scenarios these RNNs are shown to increase
policy learning speed. Importantly, they do not require prior knowledge of the
user's goal.Comment: Accepted for publication in SigDial 201
- …