4 research outputs found
Automated Curriculum Learning by Rewarding Temporally Rare Events
Reward shaping allows reinforcement learning (RL) agents to accelerate
learning by receiving additional reward signals. However, these signals can be
difficult to design manually, especially for complex RL tasks. We propose a
simple and general approach that determines the reward of pre-defined events by
their rarity alone. Here events become less rewarding as they are experienced
more often, which encourages the agent to continually explore new types of
events as it learns. The adaptiveness of this reward function results in a form
of automated curriculum learning that does not have to be specified by the
experimenter. We demonstrate that this \emph{Rarity of Events} (RoE) approach
enables the agent to succeed in challenging VizDoom scenarios without access to
the extrinsic reward from the environment. Furthermore, the results demonstrate
that RoE learns a more versatile policy that adapts well to critical changes in
the environment. Rewarding events based on their rarity could help in many
unsolved RL environments that are characterized by sparse extrinsic rewards but
a plethora of known event types.Comment: 8 page
Towards Continual Reinforcement Learning: A Review and Perspectives
In this article, we aim to provide a literature review of different
formulations and approaches to continual reinforcement learning (RL), also
known as lifelong or non-stationary RL. We begin by discussing our perspective
on why RL is a natural fit for studying continual learning. We then provide a
taxonomy of different continual RL formulations and mathematically characterize
the non-stationary dynamics of each setting. We go on to discuss evaluation of
continual RL agents, providing an overview of benchmarks used in the literature
and important metrics for understanding agent performance. Finally, we
highlight open problems and challenges in bridging the gap between the current
state of continual RL and findings in neuroscience. While still in its early
days, the study of continual RL has the promise to develop better incremental
reinforcement learners that can function in increasingly realistic applications
where non-stationarity plays a vital role. These include applications such as
those in the fields of healthcare, education, logistics, and robotics.Comment: Preprint, 52 pages, 8 figure