3 research outputs found
Evolving Inborn Knowledge For Fast Adaptation in Dynamic POMDP Problems
Rapid online adaptation to changing tasks is an important problem in machine
learning and, recently, a focus of meta-reinforcement learning. However,
reinforcement learning (RL) algorithms struggle in POMDP environments because
the state of the system, essential in a RL framework, is not always visible.
Additionally, hand-designed meta-RL architectures may not include suitable
computational structures for specific learning problems. The evolution of
online learning mechanisms, on the contrary, has the ability to incorporate
learning strategies into an agent that can (i) evolve memory when required and
(ii) optimize adaptation speed to specific online learning problems. In this
paper, we exploit the highly adaptive nature of neuromodulated neural networks
to evolve a controller that uses the latent space of an autoencoder in a POMDP.
The analysis of the evolved networks reveals the ability of the proposed
algorithm to acquire inborn knowledge in a variety of aspects such as the
detection of cues that reveal implicit rewards, and the ability to evolve
location neurons that help with navigation. The integration of inborn knowledge
and online plasticity enabled fast adaptation and better performance in
comparison to some non-evolutionary meta-reinforcement learning algorithms. The
algorithm proved also to succeed in the 3D gaming environment Malmo Minecraft.Comment: 9 pages. Accepted as a full paper in the Genetic and Evolutionary
Computation Conference (GECCO 2020
Evolving inborn knowledge for fast adaptation in dynamic POMDP problems
Rapid online adaptation to changing tasks is an important problem in machine
learning and, recently, a focus of meta-reinforcement learning. However,
reinforcement learning (RL) algorithms struggle in POMDP environments because
the state of the system, essential in a RL framework, is not always visible.
Additionally, hand-designed meta-RL architectures may not include suitable
computational structures for specific learning problems. The evolution of
online learning mechanisms, on the contrary, has the ability to incorporate
learning strategies into an agent that can (i) evolve memory when required and
(ii) optimize adaptation speed to specific online learning problems. In this
paper, we exploit the highly adaptive nature of neuromodulated neural networks
to evolve a controller that uses the latent space of an autoencoder in a POMDP.
The analysis of the evolved networks reveals the ability of the proposed
algorithm to acquire inborn knowledge in a variety of aspects such as the
detection of cues that reveal implicit rewards, and the ability to evolve
location neurons that help with navigation. The integration of inborn knowledge
and online plasticity enabled fast adaptation and better performance in
comparison to some non-evolutionary meta-reinforcement learning algorithms. The
algorithm proved also to succeed in the 3D gaming environment Malmo Minecraft