2 research outputs found
Exploring Evolution Strategies for Reinforcement Learning in the Obstacle Tower Environment
Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced AnalyticsIn 2017 OpenAI demonstrated that it was possible to train an AI agent by using Evolution
Strategies (ES), and that the results rivaled standard Reinforcement Learning (RL) techniques
on modern benchmarks. Their research effectively showed that Evolution Strategies is a viable
alternative to traditional Reinforcement Learning techniques, and that it bypasses many of
Reinforcement Learning’s inconveniences, notably the use of backpropagation.
The Obstacle Tower environment aims to set a new Reinforcement Learning
benchmark by challenging Artificial Intelligence (AI) agents to traverse 3-Dimensional
procedurally generated levels using a real-time 3-Dimensional physics system. The
environment tests an agent’s ability to generalize by requiring it to optimize aspects that are
common in many Reinforcement Learning environments, but rarely combined in the same
environment: vision, planning, and control.
In this research, the original implementation of OpenAI’s Evolution Strategies
algorithm was applied for the first time to the Obstacle Tower environment to assess how well
it performs in a more complex environment, where the agent’s generalization ability is critical.
Additionally, in the interest of exploring Evolution Strategies in this environment, common
Genetic Algorithm selection and mutation techniques were developed and applied to try and
improve the performance of the original Evolution Strategies implementation. Crossover
techniques were not explored during this research, as they are rarely applied in Evolution
Strategies. The results show that although the basic implementation of Evolution Strategies
does not perform well in the complex Obstacle Tower environment, it is possible to improve
its performance by applying different evolution methods borrowed from Genetic Algorithm
(GA), which are algorithms belonging to the same family as Evolution Strategies