6 research outputs found
Deep Reinforcement Learning using Capsules in Advanced Game Environments
Reinforcement Learning (RL) is a research area that has blossomed
tremendously in recent years and has shown remarkable potential for artificial
intelligence based opponents in computer games. This success is primarily due
to vast capabilities of Convolutional Neural Networks (ConvNet), enabling
algorithms to extract useful information from noisy environments. Capsule
Network (CapsNet) is a recent introduction to the Deep Learning algorithm group
and has only barely begun to be explored. The network is an architecture for
image classification, with superior performance for classification of the MNIST
dataset. CapsNets have not been explored beyond image classification.
This thesis introduces the use of CapsNet for Q-Learning based game
algorithms. To successfully apply CapsNet in advanced game play, three main
contributions follow. First, the introduction of four new game environments as
frameworks for RL research with increasing complexity, namely Flash RL, Deep
Line Wars, Deep RTS, and Deep Maze. These environments fill the gap between
relatively simple and more complex game environments available for RL research
and are in the thesis used to test and explore the CapsNet behavior.
Second, the thesis introduces a generative modeling approach to produce
artificial training data for use in Deep Learning models including CapsNets. We
empirically show that conditional generative modeling can successfully generate
game data of sufficient quality to train a Deep Q-Network well.
Third, we show that CapsNet is a reliable architecture for Deep Q-Learning
based algorithms for game AI. A capsule is a group of neurons that determine
the presence of objects in the data and is in the literature shown to increase
the robustness of training and predictions while lowering the amount training
data needed. It should, therefore, be ideally suited for game plays.Comment: Master Thesis in Computer Scienc
Deep Reinforcement Learning using Capsules in Advanced Game Environments
Master's thesis Information- and communication technology IKT590 - University of Agder 2017Reinforcement Learning (RL) is a research area that has blossomed tremendously
in recent years and has shown remarkable potential for arti cial
intelligence based opponents in computer games. This success is primarily
due to vast capabilities of Convolutional Neural Networks (ConvNet),
enabling algorithms to extract useful information from noisy environments.
Capsule Network (CapsNet) is a recent introduction to the Deep Learning
algorithm group and has only barely begun to be explored. The network is
an architecture for image classi cation, with superior performance for classi
cation of the MNIST dataset. CapsNets have not been explored beyond
image classi cation.
This thesis introduces the use of CapsNet for Q-Learning based game algorithms.
To successfully apply CapsNet in advanced game play, three main
contributions follow. First, the introduction of four new game environments
as frameworks for RL research with increasing complexity, namely Flash
RL, Deep Line Wars, Deep RTS, and Deep Maze. These environments ll
the gap between relatively simple and more complex game environments
available for RL research and are in the thesis used to test and explore the
CapsNet behavior.
Second, the thesis introduces a generative modeling approach to produce
arti cial training data for use in Deep Learning models including CapsNets.
We empirically show that conditional generative modeling can successfully
generate game data of su cient quality to train a Deep Q-Network well.
Third, we show that CapsNet is a reliable architecture for Deep Q-Learning
based algorithms for game AI. A capsule is a group of neurons that determine
the presence of objects in the data and is in the literature shown
to increase the robustness of training and predictions while lowering the
amount training data needed. It should, therefore, be ideally suited for
game plays. We conclusively show that capsules can be applied to Deep
Q-Learning, and present experimental results of this method in the environments
introduced. We further show that capsules do not scale as well
as convolutions, indicating that CapsNet-based algorithms alone will not be
able to play even more advanced games without improved scalabilit
Towards safe reinforcement-learning in industrial grid-warehousing
publishedVersionHybri