Automatic generation of level maps with the do what's possible representation

© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Automatic generation of level maps is a popular form of automatic content generation. In this study, a recently developed technique employing the do what's possible representation is used to create open-ended level maps. Generation of the map can continue indefinitely, yielding a highly scalable representation. A parameter study is performed to find good parameters for the evolutionary algorithm used to locate high quality map generators. Variations on the technique are presented, demonstrating its versatility, and an algorithmic variant is given that both improves performance and changes the character of maps located. The ability of the map to adapt to different regions where the map is permitted to occupy space are also tested.Final Accepted Versio

On the Importance of Exploration for Generalization in Reinforcement Learning

Existing approaches for improving generalization in deep reinforcement learning (RL) have mostly focused on representation learning, neglecting RL-specific aspects such as exploration. We hypothesize that the agent's exploration strategy plays a key role in its ability to generalize to new environments. Through a series of experiments in a tabular contextual MDP, we show that exploration is helpful not only for efficiently finding the optimal policy for the training environments but also for acquiring knowledge that helps decision making in unseen environments. Based on these observations, we propose EDE: Exploration via Distributional Ensemble, a method that encourages exploration of states with high epistemic uncertainty through an ensemble of Q-value distributions. Our algorithm is the first value-based approach to achieve state-of-the-art on both Procgen and Crafter, two benchmarks for generalization in RL with high-dimensional observations. The open-sourced implementation can be found at https://github.com/facebookresearch/ede

Using Reinforcement Learning and Task Decomposition for Learning to Play Doom

Reinforcement learning (RL) is a basic machine learning method, which has recently gained in popularity. As the field matures, RL methods are being applied on progressively more complex problems. This leads to need to design increasingly more complicated models, which are difficult to train and apply in practice. This thesis explores one potential way of solving the problem with large and slow RL models, which is using a modular approach to build the models. The idea behind this approach is to decompose the main task into smaller subtasks and have separate modules each of which concentrates on solving a single subtask. In more detail, the proposed agent will be built using the Q-decomposition algorithm, which provides a simple and robust algorithm for building modular RL agents. The problem we use as an example of usefulness of the modular approach is a simplified version of the video game Doom and we design a RL agent that learns to play it. The empirical results indicate that the proposed model is able to learn to play the simplified version of Doom on a reasonable level, but not perfectly. Additionally, we show that the proposed model might suffer from usage of too simple models for solving the subtasks. Nevertheless, taken as a whole the results and the experience of designing the agent show that the modular approach for RL is a promising way forward and warrants further exploration