A Survey of Monte Carlo Tree Search Methods

Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work

Automated Video Game Testing Using Synthetic and Human-Like Agents

In this paper, we present a new methodology that employs tester agents to automate video game testing. We introduce two types of agents -synthetic and human-like- and two distinct approaches to create them. Our agents are derived from Reinforcement Learning (RL) and Monte Carlo Tree Search (MCTS) agents, but focus on finding defects. The synthetic agent uses test goals generated from game scenarios, and these goals are further modified to examine the effects of unintended game transitions. The human-like agent uses test goals extracted by our proposed multiple greedy-policy inverse reinforcement learning (MGP-IRL) algorithm from tester trajectories. MGPIRL captures multiple policies executed by human testers. These testers' aims are finding defects while interacting with the game to break it, which is considerably different from game playing. We present interaction states to model such interactions. We use our agents to produce test sequences, run the game with these sequences, and check the game for each run with an automated test oracle. We analyze the proposed method in two parts: we compare the success of human-like and synthetic agents in bug finding, and we evaluate the similarity between humanlike agents and human testers. We collected 427 trajectories from human testers using the General Video Game Artificial Intelligence (GVG-AI) framework and created three games with 12 levels that contain 45 bugs. Our experiments reveal that human-like and synthetic agents compete with human testers' bug finding performances. Moreover, we show that MGP-IRL increases the human-likeness of agents while improving the bug finding performance

A Software Design Pattern Based Approach to Auto Dynamic Difficulty in Video Games

From the point of view of skill levels, reflex speeds, hand-eye coordination, tolerance for frustration, and motivations, video game players may vary drastically. Auto dynamic difficulty (ADD) in video games refers to the technique of automatically adjusting different aspects of a video game in real time, based on the player’s ability and emergence factors in order to provide the optimal experience to users from such a large demography and increase replay value. In this thesis, we describe a collection of software design patterns for enabling auto dynamic difficulty in video games. We also discuss the benefits of a design pattern based approach in terms of software quality factors and process improvements based on our experience of applying it in three different video games. Additionally, we present a semi-automatic framework to assist in applying our design pattern based approach in video games. Finally, we conducted a preliminary user study where a Post-Degree Diploma student at the University of Western Ontario applied the design pattern based approach to create ADD in two arcade style games

General Game Playing: Problem analysis. Hybrid computational intelligence algorithm based solution

El objetivo de este trabajo es presentar unas bases de conocimiento sobre el denominado General Game Playing (GGP) analizando los conceptos relacionados con esta área que ha surgido recientemente, de forma que nuestro trabajo pueda ser usado como base en futuras investigaciones y tesis relacionadas con la materia. Para ello, se hará un estudio de los enfoques que se han empleado para abordar el problema y se profundizará en otras técnicas algorítmicas, tales como por ejemplo la de Montecarlo Tree Search y los algoritmos bio-inspirados que no se han empleado (o se han empleado poco) en este contexto. Adicionalmente, se realiza una propuesta de un agente autónomo (es decir, un resolutor del problema), implementando un algoritmo bio-inspirado mixto, dentro de la plataforma para la General Video Game Artificial Intelligence Competition (GVGAI), analizando sus resultados y extrayendo conclusiones

Automatic Game Parameter Tuning using General Video Game Agents

Automatic Game Design is a subfield of Game Artificial Intelligence that aims to study the usage of AI algorithms for assisting in game design tasks. This dissertation presents a research work in this field, focusing on applying an evolutionary algorithm to video game parameterization. The task we are interested in is player experience. N-Tuple Bandit Evolutionary Algorithm (NTBEA) is an evolutionary algorithm that was recently proposed and successfully applied in game parameterization in a simple domain, which is the first experiment included in this project. To further investigating its ability in evolving game parameters, We applied NTBEA to evolve parameter sets for three General Video Game AI (GVGAI) games, because GVGAI has variety supplies of video games in different types and the framework has already been prepared for parameterization. 9 positive increasing functions were picked as target functions as representations of the player expected score trends. Our initial assumption was that the evolved games should provide the game environments that allow players to obtain score in the same trend as one of these functions. The experiment results confirm this for some functions, and prove that the NTBEA is very much capable of evolving GVGAI games to satisfy this task