A Survey of Monte Carlo Tree Search Methods

Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work

Finding Game Levels with the Right Difficulty in a Few Trials through Intelligent Trial-and-Error

Methods for dynamic difficulty adjustment allow games to be tailored to particular players to maximize their engagement. However, current methods often only modify a limited set of game features such as the difficulty of the opponents, or the availability of resources. Other approaches, such as experience-driven Procedural Content Generation (PCG), can generate complete levels with desired properties such as levels that are neither too hard nor too easy, but require many iterations. This paper presents a method that can generate and search for complete levels with a specific target difficulty in only a few trials. This advance is enabled by through an Intelligent Trial-and-Error algorithm, originally developed to allow robots to adapt quickly. Our algorithm first creates a large variety of different levels that vary across predefined dimensions such as leniency or map coverage. The performance of an AI playing agent on these maps gives a proxy for how difficult the level would be for another AI agent (e.g. one that employs Monte Carlo Tree Search instead of Greedy Tree Search); using this information, a Bayesian Optimization procedure is deployed, updating the difficulty of the prior map to reflect the ability of the agent. The approach can reliably find levels with a specific target difficulty for a variety of planning agents in only a few trials, while maintaining an understanding of their skill landscape.Comment: To be presented in the Conference on Games 202

A Software Design Pattern Based Approach to Auto Dynamic Difficulty in Video Games

From the point of view of skill levels, reflex speeds, hand-eye coordination, tolerance for frustration, and motivations, video game players may vary drastically. Auto dynamic difficulty (ADD) in video games refers to the technique of automatically adjusting different aspects of a video game in real time, based on the player’s ability and emergence factors in order to provide the optimal experience to users from such a large demography and increase replay value. In this thesis, we describe a collection of software design patterns for enabling auto dynamic difficulty in video games. We also discuss the benefits of a design pattern based approach in terms of software quality factors and process improvements based on our experience of applying it in three different video games. Additionally, we present a semi-automatic framework to assist in applying our design pattern based approach in video games. Finally, we conducted a preliminary user study where a Post-Degree Diploma student at the University of Western Ontario applied the design pattern based approach to create ADD in two arcade style games

The 2016 Two-Player GVGAI Competition

This paper showcases the setting and results of the first Two-Player General Video Game AI competition, which ran in 2016 at the IEEE World Congress on Computational Intelligence and the IEEE Conference on Computational Intelligence and Games. The challenges for the general game AI agents are expanded in this track from the single-player version, looking at direct player interaction in both competitive and cooperative environments of various types and degrees of difficulty. The focus is on the agents not only handling multiple problems, but also having to account for another intelligent entity in the game, who is expected to work towards their own goals (winning the game). This other player will possibly interact with first agent in a more engaging way than the environment or any non-playing character may do. The top competition entries are analyzed in detail and the performance of all agents is compared across the four sets of games. The results validate the competition system in assessing generality, as well as showing Monte Carlo Tree Search continuing to dominate by winning the overall Championship. However, this approach is closely followed by Rolling Horizon Evolutionary Algorithms, employed by the winner of the second leg of the contest

Recent Advances in General Game Playing

The goal of General Game Playing (GGP) has been to develop computer programs that can perform well across various game types. It is natural for human game players to transfer knowledge from games they already know how to play to other similar games. GGP research attempts to design systems that work well across different game types, including unknown new games. In this review, we present a survey of recent advances (2011 to 2014) in GGP for both traditional games and video games. It is notable that research on GGP has been expanding into modern video games. Monte-Carlo Tree Search and its enhancements have been the most influential techniques in GGP for both research domains. Additionally, international competitions have become important events that promote and increase GGP research. Recently, a video GGP competition was launched. In this survey, we review recent progress in the most challenging research areas of Artificial Intelligence (AI) related to universal game playing

Player experience and deceptive expectations of difficulty adaptation in digital games

Increasingly, digital games are including adaptive features that adjust the level of difficulty to match the skills of individual players. The intention is to improve and prolong the player experience by allowing the player to have the feeling of challenge without it being overwhelming and leading to repeated failure and frustration. Previous work has shown that player experience is indeed improved by such adaptations but also that the player experience can be improved by simply claiming such an adaptation is present even when it is not. It is therefore possible that claims about adaptations and the actual adaptations could interact and not lead to the intended outcomes for the players or worse disappoint players. This paper reports on two studies that were conducted to experimentally investigate the interaction between game adaptations and player information about adaptations on the player experience, specifically their sense of immersion in the game. For this, two games were developed using two different kinds of adaptations to adjust difficulty based on players’ performance in the game. Participants were provided with information about game adaptations independently of whether the adaptations were present. The results suggest that players felt more immersed in the game when told that the game adapts to them, regardless of whether the adaptation was present in the game or not. This effect was observed in both games despite their different adaptations and it remained prominent even during longer gaming sessions. These findings demonstrate that players’ knowledge of adaptations influences their experience independently of adaptations. In this particular context, the knowledge reinforced the experience of the adaptations. This suggests that, at least in some circumstances, developers do not need to be concerned about negative effects of telling players about in-game adaptations

MAPiS 2019 - First MAP-i Seminar: proceedings

This book contains a selection of Informatics papers accepted for presentation and discussion at “MAPiS 2019 - First MAP-i Seminar”, held in Aveiro, Portugal, January 31, 2019. MAPiS is the first conference organized by the MAP-i first year students, in the context of the Seminar course. The MAP-i Doctoral Programme in Computer Science is a joint Doctoral Programme in Computer Science of the University of Minho, the University of Aveiro and the University of Porto. This programme aims to form highly-qualified professionals, fostering their capacity and knowledge to the research area. This Conference was organized by the first grade students attending the Seminar Course. The aim of the course was to introduce concepts which are complementary to scientific and technological education, but fundamental to both completing a PhD successfully and entailing a career on scientific research. The students had contact with the typical procedures and difficulties of organizing and participate in such a complex event. These students were in charge of the organization and management of all the aspects of the event, such as the accommodation of participants or revision of the papers. The works presented in the Conference and the papers submitted were also developed by these students, fomenting their enthusiasm regarding the investigation in the Informatics area. (...)publishe

Automatic Game Parameter Tuning using General Video Game Agents

Automatic Game Design is a subfield of Game Artificial Intelligence that aims to study the usage of AI algorithms for assisting in game design tasks. This dissertation presents a research work in this field, focusing on applying an evolutionary algorithm to video game parameterization. The task we are interested in is player experience. N-Tuple Bandit Evolutionary Algorithm (NTBEA) is an evolutionary algorithm that was recently proposed and successfully applied in game parameterization in a simple domain, which is the first experiment included in this project. To further investigating its ability in evolving game parameters, We applied NTBEA to evolve parameter sets for three General Video Game AI (GVGAI) games, because GVGAI has variety supplies of video games in different types and the framework has already been prepared for parameterization. 9 positive increasing functions were picked as target functions as representations of the player expected score trends. Our initial assumption was that the evolved games should provide the game environments that allow players to obtain score in the same trend as one of these functions. The experiment results confirm this for some functions, and prove that the NTBEA is very much capable of evolving GVGAI games to satisfy this task

Probabilistic Modeling for Game Content Creation and Adaption

Dynamic Difficulty Adjustment studies how games can adapt content totheir users’ skill level, aiming to keep them in flow. Most of these methodsmaximize engagement or minimize churn by adapting factors like the opponentAI or the availability of resources. However, such methods do notmaintain a model of the player, and use technologies that are highly specificto the games in which they are tested (e.g. requiring forward modelsfor enemy AIs based on planning agents). Designers may also intend tofind content that is more difficult/easier on purpose, and current methodsdo not allow for such targeting.This thesis proposes and tests a framework for adapting game content tousers based on Bayesian Optimization, giving designers flexibility whenchoosing which skill level to target. Starting with a design space, a metricto be measured, a prior over this metric, and a target value, our frameworkquickly searches possible levels/tasks for one with ideal difficulty (i.e. closeto the specified target). In the process, our framework maintains a simpledata-driven model of the player, which could be used for further decisionmakingand analysis.We test this framework in two settings: adapting content to planning agentsbased on search algorithms likeMonte Carlo Tree Search and Rolling HorizonEvolution in a dungeon crawler-type game, and adapting both Sudokupuzzles and dungeon crawler levels to players. Our framework successfullyadapts content to planning agents as long as their skill level is not extreme,and takes roughly 7 iterations to find an appropriate Sudoku puzzle.Additionally, instead of relying on designers to specify a real-valued encodingof the content (e.g. the number of pre-filled cells in a Sudoku puzzle),we investigate learning this encoding automatically usingDeep GenerativeModels. In other words, we explore design spaces learned as latent spacesof Variational Autoencoders using tile-based representations of games likeSuperMario Bros and The Legend of Zelda.Our final contribution is a novel way of interpolating, sampling and optimizingin the playable regions of latent spaces of Variational Autoencoders,and addresses the challenge that generative models are not always guaranteedto decode playable content. This contribution, based on differentialgeometry, is inspired by recent advancements in domains like robotics andproteinmodeling. We combine these ideas of safe generation with contentoptimization and propose a restricted version of Bayesian Optimization,which optimizes content inside playable regions. We see a clear trade-off:restricting the latent space to playable regions decreases the diversity ofthe generated content, as well as the quality of the optimal values in theoptimization.In summary, this thesis studies applications of Bayesian Optimization andDeep Generative Models to the problem of creating and adapting gamecontent to users. We develop a framework that quickly finds relevant levelsin settings varying from corpora of levels to the latent spaces of generativemodels, and we show in experiments involving both human and artificialplayers that this framework finds appropriate game content in a few iterations.This framework is readily applicable, and could be used to creategames that learn and adapt to their players.<br/