Evaluating the Effects on Monte Carlo Tree Search of Predicting Co-operative Agent Behaviour

This thesis explores the effects of including an agent-modelling strategy into Monte-Carlo Tree Search. This is to explore how the effects of such modelling might be used to increase the performance of agents in co-operative environments such as games. The research is conducted using two applications. The first is a co-operative 2-player puzzle game in which a perfect model outperforms an agent that makes the assumption the other agent plays randomly. The second application is the partially observable co-operative card game Hanabi, in which the predictor variant is able to outperform both a standard variant of MCTS and a version that assumes a fixed-strategy for the paired agents. This thesis also investigates a technique for learning player strategies off-line based on saved game logs for use in modelling

Neuroevolution in Games: State of the Art and Open Challenges

This paper surveys research on applying neuroevolution (NE) to games. In neuroevolution, artificial neural networks are trained through evolutionary algorithms, taking inspiration from the way biological brains evolved. We analyse the application of NE in games along five different axes, which are the role NE is chosen to play in a game, the different types of neural networks used, the way these networks are evolved, how the fitness is determined and what type of input the network receives. The article also highlights important open research challenges in the field.Comment: - Added more references - Corrected typos - Added an overview table (Table 1

A panorama of artificial and computational intelligence in games

This paper attempts to give a high-level overview of the field of artificial and computational intelligence (AI/CI) in games, with particular reference to how the different core research areas within this field inform and interact with each other, both actually and potentially. We identify ten main research areas within this field: NPC behavior learning, search and planning, player modeling, games as AI benchmarks, procedural content generation, computational narrative, believable agents, AI-assisted game design, general game artificial intelligence and AI in commercial games. We view and analyze the areas from three key perspectives: (1) the dominant AI method(s) used under each area; (2) the relation of each area with respect to the end (human) user; and (3) the placement of each area within a human-computer (player-game) interaction perspective. In addition, for each of these areas we consider how it could inform or interact with each of the other areas; in those cases where we find that meaningful interaction either exists or is possible, we describe the character of that interaction and provide references to published studies, if any. We believe that this paper improves understanding of the current nature of the game AI/CI research field and the interdependences between its core areas by providing a unifying overview. We also believe that the discussion of potential interactions between research areas provides a pointer to many interesting future research projects and unexplored subfields.peer-reviewe

Automated planning for pathfinding in real-time strategy games

This thesis is focused on the design of a new path planning algorithm to solve path planning problems in dynamic, partially observable and real-time environments such as Real-Time Strategy(RTS) games. The emphasis is put on fast action selection motivating the use of Monte-Carlo planning techniques. Three main contributions are presented in this thesis. The first contribution is a Monte-Carlo planning technique, called MCRT, that performs selective action sampling and limits how many times a particular state-action pair is explored to balance the trade-off between exploration of new actions and exploitation of the current best action. The thesis also presents two variations of MCRT as the second contribution. The first variation of MCRT randomly selects an action as a sample at each state seen during the look-ahead search. The second variation, called MCRT-CAS, performs the selective action sampling using corridors. The third contribution is the design of four real-time path planners that exploit MCRT and its variations to solve path planning problems in real-time. Three of these planners are empirically evaluated using four standard pathfinding benchmarks (and over 1000 instances). Performance of these three planners is compared against two recent rival algorithms (Real-time D*-Lite (RTD) and Local Search Space-Learning Real-Time A* (LSS-LRTA)). These rival algorithms are based on real-time heuristic search. The results show that a variation of MOCART, called MOCART-CAS, performs action selection significantly faster than the rival planners. The fourth planner, called the MG-MOCART planner, is evaluated using a typical Real-Time Strategy game. The MG-MOCART planner can solve the path planning problems with multiple goals. This planner is compared against four rivals: Upper Confidence bounds applied to Trees (UCT), LSS-LRTA, Real-Time Dynamic Programming (RTDP) and a rapidly-exploring random tree (RRT) planner. The performance is measured using score and planning cost. The results show that the MG-MOCART planner performs better than its rival techniques with respect to score and planning cost.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

A Survey of Monte Carlo Tree Search Methods

Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work

Combining Optimization and Machine Learning for the Formation of Collectives

This thesis considers the problem of forming collectives of agents for real-world applications aligned with Sustainable Development Goals (e.g., shared mobility and cooperative learning). Such problems require fast approaches that can produce solutions of high quality for hundreds of agents. With this goal in mind, existing solutions for the formation of collectives focus on enhancing the optimization approach by exploiting the characteristics of a domain. However, the resulting approaches rely on specific domain knowledge and are not transferable to other collective formation problems. Therefore, approaches that can be applied to various problems need to be studied in order to obtain general approaches that do not require prior knowledge of the domain. Along these lines, this thesis proposes a general approach for the formation of collectives based on a novel combination of machine learning and an \emph{Integer Linear Program}. More precisely, a machine learning component is trained to generate a set of promising collectives that are likely to be part of a solution. Then, such collectives and their corresponding utility values are introduced into an \emph{Integer Linear Program} which finds a solution to the collective formation problem. In that way, the machine learning component learns the structure shared by ``good'' collectives in a particular domain, making the whole approach valid for various applications. In addition, the empirical analysis conducted on two real-world domains (i.e., ridesharing and team formation) shows that the proposed approach provides solutions of comparable quality to state-of-the-art approaches specific to each domain. Finally, this thesis also shows that the proposed approach can be extended to problems that combine the formation of collectives with other optimization objectives. Thus, this thesis proposes an extension of the collective formation approach for assigning pickup and delivery locations to robots in a warehouse environment. The experimental evaluation shows that, although it is possible to use the collective formation approach for that purpose, several improvements are required to compete with state-of-the-art approaches. Overall, this thesis aims to demonstrate that machine learning can be successfully intertwined with classical optimization approaches for the formation of collectives by learning the structure of a domain, reducing the need for ad-hoc algorithms devised for a specific application