Performance of the Parallelized Monte-Carlo Tree Search Approach for Dots and Boxes

The Monte-Carlo tree search (MCTS) is a method designed to solve difficult learning problems. MCTS performs random simulations from the current situation and stores the results in order to distinguish decisions based on their past success. MCTS will then select the best decision and finally repeat the process. Parallelizing the MCTS means to divide the learning process among independent learners. Then, after a fixed number of simulations, the data is shared and combined. Past research has shown that this approach is faster than non-parallelized approaches. Therefore, we anticipated that the time reduced from dividing the learning outweighs the potential costs from redundant learning. Since it is often difficult to determine the effectiveness of algorithms in complex environments, it is sometimes more advantageous to develop strategies in simple environments such as games that can then be translated for use in broader real-life fields. In this project, we explored how controlling various resources affected the win-ratio performance of the game Dots and Boxes learned through a parallelized Monte Carlo Tree Search approach. The factors that we manipulated included the number of simulations, the number of independent learners, the amount of information shared from these independent learners, and how frequently the independent learners share. The win-ratio performance was determined by taking the number of wins over the number of total games. An algorithm is presented with our findings, along with details and results of our modified Monte-Carlo tree search implementation

Monte-Carlo tree search for persona based player modeling

Is it possible to conduct player modeling without any players? In this paper we use Monte-Carlo Tree Search-controlled procedural personas to simulate a range of decision making styles in the puzzle game MiniDungeons 2. The purpose is to provide a method for synthetic play testing of game levels with synthetic players based on designer intuition and experience. Five personas are constructed, representing five different decision making styles archetypal for the game. The personas vary solely in the weights of decision-making utilities that describe their valuation of a set affordances in MiniDungeons 2. By configuring these weights using designer expert knowledge, and passing the configurations directly to the MCTS algorithm, we make the personas exhibit a number of distinct decision making and play styles.The research was supported, in part, by the FP7 ICT project C2Learn (project no: 318480), the FP7 Marie Curie CIG project AutoGameDesign (project no: 630665), and by the Stibo Foundation Travel Bursary Grant for Global IT Talents.peer-reviewe

Tekoäly ja go-peli

Tässä tutkielmassa käsittelen, kuinka kiinalaista go-peliä voidaan pelata tietokoneella käyttäen erilaisia tekoälytekniikoita. Go-pelissä on suuri haarautumiskerroin eli pelitilanteissa on useimmiten mahdollista tehdä lukuisia eri siirtoja. Tämän aiheuttamat ongelmat ovat yksi syy siihen, että go-peliä pelaavat ohjelmat ovat vielä paljon huonompia kuin parhaat ihmispelaajat. Esittelen tutkielmassa muutamia tekniikoita, kuten Monte Carlo -puuhaku, joilla on päästy tämän hetken parhaisiin go-peliä pelaaviin tietokoneohjelmiin. Lisäksi käsittelen tekniikoita, joita on lisätty Monte Carlo -puuhakuun, jotta puuhakua saataisiin tehostettua toimimaan paremmin

A Survey of Monte Carlo Tree Search Methods

Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work

An AI for dominion based on Monte-Carlo methods

Masteroppgave i Informasjons- og kommunikasjonsteknologi IKT590 Universitetet i Agder 2014To the best of our knowledge there exists no Arti_cial Intelligence (AI)for Dominion which uses Monte Carlo methods, that is competitive on ahuman level. This thesis presents such an AI, and tests it against someof the top Dominion strategies available. Although in a limited testingenvironment, the results show that our AI is capable of competing withhuman players, while keeping processing time per move at an acceptablelevel for human players. Although the approach for our AI is built onprevious knowledge about Upper Con_dence Bounds (UCB) and UCBapplied to Trees (UCT), an approach for handling the stochastic elementof drawing cards is presented, as well as an approach for handling in-teraction between players. Our best solutions win 87.5% games againstmoderately experienced human players, and outperforms the successful,rule-based, Dominion strategies SingleWitch and DoubleWitch both witha win percentage of 68.5%.Keywords: Dominion, UCT, UCB, AI, Multi-Armed Bandit Problem,Monte-Carlo, Tree Searc

Object Detection with Active Sample Harvesting

The work presented in this dissertation lies in the domains of image classification, object detection, and machine learning. Whether it is training image classifiers or object detectors, the learning phase consists in finding an optimal boundary between populations of samples. In practice, all the samples are not equally important: some examples are trivially classified and do not bring much to the training, while others close to the boundary or misclassified are the ones that truly matter. Similarly, images where the samples originate from are not all rich in informative samples. However, most training procedures select samples and images uniformly or weight them equally. The common thread of this dissertation is how to efficiently find the informative samples/images for training. Although we never consider all the possible samples "in the world", our purpose is to select the samples in a smarter manner, without looking at all the available ones. The framework adopted in this work consists in organising the data (samples or images) in a tree to reflect the statistical regularities of the training samples, by putting "similar" samples in the same branch. Each leaf carries a sample and a weight related to the "importance" of the corresponding sample, and each internal node carries statistics about the weights below. The tree is used to select the next sample/image for training, by applying a sampling policy, and the "importance" weights are updated accordingly, to bias the sampling towards informative samples/images in future iterations. Our experiments show that, in the various applications, properly focusing on informative images or informative samples improves the learning phase by either reaching better performances faster or by reducing the training loss faster

Structured parallel programming for Monte Carlo Tree Search

The thesis is part of a bigger project, the HEPGAME (High Energy Physics Game). The main objective for HEPGAME is the utilization of AI solutions, particularly by using MCTS for simplification of HEP calculations. One of the issues is solving mathematical expressions of interest with millions of terms. These calculations can be solved with the FORM program, which is software for symbolic manipulation. Since these calculations are computationally intensive and take a large amount of time, the FORM program was parallelized to solve them in a reasonable amount of time.Therefore, any new algorithm based on MCTS, should also be parallelized. This requirement was behind the problem statement of the thesis: “How do we design a structured pattern-based parallel programming approach for efficient parallelism of MCTS for both multi-core and manycore shared-memory machines?”.To answer this question, the thesis approached the MCTS parallelization problem in three levels: (1) implementation level, (2) data structure level, and (3) algorithm level.In the implementation level, we proposed task-level parallelization over thread-level parallelization. Task-level parallelization provides us with efficient parallelism for MCTS to utilize cores on both multi-core and manycore machines.In the data structure level, we presented a lock-free data structure that guarantees the correctness. A lock-free data structure (1) removes the synchronization overhead when a parallel program needs many tasks to feed its cores and (2) improves both performance and scalability.In the algorithm level, we first explained how to use pipeline pattern for parallelization of MCTS to overcome search overhead. Then, through a step by step approach, we were able to propose and detail the structured parallel programming approach for Monte Carlo Tree Search.Algorithms and the Foundations of Software technolog

Scalability and Parallelization of Monte-Carlo Tree Search

Monte-Carlo Tree Search is now a well established algorithm, in games and beyond. We analyze its scalability, and in particular its limitations, and the implications in terms of parallelization, in particular for our program MoGo but also for our Havannah program Shakti. In particular, we get a good efficiency for the parallel versions, both for multicore machines and for message-passing machines, but in spite of promising results in self-play there are situations for which increasing the time per move does not solve anything, and therefore parallelization is not the solution either. Nonetheless, for problems on which the Monte-Carlo part is less biased than in Go, parallelization should be very efficient even without shared memory