Monte Carlo Tree Search with Heuristic Evaluations using Implicit Minimax Backups

Monte Carlo Tree Search (MCTS) has improved the performance of game engines in domains such as Go, Hex, and general game playing. MCTS has been shown to outperform classic alpha-beta search in games where good heuristic evaluations are difficult to obtain. In recent years, combining ideas from traditional minimax search in MCTS has been shown to be advantageous in some domains, such as Lines of Action, Amazons, and Breakthrough. In this paper, we propose a new way to use heuristic evaluations to guide the MCTS search by storing the two sources of information, estimated win rates and heuristic evaluations, separately. Rather than using the heuristic evaluations to replace the playouts, our technique backs them up implicitly during the MCTS simulations. These minimax values are then used to guide future simulations. We show that using implicit minimax backups leads to stronger play performance in Kalah, Breakthrough, and Lines of Action.Comment: 24 pages, 7 figures, 9 tables, expanded version of paper presented at IEEE Conference on Computational Intelligence and Games (CIG) 2014 conferenc

New Results for Domineering from Combinatorial Game Theory Endgame Databases

We have constructed endgame databases for all single-component positions up to 15 squares for Domineering, filled with exact Combinatorial Game Theory (CGT) values in canonical form. The most important findings are as follows. First, as an extension of Conway's [8] famous Bridge Splitting Theorem for Domineering, we state and prove another theorem, dubbed the Bridge Destroying Theorem for Domineering. Together these two theorems prove very powerful in determining the CGT values of large positions as the sum of the values of smaller fragments, but also to compose larger positions with specified values from smaller fragments. Using the theorems, we then prove that for any dyadic rational number there exist Domineering positions with that value. Second, we investigate Domineering positions with infinitesimal CGT values, in particular ups and downs, tinies and minies, and nimbers. In the databases we find many positions with single or double up and down values, but no ups and downs with higher multitudes. However, we prove that such single-component ups and downs easily can be constructed. Further, we find Domineering positions with 11 different tinies and minies values. For each we give an example. Next, for nimbers we find many Domineering positions with values up to *3. This is surprising, since Drummond-Cole [10] suspected that no *2 and *3 positions in standard Domineering would exist. We show and characterize many *2 and *3 positions. Finally, we give some Domineering positions with values interesting for other reasons. Third, we have investigated the temperature of all positions in our databases. There appears to be exactly one position with temperature 2 (as already found before) and no positions with temperature larger than 2. This supports Berlekamp's conjecture that 2 is the highest possible temperature in Domineering

A Survey of Monte Carlo Tree Search Methods

Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work

MCTS-minimax hybrids with state evaluations

Monte-Carlo Tree Search (MCTS) has been found to show weaker play than minimax-based search in some tactical game domains. In order to combine the tactical strength of minimax and the strategic strength of MCTS, MCTS-minimax hybrids have been proposed in prior work. This arti

Symbolic Search in Planning and General Game Playing

Search is an important topic in many areas of AI. Search problems often result in an immense number of states. This work addresses this by using a special datastructure, BDDs, which can represent large sets of states efficiently, often saving space compared to explicit representations. The first part is concerned with an analysis of the complexity of BDDs for some search problems, resulting in lower or upper bounds on BDD sizes for these. The second part is concerned with action planning, an area where the programmer does not know in advance what the search problem will look like. This part presents symbolic algorithms for finding optimal solutions for two different settings, classical and net-benefit planning, as well as several improvements to these algorithms. The resulting planner was able to win the International Planning Competition IPC 2008. The third part is concerned with general game playing, which is similar to planning in that the programmer does not know in advance what game will be played. This work proposes algorithms for instantiating the input and solving games symbolically. For playing, a hybrid player based on UCT and the solver is presented

Exploring search space trees using an adapted version of Monte Carlo tree search for combinatorial optimization problems

In this article, a novel approach to solve combinatorial optimization problems is proposed. This approach makes use of a heuristic algorithm to explore the search space tree of a problem instance. The algorithm is based on Monte Carlo tree search, a popular algorithm in game playing that is used to explore game trees. By leveraging the combinatorial structure of a problem, several enhancements to the algorithm are proposed. These enhancements aim to efficiently explore the search space tree by pruning subtrees, using a heuristic simulation policy, reducing the domains of variables by eliminating dominated value assignments and using a beam width. They are demonstrated for two specific combinatorial optimization problems: the quay crane scheduling problem with non-crossing constraints and the 0-1 knapsack problem. Computational results show that the algorithm achieves promising results for both problems and eight new best solutions for a benchmark set of instances are found for the former problem. These results indicate that the algorithm is competitive with the state-of-the-art. Apart from this, the results also show evidence that the algorithm is able to learn to correct the incorrect choices made by constructive heuristics

Turn-Based War Chess Model and Its Search Algorithm per Turn

War chess gaming has so far received insufficient attention but is a significant component of turn-based strategy games (TBS) and is studied in this paper. First, a common game model is proposed through various existing war chess types. Based on the model, we propose a theory frame involving combinational optimization on the one hand and game tree search on the other hand. We also discuss a key problem, namely, that the number of the branching factors of each turn in the game tree is huge. Then, we propose two algorithms for searching in one turn to solve the problem: (1) enumeration by order; (2) enumeration by recursion. The main difference between these two is the permutation method used: the former uses the dictionary sequence method, while the latter uses the recursive permutation method. Finally, we prove that both of these algorithms are optimal, and we analyze the difference between their efficiencies. An important factor is the total time taken for the unit to expand until it achieves its reachable position. The factor, which is the total number of expansions that each unit makes in its reachable position, is set. The conclusion proposed is in terms of this factor: Enumeration by recursion is better than enumeration by order in all situations

Games, puzzles, and computation

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2006.Includes bibliographical references (p. 147-153).There is a fundamental connection between the notions of game and of computation. At its most basic level, this is implied by any game complexity result, but the connection is deeper than this. One example is the concept of alternating nondeterminism, which is intimately connected with two-player games. In the first half of this thesis, I develop the idea of game as computation to a greater degree than has been done previously. I present a general family of games, called Constraint Logic, which is both mathematically simple and ideally suited for reductions to many actual board games. A deterministic version of Constraint Logic corresponds to a novel kind of logic circuit which is monotone and reversible. At the other end of the spectrum, I show that a multiplayer version of Constraint Logic is undecidable. That there are undecidable games using finite physical resources is philosophically important, and raises issues related to the Church-Turing thesis. In the second half of this thesis, I apply the Constraint Logic formalism to many actual games and puzzles, providing new hardness proofs. These applications include sliding-block puzzles, sliding-coin puzzles, plank puzzles, hinged polygon dissections, Amazons, Kohane, Cross Purposes, Tip over, and others.(cont.) Some of these have been well-known open problems for some time. For other games, including Minesweeper, the Warehouseman's Problem, Sokoban, and Rush Hour, I either strengthen existing results, or provide new, simpler hardness proofs than the original proofs.by Robert Aubrey Hearn.Ph.D