A Survey of Monte Carlo Tree Search Methods

Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work

EF+EX Forest Algebras

We examine languages of unranked forests definable using the temporal operators EF and EX. We characterize the languages definable in this logic, and various fragments thereof, using the syntactic forest algebras introduced by Bojanczyk and Walukiewicz. Our algebraic characterizations yield efficient algorithms for deciding when a given language of forests is definable in this logic. The proofs are based on understanding the wreath product closures of a few small algebras, for which we introduce a general ideal theory for forest algebras. This combines ideas from the work of Bojanczyk and Walukiewicz for the analogous logics on binary trees and from early work of Stiffler on wreath product of finite semigroups

Characterization of Group-Strategyproof Mechanisms for Facility Location in Strictly Convex Space

We characterize the class of group-strategyproof mechanisms for the single facility location game in any unconstrained strictly convex space. A mechanism is \emph{group-strategyproof}, if no group of agents can misreport so that all its members are \emph{strictly} better off. A strictly convex space is a normed vector space where $\|x+y\|<2$ holds for any pair of different unit vectors $x \neq y$, e.g., any $L_p$ space with $p\in (1,\infty)$. We show that any deterministic, unanimous, group-strategyproof mechanism must be dictatorial, and that any randomized, unanimous, translation-invariant, group-strategyproof mechanism must be \emph{2-dictatorial}. Here a randomized mechanism is 2-dictatorial if the lottery output of the mechanism must be distributed on the line segment between two dictators' inputs. A mechanism is translation-invariant if the output of the mechanism follows the same translation of the input. Our characterization directly implies that any (randomized) translation-invariant approximation algorithm satisfying the group-strategyproofness property has a lower bound of $2$-approximation for maximum cost (whenever $n \geq 3$), and $n/2 - 1$ for social cost. We also find an algorithm that $2$-approximates the maximum cost and $n/2$-approximates the social cost, proving the bounds to be (almost) tight.Comment: Accepted to ACM Conference on Economics and Computation (EC) 202

Dual Random Utility Maximisation

Dual Random Utility Maximisation (dRUM) is Random Utility Maximisation when utility depends on only two states. This class has many relevant behavioural interpretations and practical applications. We show that dRUM is (generically) the only stochastic choice rule that satisfies Regularity and two new properties: Con- stant Expansion (if the choice probability of an alternative is the same across two menus, then it is the same in the merged menu), and Negative Expansion (if the choice probability of an alternative is less than one and differs across two menus, then it vanishes in the merged menu). We extend the theory to menu-dependent state probabilities. This accommodates prominent violations of Regularity such as the attraction, similarity and compromise effects.Publisher PDFOthe

Computabilities of Validity and Satisfiability in Probability Logics over Finite and Countable Models

The $\epsilon$-logic (which is called $\epsilon$E-logic in this paper) of Kuyper and Terwijn is a variant of first order logic with the same syntax, in which the models are equipped with probability measures and in which the $\forall x$ quantifier is interpreted as "there exists a set $A$ of measure $\ge 1 - \epsilon$ such that for each $x \in A$, ...." Previously, Kuyper and Terwijn proved that the general satisfiability and validity problems for this logic are, i) for rational $\epsilon \in (0, 1)$, respectively $\Sigma^1_1$-complete and $\Pi^1_1$-hard, and ii) for $\epsilon = 0$, respectively decidable and $\Sigma^0_1$-complete. The adjective "general" here means "uniformly over all languages." We extend these results in the scenario of finite models. In particular, we show that the problems of satisfiability by and validity over finite models in $\epsilon$E-logic are, i) for rational $\epsilon \in (0, 1)$, respectively $\Sigma^0_1$- and $\Pi^0_1$-complete, and ii) for $\epsilon = 0$, respectively decidable and $\Pi^0_1$-complete. Although partial results toward the countable case are also achieved, the computability of $\epsilon$E-logic over countable models still remains largely unsolved. In addition, most of the results, of this paper and of Kuyper and Terwijn, do not apply to individual languages with a finite number of unary predicates. Reducing this requirement continues to be a major point of research. On the positive side, we derive the decidability of the corresponding problems for monadic relational languages --- equality- and function-free languages with finitely many unary and zero other predicates. This result holds for all three of the unrestricted, the countable, and the finite model cases. Applications in computational learning theory, weighted graphs, and neural networks are discussed in the context of these decidability and undecidability results.Comment: 47 pages, 4 tables. Comments welcome. Fixed errors found by Rutger Kuype

Generalized Domination.

This thesis develops the theory of the everywhere domination relation between functions from one infinite cardinal to another. When the domain of the functions is the cardinal of the continuum and the range is the set of natural numbers, we may restrict our attention to nicely definable functions from R to N. When we consider a class of such functions which contains all Baire class one functions, it becomes possible to encode information into these functions which can be decoded from any dominator. Specifically, we show that there is a generalized Galois-Tukey connection from the appropriate domination relation to a classical ordering studied in recursion theory. The proof techniques are developed to prove new implications regarding the distributivity of complete Boolean algebras. Next, we investigate a more technical relation relevant to the study of Borel equivalence relations on R with countable equivalence classes. We show than an analogous generalized Galois-Tukey connection exists between this relation and another ordering studied in recursion theory.PhDMathematicsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/113539/1/danhath_1.pd

Monte Carlo Tree Search for games with Hidden Information and Uncertainty

Monte Carlo Tree Search (MCTS) is an AI technique that has been successfully applied to many deterministic games of perfect information, leading to large advances in a number of domains, such as Go and General Game Playing. Imperfect information games are less well studied in the field of AI despite being popular and of significant commercial interest, for example in the case of computer and mobile adaptations of turn based board and card games. This is largely because hidden information and uncertainty leads to a large increase in complexity compared to perfect information games. In this thesis MCTS is extended to games with hidden information and uncertainty through the introduction of the Information Set MCTS (ISMCTS) family of algorithms. It is demonstrated that ISMCTS can handle hidden information and uncertainty in a variety of complex board and card games. This is achieved whilst preserving the general applicability of MCTS and using computational budgets appropriate for use in a commercial game. The ISMCTS algorithm is shown to outperform the existing approach of Perfect Information Monte Carlo (PIMC) search. Additionally it is shown that ISMCTS can be used to solve two known issues with PIMC search, namely strategy fusion and non-locality. ISMCTS has been integrated into a commercial game, Spades by AI Factory, with over 2.5 million downloads. The Information Capture And ReUSe (ICARUS) framework is also introduced in this thesis. The ICARUS framework generalises MCTS enhancements in terms of information capture (from MCTS simulations) and reuse (to improve MCTS tree and simulation policies). The ICARUS framework is used to express existing enhancements, to provide a tool to design new ones, and to rigorously define how MCTS enhancements can be combined. The ICARUS framework is tested across a wide variety of games