Current Frontiers in Computer Go

International audienceThis paper presents the recent technical advances in Monte-Carlo Tree Search for the Game of Go, shows the many similarities and the rare differences between the current best programs, and reports the results of the computer-Go event organized at FUZZ-IEEE 2009, in which four main Go programs played against top level humans. We see that in 9x9, computers are very close to the best human level, and can be improved easily for the opening book; whereas in 19x19, handicap 7 is not enough for the computers to win against top level professional players, due to some clearly understood (but not solved) weaknesses of the current algorithms. Applications far from the game of Go are also cited. Importantly, the first ever win of a computer against a 9th Dan professional player in 9x9 Go occurred in this event

Intelligent Agents for the Game of Go

International audienceMonte-Carlo Tree Search (MCTS) is a very efficient recent technology for games and planning, par- ticularly in the high-dimensional case, when the number of time steps is moderate and when there is no natural evaluation function. Surprisingly, MCTS makes very little use of learning. In this paper, we present four techniques (ontologies, Bernstein races, Contextual Monte-Carlo and poolRave) for learning agents in Monte-Carlo Tree Search, and experiment them in difficult games and in particular, the game of Go

Monte-Carlo tree search with heuristic knowledge: A novel way in solving capturing and life and death problems in Go

Monte-Carlo (MC) tree search is a new research field. Its effectiveness in searching large state spaces, such as the Go game tree, is well recognized in the computer Go community. Go domain- specific heuristics and techniques as well as domain-independent heuristics and techniques are sys- tematically investigated in the context of the MC tree search in this dissertation. The search extensions based on these heuristics and techniques can significantly improve the effectiveness and efficiency of the MC tree search. Two major areas of investigation are addressed in this dissertation research: I. The identification and use of the effective heuristic knowledge in guiding the MC simulations, II. The extension of the MC tree search algorithm with heuristics. Go, the most challenging board game to the machine, serves as the test bed. The effectiveness of the MC tree search extensions is demonstrated through the performances of Go tactic problem solvers using these techniques. The main contributions of this dissertation include: 1. A heuristics based Monte-Carlo tactic tree search framework is proposed to extend the standard Monte-Carlo tree search. 2. (Go) Knowledge based heuristics are systematically investigated to improve the Monte-Carlo tactic tree search. 3. Pattern learning is demonstrated as effective in improving the Monte-Carlo tactic tree search. 4. Domain knowledge independent tree search enhancements are shown as effective in improving the Monte-Carlo tactic tree search performances. 5. A strong Go Tactic solver based on proposed algorithms outperforms traditional game tree search algorithms. The techniques developed in this dissertation research can benefit other game domains and ap- plication fields

Using Opponent Modeling to Adapt Team Play in American Football

An issue with learning effective policies in multi-agent adversarial games is that the size of the search space can be prohibitively large when the actions of both teammates and opponents are considered simultaneously. Opponent modeling, predicting an opponent’s actions in advance of execution, is one approach for selecting actions in adversarial settings, but it is often performed in an ad hoc way. In this chapter, we introduce several methods for using opponent modeling, in the form of predictions about the players ’ physical movements, to learn team policies. To explore the problem of decision-making in multi-agent adversarial scenarios, we use our approach for both offline play generation and real-time team response in the Rush 2008 American football simulator. Simultaneously predicting the movement trajectories, future reward, and play strategies of mul-tiple players in real-time is a daunting task but we illustrate how it is possible to divide and conquer this problem with an assortment of data-driven models

Monte Carlo -puuhakua käyttävien tekoälymenetelmien soveltuvuus vuoropohjaisiin strategiapeleihin

Tässä tutkielmassa tarkastellaan Monte Carlo -puuhaun soveltuvuutta vuoropohjaisten strategiapelien tekoälyratkaisuihin kirjallisuuskatsausta hyödyntäen. Aluksi esitellään sekä minimax-algoritmi että Monte Carlo -puuhaku suosittuine muunnelmineen, ja sen jälkeen perehdytään tarkemmin neljään vuoropohjaiseen strategiapeliin: Shakkiin, go-lautapeliin, pokeriin ja Magic: The Gathering -keräilykorttipeliin. Kunkin neljän pelin kohdalla tutustutaan kyseisen pelin tekoälylle asettamiin haasteisiin, olemassa oleviin tekoälyratkaisuihin ja etenkin Monte Carlo -puuhakua hyödyntäviin tekoälytoimijoihin. Lopuksi luodaan vielä lyhyt katsaus joukkoon Monte Carlo -puuhakua hyödyntäviä tekoälyratkaisuja muiden vuoropohjaisten strategiapelien kohdalla. Huomataan, että Monte Carlo -puuhakua käyttämällä saavutetaan sen yleispätevän luonteen vuoksi usein merkittäviä hyötyjä etenkin sellaisissa peleissä, joille mielekkään evaluaatiofunktion kirjoittaminen on hankalaa

Exploiting Opponent Modeling For Learning In Multi-agent Adversarial Games

An issue with learning effective policies in multi-agent adversarial games is that the size of the search space can be prohibitively large when the actions of both teammates and opponents are considered simultaneously. Opponent modeling, predicting an opponent’s actions in advance of execution, is one approach for selecting actions in adversarial settings, but it is often performed in an ad hoc way. In this dissertation, we introduce several methods for using opponent modeling, in the form of predictions about the players’ physical movements, to learn team policies. To explore the problem of decision-making in multi-agent adversarial scenarios, we use our approach for both offline play generation and real-time team response in the Rush 2008 American football simulator. Simultaneously predicting the movement trajectories, future reward, and play strategies of multiple players in real-time is a daunting task but we illustrate how it is possible to divide and conquer this problem with an assortment of data-driven models. By leveraging spatio-temporal traces of player movements, we learn discriminative models of defensive play for opponent modeling. With the reward information from previous play matchups, we use a modified version of UCT (Upper Conference Bounds applied to Trees) to create new offensive plays and to learn play repairs to counter predicted opponent actions. iii In team games, players must coordinate effectively to accomplish tasks while foiling their opponents either in a preplanned or emergent manner. An effective team policy must generate the necessary coordination, yet considering all possibilities for creating coordinating subgroups is computationally infeasible. Automatically identifying and preserving the coordination between key subgroups of teammates can make search more productive by pruning policies that disrupt these relationships. We demonstrate that combining opponent modeling with automatic subgroup identification can be used to create team policies with a higher average yardage than either the baseline game or domain-specific heuristics

Combining tactical search and monte-carlo in the game of go

We present a way to integrate search and Monte-Carlo methods in the game of Go. Our program uses search to find the status of tactical goals, builds groups, selects interesting goals, and computes statistics on the realization of tactical goals during the random games. The mean score of the random games where a selected tactical goal has been reached and the mean score of the random games where it has failed are computed. They are used to evaluate the selected goals. Experimental results attest that combining search and Monte-Carlo significantly improves the playing level

Combining Tactical Search and Monte-Carlo in the Game of Go

Abstract- We present a way to integrate search andMonte-Carlo methods in the game of Go. Our program uses search to find the status of tactical goals, buildsgroups, selects interesting goals, and computes statistics on the realization of tactical goals during the randomgames. The mean score of the random games where a selected tactical goal has been reached and the mean scoreof the random games where it has failed are computed. They are used to evaluate the selected goals. Experimen-tal results attest that combining search and Monte-Carlo significantly improves the playing level. 1 Introduction Monte-Carlo Go has been invented in 1993 [1]; it is a sim-ple way to program a decent computer Go program usin