Monte Carlo -puuhakua käyttävien tekoälymenetelmien soveltuvuus vuoropohjaisiin strategiapeleihin

Tässä tutkielmassa tarkastellaan Monte Carlo -puuhaun soveltuvuutta vuoropohjaisten strategiapelien tekoälyratkaisuihin kirjallisuuskatsausta hyödyntäen. Aluksi esitellään sekä minimax-algoritmi että Monte Carlo -puuhaku suosittuine muunnelmineen, ja sen jälkeen perehdytään tarkemmin neljään vuoropohjaiseen strategiapeliin: Shakkiin, go-lautapeliin, pokeriin ja Magic: The Gathering -keräilykorttipeliin. Kunkin neljän pelin kohdalla tutustutaan kyseisen pelin tekoälylle asettamiin haasteisiin, olemassa oleviin tekoälyratkaisuihin ja etenkin Monte Carlo -puuhakua hyödyntäviin tekoälytoimijoihin. Lopuksi luodaan vielä lyhyt katsaus joukkoon Monte Carlo -puuhakua hyödyntäviä tekoälyratkaisuja muiden vuoropohjaisten strategiapelien kohdalla. Huomataan, että Monte Carlo -puuhakua käyttämällä saavutetaan sen yleispätevän luonteen vuoksi usein merkittäviä hyötyjä etenkin sellaisissa peleissä, joille mielekkään evaluaatiofunktion kirjoittaminen on hankalaa

A Profitable Online Poker Agent

Jogos de informação incompleta tais como poker são uma fonte contínua de estudo e pesquisa no âmbito da inteligência artificial. No poker problemas como: modelação de oponentes; gestão de riscos e detecção de bluffs representam um desafio. O desenvolvimento de agentes capazes de considerar esses problemas e realizar cálculos probabilísticos é considerado como uma tarefa árdua de se realizar, uma vez que é exigida uma adaptação dinâmica para que seja criado um agente de poker robusto. Esta tese irá focar-se no desenvolvimento de um agente de poker capaz de jogar contra jogadores humanos e alcançar a adaptação dinâmica necessária para superar alguns jogadores humanos de poker online. Algo que será possível usando um conjunto de informações sobre cada jogador que o agente enfrenta. Utilizando como auxílio o Holdem Manager, uma ferramenta que regista mãos jogadas em salas de poker online, é possível obter estatísticas sobre todos os jogadores que o agente enfrenta nas mesas. O agente é capaz de explorar algumas destas estatísticas de maneira que possa decidir melhor sobre a acção a tomar. Alguns factores como quão agressivo é um adversário, a posição ocupada na mesa, quantos jogadores estão envolvidos, quanto dinheiro está em causa, e o par de cartas que o agente recebe são uma pequena porção do conjunto de informações utilizadas na determinação do comportamento do agente. Este agente foi desenvolvido baseando-se numa estratégia "short stack", e modelando adversários com o auxílio do conjunto de informações reunido através do Holdem Manager. Pela primeira vez na literatura do Computer Poker, são apresentados resultados de jogos de poker online, num ambiente controlado, contra jogadores humanos sem estes saberem que estão em jogo contra um agente. O agente é capaz de jogar poker online ao vivo contra jogadores humanos, e apresenta um pequeno lucro na vertente Texas Hold'em em micro limites6 de apostas, nomeadamente 0.01 e 0.02 cêntimos.Games of incomplete information, such as poker, are a continuous source of research and study in the area of artificial intelligence. Poker presents challenging problems such as opponent modeling, risk management and bluff detection. The development of agents capable of probabilistic calculations considering those problems is considered to be difficult to achieve, since dynamic adaption is required in order to create a robust computer poker player. This thesis focuses on the development of a poker agent able to play against human players and aiming to achieve the dynamic adaptation needed to beat some human players online. This will be achieved by using some sets of information about each player the agent plays against. Using Holdem Manager, a tool that registers the hands played in an online poker room; it is possible to obtain statistics about every player the agent is playing against. The agent is able to explore some of these statistics so that it can better decide on which action to take. Some factors like how aggressive an opponent is, the position held at the table, how many players are involved, how much money is involved, and the hand dealt to the agent are a few portions of the information sets used to compute the agent's behavior. This agent was developed based on a short-stack strategy, and through the use of the sets of information provided by the Holdem Manager. For the first time in the Computer Poker literature, results on online Poker agent games versus human players in a controlled environment are presented, and without the players being aware their opponent was a computer agent. The agent is able to play live online poker versus human players, and presents a small profit in the No-Limit Texas Hold'em poker game at micro stakes, namely 0.02 and 0.01 cents

Applying machine learning techniques to an imperfect information game

The game of poker presents a challenging game to Artificial Intelligence researchers because it is a complex asymmetric information game. In such games, a player can improve his performance by inferring the private information held by the other players from their prior actions. A novel connectionist structure was designed to play a version of poker (multi-player limit Hold‟em). This allows simple reinforcement learning techniques to be used which previously not been considered for the game of multi-player hold‟em. A related hidden Markov model was designed to be fitted to records of poker play without using any private information. Belief vectors generated by this model provide a more convenient and flexible representation of an opponent‟s action history than alternative approaches. The structure was tested in two settings. Firstly self-play simulation was used to generate an approximation to a Nash equilibrium strategy. A related, but slower, rollout strategy that uses Monte-Carlo samples was used to evaluate the performance. Secondly the structure was used to model and hence exploit a population of opponents within a relatively small number of games. When and how to adapt quickly to new opponents are open questions in poker AI research. A opponent model with a small number of discrete types is used to identify the largest differences in strategy between members of the population. A commercial software package (Poker Academy) was used to provide a population of sophisticated opponents to test against. A series of experiments was conducted to compare adaptive and static systems. All systems showed positive results but surprisingly the adaptive systems did not show a significant improvement over similar static systems. The possible reasons for this result are discussed. This work formed the basis of a series of entries to the computer poker competition hosted at the annual conferences of the Association for the Advancement of Artificial Intelligence (AAAI). Its best rankings were 3rd in the 2006 6-player limit hold‟em competition and 2nd in the 2008 3-player limit hold‟em competition

An Exploitative Monte-Carlo Poker Agent

Abstract. We describe the poker agent AKI-REALBOT which participated i