Building a computer poker agent with emphasis on opponent modeling

Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (pages 53-54).In this thesis, we present a computer agent for the game of no-limit Texas Hold'em Poker for two players. Poker is a partially observable, stochastic, multi-agent, sequential game. This combination of characteristics makes it a very challenging game to master for both human and computer players. We explore this problem from an opponent modeling perspective, using data mining to build a database of player styles that allows our agent to quickly model the strategy of any new opponent. The opponent model is then used to develop a robust counter strategy. A simpler version of this agent modified for a three player game was able to win the 2011 MIT Poker Bot Competition.by Jian Huang.M. Eng

Solving Large Extensive-Form Games with Strategy Constraints

Extensive-form games are a common model for multiagent interactions with imperfect information. In two-player zero-sum games, the typical solution concept is a Nash equilibrium over the unconstrained strategy set for each player. In many situations, however, we would like to constrain the set of possible strategies. For example, constraints are a natural way to model limited resources, risk mitigation, safety, consistency with past observations of behavior, or other secondary objectives for an agent. In small games, optimal strategies under linear constraints can be found by solving a linear program; however, state-of-the-art algorithms for solving large games cannot handle general constraints. In this work we introduce a generalized form of Counterfactual Regret Minimization that provably finds optimal strategies under any feasible set of convex constraints. We demonstrate the effectiveness of our algorithm for finding strategies that mitigate risk in security games, and for opponent modeling in poker games when given only partial observations of private information.Comment: Appeared in AAAI 201

Poker as a Skill Game: Rational vs Irrational Behaviors

In many countries poker is one of the most popular card games. Although each variant of poker has its own rules, all involve the use of money to make the challenge meaningful. Nowadays, in the collective consciousness, some variants of poker are referred to as games of skill, others as gambling. A poker table can be viewed as a psychology lab, where human behavior can be observed and quantified. This work provides a preliminary analysis of the role of rationality in poker games, using a stylized version of Texas Hold'em. In particular, we compare the performance of two different kinds of players, i.e., rational vs irrational players, during a poker tournament. Results show that these behaviors (i.e., rationality and irrationality) affect both the outcomes of challenges and the way poker should be classified.Comment: 15 pages, 5 figure

Multiparty Dynamics and Failure Modes for Machine Learning and Artificial Intelligence

An important challenge for safety in machine learning and artificial intelligence systems is a~set of related failures involving specification gaming, reward hacking, fragility to distributional shifts, and Goodhart's or Campbell's law. This paper presents additional failure modes for interactions within multi-agent systems that are closely related. These multi-agent failure modes are more complex, more problematic, and less well understood than the single-agent case, and are also already occurring, largely unnoticed. After motivating the discussion with examples from poker-playing artificial intelligence (AI), the paper explains why these failure modes are in some senses unavoidable. Following this, the paper categorizes failure modes, provides definitions, and cites examples for each of the modes: accidental steering, coordination failures, adversarial misalignment, input spoofing and filtering, and goal co-option or direct hacking. The paper then discusses how extant literature on multi-agent AI fails to address these failure modes, and identifies work which may be useful for the mitigation of these failure modes.Comment: 12 Pages, This version re-submitted to Big Data and Cognitive Computing, Special Issue "Artificial Superintelligence: Coordination & Strategy

Traditional Wisdom and Monte Carlo Tree Search Face-to-Face in the Card Game Scopone

We present the design of a competitive artificial intelligence for Scopone, a popular Italian card game. We compare rule-based players using the most established strategies (one for beginners and two for advanced players) against players using Monte Carlo Tree Search (MCTS) and Information Set Monte Carlo Tree Search (ISMCTS) with different reward functions and simulation strategies. MCTS requires complete information about the game state and thus implements a cheating player while ISMCTS can deal with incomplete information and thus implements a fair player. Our results show that, as expected, the cheating MCTS outperforms all the other strategies; ISMCTS is stronger than all the rule-based players implementing well-known and most advanced strategies and it also turns out to be a challenging opponent for human players.Comment: Preprint. Accepted for publication in the IEEE Transaction on Game

Simulation of a Texas Hold'Em poker player

Copyright 2011 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. This is the accepted version of the article. The published version is available at