Analysis and Optimization of Deep Counterfactual Value Networks

Recently a strong poker-playing algorithm called DeepStack was published, which is able to find an approximate Nash equilibrium during gameplay by using heuristic values of future states predicted by deep neural networks. This paper analyzes new ways of encoding the inputs and outputs of DeepStack's deep counterfactual value networks based on traditional abstraction techniques, as well as an unabstracted encoding, which was able to increase the network's accuracy.Comment: Long version of publication appearing at KI 2018: The 41st German Conference on Artificial Intelligence (http://dx.doi.org/10.1007/978-3-030-00111-7_26). Corrected typo in titl

Toward Legalization of Poker: The Skill vs. Chance Debate

This paper sheds light on the age-old argument as to whether poker is a game in which skill predominates over chance or vice versa. Recent work addressing the issue of skill vs. chance is reviewed. This current study considers two different scenarios to address the issue: 1) a mathematical analysis supported by computer simulations of one random player and one skilled player in Texas Hold\u27Em, and 2) full-table simulation games of Texas Hold\u27Em and Seven Card Stud. Findings for scenario 1 showed the skilled player winning 97 percent of the hands. Findings for scenario 2 further reinforced that highly skilled players convincingly beat unskilled players. Following this study that shows poker as predominantly a skill game, various gaming jurisdictions might declare poker as such, thus legalizing and broadening the game for new venues, new markets, new demographics, and new media. Internet gaming in particular could be expanded and released from its current illegality in the U.S. with benefits accruing to casinos who wish to offer online poker

Solving Games with Functional Regret Estimation

We propose a novel online learning method for minimizing regret in large extensive-form games. The approach learns a function approximator online to estimate the regret for choosing a particular action. A no-regret algorithm uses these estimates in place of the true regrets to define a sequence of policies. We prove the approach sound by providing a bound relating the quality of the function approximation and regret of the algorithm. A corollary being that the method is guaranteed to converge to a Nash equilibrium in self-play so long as the regrets are ultimately realizable by the function approximator. Our technique can be understood as a principled generalization of existing work on abstraction in large games; in our work, both the abstraction as well as the equilibrium are learned during self-play. We demonstrate empirically the method achieves higher quality strategies than state-of-the-art abstraction techniques given the same resources.Comment: AAAI Conference on Artificial Intelligence 201

Measuring Skill in More-Person Games with Applications to Poker

In several jurisdictions, commercially exploiting a game of chance (rather than skill) is subject to a licensing regime. It is obvious that roulette is a game of chance and chess a game of skill, but the law does not provide a precise description of where the boundary between the two classes is drawn. We build upon the framework of Borm and Van der Genugten (2001) and Dreef et al. (2004) and propose a modification of the skill concept for more-person games. We apply our new skill measure to a simplified version of poker called Straight Poker and conclude that this game should be classified as a game of skill.games of chance;games of skill;poker

Approximating n-player behavioural strategy nash equilibria using coevolution

Coevolutionary algorithms are plagued with a set of problems related to intransitivity that make it questionable what the end product of a coevolutionary run can achieve. With the introduction of solution concepts into coevolution, part of the issue was alleviated, however efficiently representing and achieving game theoretic solution concepts is still not a trivial task. In this paper we propose a coevolutionary algorithm that approximates behavioural strategy Nash equilibria in n-player zero sum games, by exploiting the minimax solution concept. In order to support our case we provide a set of experiments in both games of known and unknown equilibria. In the case of known equilibria, we can confirm our algorithm converges to the known solution, while in the case of unknown equilibria we can see a steady progress towards Nash. Copyright 2011 ACM

Imperfect-Recall Abstractions with Bounds in Games

Imperfect-recall abstraction has emerged as the leading paradigm for practical large-scale equilibrium computation in incomplete-information games. However, imperfect-recall abstractions are poorly understood, and only weak algorithm-specific guarantees on solution quality are known. In this paper, we show the first general, algorithm-agnostic, solution quality guarantees for Nash equilibria and approximate self-trembling equilibria computed in imperfect-recall abstractions, when implemented in the original (perfect-recall) game. Our results are for a class of games that generalizes the only previously known class of imperfect-recall abstractions where any results had been obtained. Further, our analysis is tighter in two ways, each of which can lead to an exponential reduction in the solution quality error bound. We then show that for extensive-form games that satisfy certain properties, the problem of computing a bound-minimizing abstraction for a single level of the game reduces to a clustering problem, where the increase in our bound is the distance function. This reduction leads to the first imperfect-recall abstraction algorithm with solution quality bounds. We proceed to show a divide in the class of abstraction problems. If payoffs are at the same scale at all information sets considered for abstraction, the input forms a metric space. Conversely, if this condition is not satisfied, we show that the input does not form a metric space. Finally, we use these results to experimentally investigate the quality of our bound for single-level abstraction