18 research outputs found
No-Regret Learning in Extensive-Form Games with Imperfect Recall
Counterfactual Regret Minimization (CFR) is an efficient no-regret learning
algorithm for decision problems modeled as extensive games. CFR's regret bounds
depend on the requirement of perfect recall: players always remember
information that was revealed to them and the order in which it was revealed.
In games without perfect recall, however, CFR's guarantees do not apply. In
this paper, we present the first regret bound for CFR when applied to a general
class of games with imperfect recall. In addition, we show that CFR applied to
any abstraction belonging to our general class results in a regret bound not
just for the abstract game, but for the full game as well. We verify our theory
and show how imperfect recall can be used to trade a small increase in regret
for a significant reduction in memory in three domains: die-roll poker, phantom
tic-tac-toe, and Bluff.Comment: 21 pages, 4 figures, expanded version of article to appear in
Proceedings of the Twenty-Ninth International Conference on Machine Learnin
Computing large market equilibria using abstractions
Computing market equilibria is an important practical problem for market
design (e.g. fair division, item allocation). However, computing equilibria
requires large amounts of information (e.g. all valuations for all buyers for
all items) and compute power. We consider ameliorating these issues by applying
a method used for solving complex games: constructing a coarsened abstraction
of a given market, solving for the equilibrium in the abstraction, and lifting
the prices and allocations back to the original market. We show how to bound
important quantities such as regret, envy, Nash social welfare, Pareto
optimality, and maximin share when the abstracted prices and allocations are
used in place of the real equilibrium. We then study two abstraction methods of
interest for practitioners: 1) filling in unknown valuations using techniques
from matrix completion, 2) reducing the problem size by aggregating groups of
buyers/items into smaller numbers of representative buyers/items and solving
for equilibrium in this coarsened market. We find that in real data
allocations/prices that are relatively close to equilibria can be computed from
even very coarse abstractions
Imperfect-Recall Abstractions with Bounds in Games
Imperfect-recall abstraction has emerged as the leading paradigm for
practical large-scale equilibrium computation in incomplete-information games.
However, imperfect-recall abstractions are poorly understood, and only weak
algorithm-specific guarantees on solution quality are known. In this paper, we
show the first general, algorithm-agnostic, solution quality guarantees for
Nash equilibria and approximate self-trembling equilibria computed in
imperfect-recall abstractions, when implemented in the original
(perfect-recall) game. Our results are for a class of games that generalizes
the only previously known class of imperfect-recall abstractions where any
results had been obtained. Further, our analysis is tighter in two ways, each
of which can lead to an exponential reduction in the solution quality error
bound.
We then show that for extensive-form games that satisfy certain properties,
the problem of computing a bound-minimizing abstraction for a single level of
the game reduces to a clustering problem, where the increase in our bound is
the distance function. This reduction leads to the first imperfect-recall
abstraction algorithm with solution quality bounds. We proceed to show a divide
in the class of abstraction problems. If payoffs are at the same scale at all
information sets considered for abstraction, the input forms a metric space.
Conversely, if this condition is not satisfied, we show that the input does not
form a metric space. Finally, we use these results to experimentally
investigate the quality of our bound for single-level abstraction
Solving Large Extensive-Form Games with Strategy Constraints
Extensive-form games are a common model for multiagent interactions with
imperfect information. In two-player zero-sum games, the typical solution
concept is a Nash equilibrium over the unconstrained strategy set for each
player. In many situations, however, we would like to constrain the set of
possible strategies. For example, constraints are a natural way to model
limited resources, risk mitigation, safety, consistency with past observations
of behavior, or other secondary objectives for an agent. In small games,
optimal strategies under linear constraints can be found by solving a linear
program; however, state-of-the-art algorithms for solving large games cannot
handle general constraints. In this work we introduce a generalized form of
Counterfactual Regret Minimization that provably finds optimal strategies under
any feasible set of convex constraints. We demonstrate the effectiveness of our
algorithm for finding strategies that mitigate risk in security games, and for
opponent modeling in poker games when given only partial observations of
private information.Comment: Appeared in AAAI 201
A Bridge between Polynomial Optimization and Games with Imperfect Recall
We provide several positive and negative complexity results for solving games
with imperfect recall. Using a one-to-one correspondence between these games on
one side and multivariate polynomials on the other side, we show that solving
games with imperfect recall is as hard as solving certain problems of the first
order theory of reals. We establish square root sum hardness even for the
specific class of A-loss games. On the positive side, we find restrictions on
games and strategies motivated by Bridge bidding that give polynomial-time
complexity
On Imperfect Recall in Multi-Agent Influence Diagrams
Multi-agent influence diagrams (MAIDs) are a popular game-theoretic model
based on Bayesian networks. In some settings, MAIDs offer significant
advantages over extensive-form game representations. Previous work on MAIDs has
assumed that agents employ behavioural policies, which set independent
conditional probability distributions over actions for each of their decisions.
In settings with imperfect recall, however, a Nash equilibrium in behavioural
policies may not exist. We overcome this by showing how to solve MAIDs with
forgetful and absent-minded agents using mixed policies and two types of
correlated equilibrium. We also analyse the computational complexity of key
decision problems in MAIDs, and explore tractable cases. Finally, we describe
applications of MAIDs to Markov games and team situations, where imperfect
recall is often unavoidable.Comment: In Proceedings TARK 2023, arXiv:2307.0400