139 research outputs found
Computational Results for Extensive-Form Adversarial Team Games
We provide, to the best of our knowledge, the first computational study of
extensive-form adversarial team games. These games are sequential, zero-sum
games in which a team of players, sharing the same utility function, faces an
adversary. We define three different scenarios according to the communication
capabilities of the team. In the first, the teammates can communicate and
correlate their actions both before and during the play. In the second, they
can only communicate before the play. In the third, no communication is
possible at all. We define the most suitable solution concepts, and we study
the inefficiency caused by partial or null communication, showing that the
inefficiency can be arbitrarily large in the size of the game tree.
Furthermore, we study the computational complexity of the equilibrium-finding
problem in the three scenarios mentioned above, and we provide, for each of the
three scenarios, an exact algorithm. Finally, we empirically evaluate the
scalability of the algorithms in random games and the inefficiency caused by
partial or null communication
A Generic Multi-Player Transformation Algorithm for Solving Large-Scale Zero-Sum Extensive-Form Adversarial Team Games
Many recent practical and theoretical breakthroughs focus on adversarial team
multi-player games (ATMGs) in ex ante correlation scenarios. In this setting,
team members are allowed to coordinate their strategies only before the game
starts. Although there existing algorithms for solving extensive-form ATMGs,
the size of the game tree generated by the previous algorithms grows
exponentially with the number of players. Therefore, how to deal with
large-scale zero-sum extensive-form ATMGs problems close to the real world is
still a significant challenge. In this paper, we propose a generic multi-player
transformation algorithm, which can transform any multi-player game tree
satisfying the definition of AMTGs into a 2-player game tree, such that finding
a team-maxmin equilibrium with correlation (TMECor) in large-scale ATMGs can be
transformed into solving NE in 2-player games. To achieve this goal, we first
introduce a new structure named private information pre-branch, which consists
of a temporary chance node and coordinator nodes and aims to make decisions for
all potential private information on behalf of the team members. We also show
theoretically that NE in the transformed 2-player game is equivalent TMECor in
the original multi-player game. This work significantly reduces the growth of
action space and nodes from exponential to constant level. This enables our
work to outperform all the previous state-of-the-art algorithms in finding a
TMECor, with 182.89, 168.47, 694.44, and 233.98 significant improvements in the
different Kuhn Poker and Leduc Poker cases (21K3, 21K4, 21K6 and 21L33). In
addition, this work first practically solves the ATMGs in a 5-player case which
cannot be conducted by existing algorithms.Comment: 9 pages, 5 figures, NIPS 202
A marriage between adversarial team games and 2-player games: enabling abstractions, no-regret learning, and subgame solving
Ex ante correlation is becoming the mainstream approach for sequential adversarial team games,where a team of players faces another team in a
zero-sum game. It is known that team members’asymmetric information makes both equilibrium computation APX-hard and team’s strategies not
directly representable on the game tree. This latter issue prevents the adoption of successful tools for huge 2-player zero-sum games such as,
e.g., abstractions, no-regret learning, and sub game solving. This work shows that we can re cover from this weakness by bridging the gap be tween sequential adversarial team games and 2-player games. In particular, we propose a new,suitable game representation that we call team public-information, in which a team is repre sented as a single coordinator who only knows information common to the whole team and pre scribes to each member an action for any pos sible private state. The resulting representation is highly explainable, being a 2-player tree in
which the team’s strategies are behavioral with a direct interpretation and more expressive than he original extensive form when designing ab stractions. Furthermore, we prove payoff equiva lence of our representation, and we provide tech niques that, starting directly from the extensive form, generate dramatically more compact repre sentations without information loss. Finally, we experimentally evaluate our techniques when ap plied to a standard testbed, comparing their per formance with the current state of the art
Computing Ex Ante Coordinated Team-Maxmin Equilibria in Zero-Sum Multiplayer Extensive-Form Games
Computational game theory has many applications in the modern world in both
adversarial situations and the optimization of social good. While there exist
many algorithms for computing solutions in two-player interactions, finding
optimal strategies in multiplayer interactions efficiently remains an open
challenge. This paper focuses on computing the multiplayer Team-Maxmin
Equilibrium with Coordination device (TMECor) in zero-sum extensive-form games.
TMECor models scenarios when a team of players coordinates ex ante against an
adversary. Such situations can be found in card games (e.g., in Bridge and
Poker), when a team works together to beat a target player but communication is
prohibited; and also in real world, e.g., in forest-protection operations, when
coordinated groups have limited contact during interdicting illegal loggers.
The existing algorithms struggle to find a TMECor efficiently because of their
high computational costs. To compute a TMECor in larger games, we make the
following key contributions: (1) we propose a hybrid-form strategy
representation for the team, which preserves the set of equilibria; (2) we
introduce a column-generation algorithm with a guaranteed finite-time
convergence in the infinite strategy space based on a novel best-response
oracle; (3) we develop an associated-representation technique for the exact
representation of the multilinear terms in the best-response oracle; and (4) we
experimentally show that our algorithm is several orders of magnitude faster
than prior state-of-the-art algorithms in large games.Comment: AAAI 2021. This paper also is a part of the thesis: Youzhi Zhang,
February 2020. Computing Team-Maxmin Equilibria in Zero-Sum Multiplayer
Games. PhD Thesis,
https://personal.ntu.edu.sg/boan/thesis/Zhang_Youzhi_PhD_Thesis.pd
Public Information Representation for Adversarial Team Games
The peculiarity of adversarial team games resides in the asymmetric
information available to the team members during the play, which makes the
equilibrium computation problem hard even with zero-sum payoffs. The algorithms
available in the literature work with implicit representations of the strategy
space and mainly resort to Linear Programming and column generation techniques
to enlarge incrementally the strategy space. Such representations prevent the
adoption of standard tools such as abstraction generation, game solving, and
subgame solving, which demonstrated to be crucial when solving huge, real-world
two-player zero-sum games. Differently from these works, we answer the question
of whether there is any suitable game representation enabling the adoption of
those tools. In particular, our algorithms convert a sequential team game with
adversaries to a classical two-player zero-sum game. In this converted game,
the team is transformed into a single coordinator player who only knows
information common to the whole team and prescribes to the players an action
for any possible private state. Interestingly, we show that our game is more
expressive than the original extensive-form game as any state/action
abstraction of the extensive-form game can be captured by our representation,
while the reverse does not hold. Due to the NP-hard nature of the problem, the
resulting Public Team game may be exponentially larger than the original one.
To limit this explosion, we provide three algorithms, each returning an
information-lossless abstraction that dramatically reduces the size of the
tree. These abstractions can be produced without generating the original game
tree. Finally, we show the effectiveness of the proposed approach by presenting
experimental results on Kuhn and Leduc Poker games, obtained by applying
state-of-art algorithms for two-player zero-sum games on the converted gamesComment: 19 pages, 7 figures, Best Paper Award in Cooperative AI Workshop at
NeurIPS 202
Efficiently Computing Nash Equilibria in Adversarial Team Markov Games
Computing Nash equilibrium policies is a central problem in multi-agent
reinforcement learning that has received extensive attention both in theory and
in practice. However, provable guarantees have been thus far either limited to
fully competitive or cooperative scenarios or impose strong assumptions that
are difficult to meet in most practical applications. In this work, we depart
from those prior results by investigating infinite-horizon \emph{adversarial
team Markov games}, a natural and well-motivated class of games in which a team
of identically-interested players -- in the absence of any explicit
coordination or communication -- is competing against an adversarial player.
This setting allows for a unifying treatment of zero-sum Markov games and
Markov potential games, and serves as a step to model more realistic strategic
interactions that feature both competing and cooperative interests. Our main
contribution is the first algorithm for computing stationary
-approximate Nash equilibria in adversarial team Markov games with
computational complexity that is polynomial in all the natural parameters of
the game, as well as . The proposed algorithm is particularly
natural and practical, and it is based on performing independent policy
gradient steps for each player in the team, in tandem with best responses from
the side of the adversary; in turn, the policy for the adversary is then
obtained by solving a carefully constructed linear program. Our analysis
leverages non-standard techniques to establish the KKT optimality conditions
for a nonlinear program with nonconvex constraints, thereby leading to a
natural interpretation of the induced Lagrange multipliers. Along the way, we
significantly extend an important characterization of optimal policies in
adversarial (normal-form) team games due to Von Stengel and Koller (GEB `97)
Elementary game theory
The theory of games (or game theory) is a mathematical theory that deals with the general features of competitive situations. It involves strategic thinking, and studies the way people interact while making economic policies, contesting elections and other such decisions. There are various types of game models, which are based on factors, like the number of players participating, the sum of gains or losses and the number of strategies available. According to strategic reasoning, we can say that the phenomenon where each player responds best to the other is Nash Equilibrium. It is a solution concept of a non-cooperative game comprising of two or more players, in which each player is assumed to know the equilibrium strategies of the other players, and no player has anything to gain by only changing their own strategy. Nash equilibrium is best for both players, if all players abide by it. The normal form game (strategic form) does not incorporate any notion of sequence or time of the action of the players. In a normal form game, both players choose their strategy together without knowing the strategies of other players in the game. While the extensive form game is a game, which makes the temporal structure explicit i.e. it allows us to think more naturally about factors such as time. In an extensive game with perfect information there are no simultaneous moves and every player at any point of time is made aware of all the previous choices of all other players.In coalitional games, our focus is on what group of agents, rather than individual agents can achieve
- …