Stochastic fictitious play with continuous action sets

Continuous action space games are ubiquitous in economics. However, whilst learning dynamics in normal form games with finite action sets are now well studied, it is not until recently that their continuous action space counterparts have been examined. We extend stochastic fictitious play to the continuous action space framework. In normal form games with finite action sets the limiting behaviour of a discrete time learning process is often studied using its continuous time counterpart via stochastic approximation. In this paper we study stochastic fictitious play in games with continuous action spaces using the same method. This requires the asymptotic pseudo-trajectory approach to stochastic approximation to be extended to Banach spaces. In particular the limiting behaviour of stochastic fictitious play is studied using the associated smooth best response dynamics on the space of finite signed measures. Using this approach, stochastic fictitious play is shown to converge to an equilibrium point in two-player zero-sum games and a stochastic fictitious play-like process is shown to converge to an equilibrium in negative definite single population games

On the Global Convergence of Stochastic Fictitious Play in Stochastic Games with Turn-based Controllers

This paper presents a learning dynamic with almost sure convergence guarantee for any stochastic game with turn-based controllers (on state transitions) as long as stage-payoffs have stochastic fictitious-play-property. For example, two-player zero-sum and n-player potential strategic-form games have this property. Note also that stage-payoffs for different states can have different structures such as they can sum to zero in some states and be identical in others. The dynamics presented combines the classical stochastic fictitious play with value iteration for stochastic games. There are two key properties: (i) players play finite horizon stochastic games with increasing lengths within the underlying infinite-horizon stochastic game, and (ii) the turn-based controllers ensure that the auxiliary stage-games (induced from the continuation payoff estimated) have the stochastic fictitious-play-property

Learning in Perturbed Asymmetric Games

We investigate the stability of mixed strategy equilibria in 2 person (bimatrix) games under perturbed best response dynamics. A mixed equilibrium is asymptotically stable under all such dynamics if and only if the game is linearly equivalent to a zero sum game. In this case, the mixed equilibrium is also globally asymptotically stable. Global convergence to the set of perturbed equilibria is shown also for (rescaled) partnership games (also know as games of identical interest). Some applications of these result to stochastic learning models are given.Games, Learning, Best Response Dynamics, Stochastic Fictitious Play, Mixed Strategy Equilibria, Zero Sum Games

On the Nonconvergence of Fictitious Play in Coordination Games

It is natural to conjecture that fictitious play converges in coordination games, but this is shown by counterexample to be false. Variants of fictitious play in which past actions are eventually forgotten and there are small stochastic perturbations are much better behaved: over the long run players manage to coordinate with high probability

Fictitious Play with Time-Invariant Frequency Update for Network Security

We study two-player security games which can be viewed as sequences of nonzero-sum matrix games played by an Attacker and a Defender. The evolution of the game is based on a stochastic fictitious play process, where players do not have access to each other's payoff matrix. Each has to observe the other's actions up to present and plays the action generated based on the best response to these observations. In a regular fictitious play process, each player makes a maximum likelihood estimate of her opponent's mixed strategy, which results in a time-varying update based on the previous estimate and current action. In this paper, we explore an alternative scheme for frequency update, whose mean dynamic is instead time-invariant. We examine convergence properties of the mean dynamic of the fictitious play process with such an update scheme, and establish local stability of the equilibrium point when both players are restricted to two actions. We also propose an adaptive algorithm based on this time-invariant frequency update.Comment: Proceedings of the 2010 IEEE Multi-Conference on Systems and Control (MSC10), September 2010, Yokohama, Japa