Convergence to Equilibrium of Logit Dynamics for Strategic Games

We present the first general bounds on the mixing time of the Markov chain associated to the logit dynamics for wide classes of strategic games. The logit dynamics with inverse noise beta describes the behavior of a complex system whose individual components act selfishly and keep responding according to some partial ("noisy") knowledge of the system, where the capacity of the agent to know the system and compute her best move is measured by the inverse of the parameter beta. In particular, we prove nearly tight bounds for potential games and games with dominant strategies. Our results show that, for potential games, the mixing time is upper and lower bounded by an exponential in the inverse of the noise and in the maximum potential difference. Instead, for games with dominant strategies, the mixing time cannot grow arbitrarily with the inverse of the noise. Finally, we refine our analysis for a subclass of potential games called graphical coordination games, a class of games that have been previously studied in Physics and, more recently, in Computer Science in the context of diffusion of new technologies. We give evidence that the mixing time of the logit dynamics for these games strongly depends on the structure of the underlying graph. We prove that the mixing time of the logit dynamics for these games can be upper bounded by a function that is exponential in the cutwidth of the underlying graph and in the inverse of noise. Moreover, we consider two specific and popular network topologies, the clique and the ring. For games played on a clique we prove an almost matching lower bound on the mixing time of the logit dynamics that is exponential in the inverse of the noise and in the maximum potential difference, while for games played on a ring we prove that the time of convergence of the logit dynamics to its stationary distribution is significantly shorter

Convergence to Equilibrium of Logit Dynamics for Strategic Games

We present the first general bounds on the mixing time of the Markov chain associated to the logit dynamics for wide classes of strategic games. The logit dynamics with inverse noise β describes the behavior of a complex system whose individual components act selfishly according to some partial (“noisy”) knowledge of the system, where the capacity of the agent to know the system and compute her best move is measured by parameter β. In particular, we prove nearly tight bounds for potential games and games with dominant strategies. Our results show that for potential games the mixing time is bounded by an exponential in β and in the maximum potential difference. Instead, for games with dominant strategies the mixing time cannot grow arbitrarily with β. Finally, we refine our analysis for a subclass of potential games called graphical coordination games, often used for modeling the diffusion of new technologies. We prove that the mixing time of the logit dynamics for these games can be upper bounded by a function that is exponential in the cutwidth of the underlying graph and in β. Moreover, we consider two specific and popular network topologies, the clique and the ring. For the clique, we prove an almost matching lower bound on the mixing time of the logit dynamics that is exponential in β and in the maximum potential difference, while for the ring we prove that the time of convergence of the logit dynamics to its stationary distribution is significantly shorter

Dynamics in Near-Potential Games

Except for special classes of games, there is no systematic framework for analyzing the dynamical properties of multi-agent strategic interactions. Potential games are one such special but restrictive class of games that allow for tractable dynamic analysis. Intuitively, games that are "close" to a potential game should share similar properties. In this paper, we formalize and develop this idea by quantifying to what extent the dynamic features of potential games extend to "near-potential" games. We study convergence of three commonly studied classes of adaptive dynamics: discrete-time better/best response, logit response, and discrete-time fictitious play dynamics. For better/best response dynamics, we focus on the evolution of the sequence of pure strategy profiles and show that this sequence converges to a (pure) approximate equilibrium set, whose size is a function of the "distance" from a close potential game. We then study logit response dynamics and provide a characterization of the stationary distribution of this update rule in terms of the distance of the game from a close potential game and the corresponding potential function. We further show that the stochastically stable strategy profiles are pure approximate equilibria. Finally, we turn attention to fictitious play, and establish that the sequence of empirical frequencies of player actions converges to a neighborhood of (mixed) equilibria of the game, where the size of the neighborhood increases with distance of the game to a potential game. Thus, our results suggest that games that are close to a potential game inherit the dynamical properties of potential games. Since a close potential game to a given game can be found by solving a convex optimization problem, our approach also provides a systematic framework for studying convergence behavior of adaptive learning dynamics in arbitrary finite strategic form games.Comment: 42 pages, 8 figure

Metastability of Asymptotically Well-Behaved Potential Games

One of the main criticisms to game theory concerns the assumption of full rationality. Logit dynamics is a decentralized algorithm in which a level of irrationality (a.k.a. "noise") is introduced in players' behavior. In this context, the solution concept of interest becomes the logit equilibrium, as opposed to Nash equilibria. Logit equilibria are distributions over strategy profiles that possess several nice properties, including existence and uniqueness. However, there are games in which their computation may take time exponential in the number of players. We therefore look at an approximate version of logit equilibria, called metastable distributions, introduced by Auletta et al. [SODA 2012]. These are distributions that remain stable (i.e., players do not go too far from it) for a super-polynomial number of steps (rather than forever, as for logit equilibria). The hope is that these distributions exist and can be reached quickly by logit dynamics. We identify a class of potential games, called asymptotically well-behaved, for which the behavior of the logit dynamics is not chaotic as the number of players increases so to guarantee meaningful asymptotic results. We prove that any such game admits distributions which are metastable no matter the level of noise present in the system, and the starting profile of the dynamics. These distributions can be quickly reached if the rationality level is not too big when compared to the inverse of the maximum difference in potential. Our proofs build on results which may be of independent interest, including some spectral characterizations of the transition matrix defined by logit dynamics for generic games and the relationship of several convergence measures for Markov chains

Decentralized Dynamics for Finite Opinion Games

Game theory studies situations in which strategic players can modify the state of a given system, in the absence of a central authority. Solution concepts, such as Nash equilibrium, have been defined in order to predict the outcome of such situations. In multi-player settings, it has been pointed out that to be realistic, a solution concept should be obtainable via processes that are decentralized and reasonably simple. Accordingly we look at the computation of solution concepts by means of decentralized dynamics. These are algorithms in which players move in turns to decrease their own cost and the hope is that the system reaches an “equilibrium” quickly. We study these dynamics for the class of opinion games, recently introduced by Bindel et al. [FOCS, 2011]. These are games, important in economics and sociology, that model the formation of an opinion in a social network. We study best-response dynamics and show upper and lower bounds on the convergence to Nash equilibria. We also study a noisy version of best-response dynamics, called logit dynamics, and prove a host of results about its convergence rate as the noise in the system varies. To get these results, we use a variety of techniques developed to bound the mixing time of Markov chains, including coupling, spectral characterizations and bottleneck ratio

Metastability of Logit Dynamics for Coordination Games

Logit Dynamics [Blume, Games and Economic Behavior, 1993] are randomized best response dynamics for strategic games: at every time step a player is selected uniformly at random and she chooses a new strategy according to a probability distribution biased toward strategies promising higher payoffs. This process defines an ergodic Markov chain, over the set of strategy profiles of the game, whose unique stationary distribution is the long-term equilibrium concept for the game. However, when the mixing time of the chain is large (e.g., exponential in the number of players), the stationary distribution loses its appeal as equilibrium concept, and the transient phase of the Markov chain becomes important. It can happen that the chain is "metastable", i.e., on a time-scale shorter than the mixing time, it stays close to some probability distribution over the state space, while in a time-scale multiple of the mixing time it jumps from one distribution to another. In this paper we give a quantitative definition of "metastable probability distributions" for a Markov chain and we study the metastability of the logit dynamics for some classes of coordination games. We first consider a pure $n$-player coordination game that highlights the distinctive features of our metastability notion based on distributions. Then, we study coordination games on the clique without a risk-dominant strategy (which are equivalent to the well-known Glauber dynamics for the Curie-Weiss model) and coordination games on a ring (both with and without risk-dominant strategy)

Stability of Mixed-Strategy-Based Iterative Logit Quantal Response Dynamics in Game Theory

Using the Logit quantal response form as the response function in each step, the original definition of static quantal response equilibrium (QRE) is extended into an iterative evolution process. QREs remain as the fixed points of the dynamic process. However, depending on whether such fixed points are the long-term solutions of the dynamic process, they can be classified into stable (SQREs) and unstable (USQREs) equilibriums. This extension resembles the extension from static Nash equilibriums (NEs) to evolutionary stable solutions in the framework of evolutionary game theory. The relation between SQREs and other solution concepts of games, including NEs and QREs, is discussed. Using experimental data from other published papers, we perform a preliminary comparison between SQREs, NEs, QREs and the observed behavioral outcomes of those experiments. For certain games, we determine that SQREs have better predictive power than QREs and NEs

Incentives-Based Mechanism for Efficient Demand Response Programs

In this work we investigate the inefficiency of the electricity system with strategic agents. Specifically, we prove that without a proper control the total demand of an inefficient system is at most twice the total demand of the optimal outcome. We propose an incentives scheme that promotes optimal outcomes in the inefficient electricity market. The economic incentives can be seen as an indirect revelation mechanism that allocates resources using a one-dimensional message space per resource to be allocated. The mechanism does not request private information from users and is valid for any concave customer's valuation function. We propose a distributed implementation of the mechanism using population games and evaluate the performance of four popular dynamics methods in terms of the cost to implement the mechanism. We find that the achievement of efficiency in strategic environments might be achieved at a cost, which is dependent on both the users' preferences and the dynamic evolution of the system. Some simulation results illustrate the ideas presented throughout the paper.Comment: 38 pages, 9 figures, submitted to journa

Multiple Steady States, Limit Cycles and Chaotic Attractors in Evolutionary Games with Logit Dynamics

This paper investigates, by means of simple, three and four strategy games, the occurrence of periodic and chaotic behaviour in a smooth version of the Best Response Dynamics, the Logit Dynamics. The main finding is that, unlike Replicator Dynamics, generic Hopf bifurcation and thus, stable limit cycles, do occur under the Logit Dynamics, even for three strategy games. Moreover, we show that the Logit Dynamics displays another bifurcation which cannot to occur under the Replicator Dynamics: the fold catastrophe. Finally, we find, in a four strategy game, a period-doubling route to chaotic dynamics under a 'weighted' version of the Logit Dynamics.