Asynchronous Decision-Making Dynamics Under Best-Response Update Rule in Finite Heterogeneous Populations

To study how sustainable cooperation might emerge among self-interested interacting individuals, we investigate the long-run behavior of the decision-making dynamics in a finite, well-mixed population of individuals, who play collectively over time a population game. Repeatedly each individual is activated asynchronously to update her decision to either cooperate or defect according to the myopic best-response rule. The game's payoff matrices, chosen to be those of either prisoner's dilemma or snowdrift games to underscore cooperation-centered social dilemmas, are fixed, but can be distinct for different individuals. So, the overall population is heterogeneous. We first classify such heterogeneous individuals into different types according to their cooperating tendencies stipulated by their payoff matrices. Then, we show that no matter what initial strategies the individuals decide to use, surprisingly one can always identify one type of individuals as a benchmark such that after a sufficiently long but finite time, individuals more cooperative compared to the benchmark always cooperate, while those less cooperative compared to the benchmark defect. When such fixation takes place, the total number of cooperators in the population either becomes fixed or fluctuates at most by one. Such insight provides theoretical explanation for some complex behavior recently reported in simulation studies that highlight the puzzling effect of individuals' heterogeneity on collective decision-making dynamics

The lower convergence tendency of imitators compared to best responders

Imitation is widely observed in nature and often used to model populations of decision-making agents, but it is not yet known under what conditions a network of imitators will reach a state where they are satisfied with their decisions. We show that every network in which agents imitate the best performing strategy in their neighborhood will reach an equilibrium in finite time, provided that all agents are opponent coordinating, i.e., earn a higher payoff if their opponent plays the same strategy as they do. It follows that any non-convergence observed in imitative networks is not necessarily a result of population heterogeneity nor special network topology, but rather must be caused by other factors such as the presence of non-opponent-coordinating agents. To strengthen this result, we show that large classes of imitative networks containing non-opponent-coordinating agents never equilibrate even when the population is homogeneous. Comparing to best-response dynamics where equilibration is guaranteed for every network of homogeneous agents playing 2 × 2 matrix games, our results imply that networks of imitators have a lower equilibration tendency

Incentive-Based Control of Asynchronous Best-Response Dynamics on Binary Decision Networks

Various populations of interacting decision-making agents can be modeled by asynchronous best-response dynamics, or equivalently, linear threshold dynamics. Building upon recent convergence results in the absence of control, we now consider how such a network can be efficiently driven to a desired equilibrium state by offering payoff incentives or rewards for using a particular strategy, either uniformly or targeted to individuals. We begin by showing that strategy changes are monotone following an increase in payoffs in coordination games, and that the resulting equilibrium is unique. Based on these results, for the case when a uniform incentive is offered to all agents, we show how to compute the optimal incentive using a binary search algorithm. When different incentives can be offered to each agent, we propose a new algorithm to select which agents should be targeted based on maximizing a ratio between the cascading effect of a strategy switch by each agent and the incentive required to cause the agent to switch. Simulations show that this algorithm computes near-optimal targeted incentives for a wide range of networks and payoff distributions in coordination games and can also be effective for anti-coordination games

Convergence Analysis of Heterogeneous Decision-making Populations Under the Coordinating Best-response and Imitation Update Rules

This thesis emphasis is on coordination games. In a coordination game, selecting the same strategy or decision as the opponent is mutually beneficial for both parties. We studied the problem of equilibrium convergence in such games in both discrete and continuous (time) cases. In the first Chapter, we provide a brief introduction to the field of game theory. We discuss different categories of agents based on their levels of rationality and decision-making strategies, along with a variety of games. Additionally, we address important issues and challenges within this field. The second Chapter of this work is dedicated to a heterogeneous mixed population of imitators and best-responders. In this model, agents’ update rules are assumed to be discrete functions of time. Imitators refer to agents who simply replicate the strategy of another agent with the highest payoff, while best-responders pick the strategies that maximise their individual outcomes. Suggesting the concept of ’sections’--a consecutive sequence of agents with similar strategies– helped us in establishing convergence to an equilibrium state. This convergence was demonstrated under any arbitrary asynchronous activation sequence within a linear network. The proof was then extended to networks with ring, starike, and sparse-tree structures. However, the question of equilibrium convergence for other network structures remains an open challenge. In the third Chapter, we examined a large well-mixed population of imitators within a coordination context. Our analysis is grounded in the assumption that imitation here is driven by dissatisfaction. Equivalently, agents with lower payoffs are more dissatisfied and have more tendency to change and imitate higher earners within the population. The analysis reveals the presence of three fixed points, of which two are stable and one is a saddle point. The stable manifold of the unstable fixed point is also calculated. Additionally, It is demonstrated that starting from any initial state, the population eventually converges towards one of these introduced fixed points

Convergence Analysis of Heterogeneous Decision-making Populations Under the Coordinating Best-response and Imitation Update Rules

This thesis emphasis is on coordination games. In a coordination game, selecting the same strategy or decision as the opponent is mutually beneficial for both parties. We studied the problem of equilibrium convergence in such games in both discrete and continuous (time) cases. In the first Chapter, we provide a brief introduction to the field of game theory. We discuss different categories of agents based on their levels of rationality and decision-making strategies, along with a variety of games. Additionally, we address important issues and challenges within this field. The second Chapter of this work is dedicated to a heterogeneous mixed population of imitators and best-responders. In this model, agents’ update rules are assumed to be discrete functions of time. Imitators refer to agents who simply replicate the strategy of another agent with the highest payoff, while best-responders pick the strategies that maximise their individual outcomes. Suggesting the concept of ’sections’--a consecutive sequence of agents with similar strategies– helped us in establishing convergence to an equilibrium state. This convergence was demonstrated under any arbitrary asynchronous activation sequence within a linear network. The proof was then extended to networks with ring, starike, and sparse-tree structures. However, the question of equilibrium convergence for other network structures remains an open challenge. In the third Chapter, we examined a large well-mixed population of imitators within a coordination context. Our analysis is grounded in the assumption that imitation here is driven by dissatisfaction. Equivalently, agents with lower payoffs are more dissatisfied and have more tendency to change and imitate higher earners within the population. The analysis reveals the presence of three fixed points, of which two are stable and one is a saddle point. The stable manifold of the unstable fixed point is also calculated. Additionally, It is demonstrated that starting from any initial state, the population eventually converges towards one of these introduced fixed points

A survey on the analysis and control of evolutionary matrix games

In support of the growing interest in how to efficiently influence complex systems of interacting self interested agents, we present this review of fundamental concepts, emerging research, and open problems related to the analysis and control of evolutionary matrix games, with particular emphasis on applications in social, economic, and biological networks. (C) 2018 Elsevier Ltd. All rights reserved

Evolutionary Matrix-Game Dynamics Under Imitation in Heterogeneous Populations

Decision-making individuals often imitate their highest-earning fellows rather than optimize their own utilities, due to bounded rationality and incomplete information. Perpetual fluctuations between decisions have been reported as the dominant asymptotic outcome of imitative behaviors, yet little attempt has been made to characterize them, particularly in heterogeneous populations. We study a finite well-mixed heterogeneous population of individuals choosing between the two strategies, cooperation, and defection, and earning based on their payoff matrices that can be unique to each individual. At each time step, an arbitrary individual becomes active to update her decision by imitating the highest earner in the population. We show that almost surely the dynamics reach either an equilibrium state or a minimal positively invariant set, a \emph{fluctuation set}, in the long run. In addition to finding all equilibria, for the first time, we characterize the fluctuation sets, provide necessary and sufficient conditions for their existence, and approximate their basins of attraction. We also find that exclusive populations of individuals playing coordination or prisoner's dilemma games always equilibrate, implying that cycles and non-convergence in imitative populations are due to individuals playing anticoordination games. Moreover, we show that except for the two extreme equilibria where all individuals play the same strategy, almost all other equilibria are unstable as long as the population is heterogeneous. Our results theoretically explain earlier reported simulation results and shed new light on the boundedly rational nature of imitation behaviors

Equilibration of Coordinating Imitation and Best-Response Dynamics

Decision-making individuals are often considered to be either imitators who copy the action of their most successful neighbors or best-responders who maximize their benefit against the current actions of their neighbors. In the context of coordination games, where neighboring individuals earn more if they take the same action, by means of potential functions, it was shown that populations of all imitators and populations of all best-responders equilibrate in finite time when they become active to update their decisions sequentially. However, for mixed populations of the two, the equilibration was shown only for specific activation sequences. It is therefore, unknown, whether a potential function also exists for mixed populations or if there actually exists a counter example where an activation sequence prevents equilibration. We show that in a linear graph, the number of ``sections'' (a sequence of consecutive individuals taking the same action) serves as a potential function, leading to equilibration, and that this result can be extended to sparse trees. The existence of a potential function for other types of networks remains an open problem

Evolutionary games on graphs

Game theory is one of the key paradigms behind many scientific disciplines from biology to behavioral sciences to economics. In its evolutionary form and especially when the interacting agents are linked in a specific social network the underlying solution concepts and methods are very similar to those applied in non-equilibrium statistical physics. This review gives a tutorial-type overview of the field for physicists. The first three sections introduce the necessary background in classical and evolutionary game theory from the basic definitions to the most important results. The fourth section surveys the topological complications implied by non-mean-field-type social network structures in general. The last three sections discuss in detail the dynamic behavior of three prominent classes of models: the Prisoner's Dilemma, the Rock-Scissors-Paper game, and Competing Associations. The major theme of the review is in what sense and how the graph structure of interactions can modify and enrich the picture of long term behavioral patterns emerging in evolutionary games.Comment: Review, final version, 133 pages, 65 figure