Control of large distributed systems using games with pure strategy nash equilibria

Control mechanisms for optimisation in large distributed systems cannot be constructed based on traditional methods of control because they are typically characterised by distributed information and costly and/or noisy communication. Furthermore, noisy observations and dynamism are also inherent to these systems, so their control mechanisms need to be flexible, agile and robust in the face of these characteristics. In such settings, a good control mechanism should satisfy the following four design requirements: (i) it should produce high quality solutions, (ii) it should be robustness and flexibility in the face of additions, removals and failures of components, (iii) it should operate by making limited use of communication, and (iv) its operation should be computational feasible. Against this background, in order to satisfy these requirements, in this thesis we adopt a design approach based on dividing control over the system across a team of self–interested agents. Such multi–agent systems (MAS) are naturally distributed (matching the application domains in question), and by pursing their own private goals, the agents can collectively implement robust, flexible and scalable control mechanisms. In more detail, the design approach we adopt is (i) to use games with pure strategy Nash equilibria as a framework or template for constructing the agents’ utility functions, such that good solutions to the optimisation problem arise at the pure strategy Nash equilibria of the game, and (ii) to derive distributed techniques for solving the games for their Nash equilibria. The specific problems we tackle can be grouped into four main topics. First, we investigate a class of local algorithms for distributed constraint optimisation problems (DCOPs). We introduce a unifying analytical framework for studying such algorithms, and develop a parameterisation of the algorithm design space, which represents a mapping from the algorithms’ components to their performance according to each of our design requirements. Second, we develop a game–theoretic control mechanism for distributed dynamic task allocation and scheduling problems. The model in question is an expansion of DCOPs to encompass dynamic problems, and the control mechanism we derive builds on the insights from our first topic to address our four design requirements. Third, we elaborate a general class of problems including DCOPs with noisy rewards and state observations, which are realistic traits of great concern in real–world problems, and derive control mechanisms for these environments. These control mechanism allow the agents to either learn their reward functions or decide when to make observations of the world’s state and/or communicate their beliefs over the state of the world, in such a manner that they perform well according to our design requirements. Fourth, we derive an optimal algorithm for computing and optimising over pure strategy Nash equilibria in games with sparse interaction structure. By exploiting the structure present in many multi-agent interactions, this distributed algorithm can efficiently compute equilibria that optimise various criteria, thus reducing the computational burden on any one agent and operating using less communication than an equivalent centralised algorithms.For each of these topics, the control mechanisms that we derive are developed such that they perform well according to all four f our design requirements. In sum, by making the above contributions to these specific topics, we demonstrate that the general approach of using games with pure strategy Nash equilibria as a template for designing MAS produces good control mechanisms for large distributed systems

Game theory of mind

This paper introduces a model of ‘theory of mind’, namely, how we represent the intentions and goals of others to optimise our mutual interactions. We draw on ideas from optimum control and game theory to provide a ‘game theory of mind’. First, we consider the representations of goals in terms of value functions that are prescribed by utility or rewards. Critically, the joint value functions and ensuing behaviour are optimised recursively, under the assumption that I represent your value function, your representation of mine, your representation of my representation of yours, and so on ad infinitum. However, if we assume that the degree of recursion is bounded, then players need to estimate the opponent's degree of recursion (i.e., sophistication) to respond optimally. This induces a problem of inferring the opponent's sophistication, given behavioural exchanges. We show it is possible to deduce whether players make inferences about each other and quantify their sophistication on the basis of choices in sequential games. This rests on comparing generative models of choices with, and without, inference. Model comparison is demonstrated using simulated and real data from a ‘stag-hunt’. Finally, we note that exactly the same sophisticated behaviour can be achieved by optimising the utility function itself (through prosocial utility), producing unsophisticated but apparently altruistic agents. This may be relevant ethologically in hierarchal game theory and coevolution

Incentive Stackelberg Mean-payoff Games

We introduce and study incentive equilibria for multi-player meanpayoff games. Incentive equilibria generalise well-studied solution concepts such as Nash equilibria and leader equilibria (also known as Stackelberg equilibria). Recall that a strategy profile is a Nash equilibrium if no player can improve his payoff by changing his strategy unilaterally. In the setting of incentive and leader equilibria, there is a distinguished player called the leader who can assign strategies to all other players, referred to as her followers. A strategy profile is a leader strategy profile if no player, except for the leader, can improve his payoff by changing his strategy unilaterally, and a leader equilibrium is a leader strategy profile with a maximal return for the leader. In the proposed case of incentive equilibria, the leader can additionally influence the behaviour of her followers by transferring parts of her payoff to her followers. The ability to incentivise her followers provides the leader with more freedom in selecting strategy profiles, and we show that this can indeed improve the payoff for the leader in such games. The key fundamental result of the paper is the existence of incentive equilibria in mean-payoff games. We further show that the decision problem related to constructing incentive equilibria is NP-complete. On a positive note, we show that, when the number of players is fixed, the complexity of the problem falls in the same class as two-player mean-payoff games. We also present an implementation of the proposed algorithms, and discuss experimental results that demonstrate the feasibility of the analysis of medium sized games.Comment: 15 pages, references, appendix, 5 figure

Learning the Structure and Parameters of Large-Population Graphical Games from Behavioral Data

We consider learning, from strictly behavioral data, the structure and parameters of linear influence games (LIGs), a class of parametric graphical games introduced by Irfan and Ortiz (2014). LIGs facilitate causal strategic inference (CSI): Making inferences from causal interventions on stable behavior in strategic settings. Applications include the identification of the most influential individuals in large (social) networks. Such tasks can also support policy-making analysis. Motivated by the computational work on LIGs, we cast the learning problem as maximum-likelihood estimation (MLE) of a generative model defined by pure-strategy Nash equilibria (PSNE). Our simple formulation uncovers the fundamental interplay between goodness-of-fit and model complexity: good models capture equilibrium behavior within the data while controlling the true number of equilibria, including those unobserved. We provide a generalization bound establishing the sample complexity for MLE in our framework. We propose several algorithms including convex loss minimization (CLM) and sigmoidal approximations. We prove that the number of exact PSNE in LIGs is small, with high probability; thus, CLM is sound. We illustrate our approach on synthetic data and real-world U.S. congressional voting records. We briefly discuss our learning framework's generality and potential applicability to general graphical games.Comment: Journal of Machine Learning Research. (accepted, pending publication.) Last conference version: submitted March 30, 2012 to UAI 2012. First conference version: entitled, Learning Influence Games, initially submitted on June 1, 2010 to NIPS 201

The influence of topology and information diffusion on networked game dynamics

This thesis studies the influence of topology and information diffusion on the strategic interactions of agents in a population. It shows that there exists a reciprocal relationship between the topology, information diffusion and the strategic interactions of a population of players. In order to evaluate the influence of topology and information flow on networked game dynamics, strategic games are simulated on populations of players where the players are distributed in a non-homogeneous spatial arrangement. The initial component of this research consists of a study of evolution of the coordination of strategic players, where the topology or the structure of the population is shown to be critical in defining the coordination among the players. Next, the effect of network topology on the evolutionary stability of strategies is studied in detail. Based on the results obtained, it is shown that network topology plays a key role in determining the evolutionary stability of a particular strategy in a population of players. Then, the effect of network topology on the optimum placement of strategies is studied. Using genetic optimisation, it is shown that the placement of strategies in a spatially distributed population of players is crucial in maximising the collective payoff of the population. Exploring further the effect of network topology and information diffusion on networked games, the non-optimal or bounded rationality of players is modelled using topological and directed information flow of the network. Based on the topologically distributed bounded rationality model, it is shown that the scale-free and small-world networks emerge in randomly connected populations of sub-optimal players. Thus, the topological and information theoretic interpretations of bounded rationality suggest the topology, information diffusion and the strategic interactions of socio-economical structures are cyclically interdependent

The influence of topology and information diffusion on networked game dynamics

This thesis studies the influence of topology and information diffusion on the strategic interactions of agents in a population. It shows that there exists a reciprocal relationship between the topology, information diffusion and the strategic interactions of a population of players. In order to evaluate the influence of topology and information flow on networked game dynamics, strategic games are simulated on populations of players where the players are distributed in a non-homogeneous spatial arrangement. The initial component of this research consists of a study of evolution of the coordination of strategic players, where the topology or the structure of the population is shown to be critical in defining the coordination among the players. Next, the effect of network topology on the evolutionary stability of strategies is studied in detail. Based on the results obtained, it is shown that network topology plays a key role in determining the evolutionary stability of a particular strategy in a population of players. Then, the effect of network topology on the optimum placement of strategies is studied. Using genetic optimisation, it is shown that the placement of strategies in a spatially distributed population of players is crucial in maximising the collective payoff of the population. Exploring further the effect of network topology and information diffusion on networked games, the non-optimal or bounded rationality of players is modelled using topological and directed information flow of the network. Based on the topologically distributed bounded rationality model, it is shown that the scale-free and small-world networks emerge in randomly connected populations of sub-optimal players. Thus, the topological and information theoretic interpretations of bounded rationality suggest the topology, information diffusion and the strategic interactions of socio-economical structures are cyclically interdependent

Equilibrium selection in stag hunt games

Game Theory