Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence

Learning agents that are not only capable of taking tests, but also innovating is becoming a hot topic in AI. One of the most promising paths towards this vision is multi-agent learning, where agents act as the environment for each other, and improving each agent means proposing new problems for others. However, existing evaluation platforms are either not compatible with multi-agent settings, or limited to a specific game. That is, there is not yet a general evaluation platform for research on multi-agent intelligence. To this end, we introduce Arena, a general evaluation platform for multi-agent intelligence with 35 games of diverse logics and representations. Furthermore, multi-agent intelligence is still at the stage where many problems remain unexplored. Therefore, we provide a building toolkit for researchers to easily invent and build novel multi-agent problems from the provided game set based on a GUI-configurable social tree and five basic multi-agent reward schemes. Finally, we provide Python implementations of five state-of-the-art deep multi-agent reinforcement learning baselines. Along with the baseline implementations, we release a set of 100 best agents/teams that we can train with different training schemes for each game, as the base for evaluating agents with population performance. As such, the research community can perform comparisons under a stable and uniform standard. All the implementations and accompanied tutorials have been open-sourced for the community at https://sites.google.com/view/arena-unity/

Austrian Economics and Game Theory a Preliminary Methodological Stocktaking

This paper discusses, from the perspective of Austrian economics, the merits and drawbacks of game theory in economics. It begins by arguing that Austrians have neglected game theory at their peril, and then argues that game theoretic reasoning may be one way of modelling key Austrian insights, although some aspects of game theory doesn’t square easily with Austrian economics. However, a major stumbling block for an Austrian acceptance of game theory may lie in the traditional Austrian resistance to formal methods.Austrian economics, game theory

Game theory of mind

This paper introduces a model of ‘theory of mind’, namely, how we represent the intentions and goals of others to optimise our mutual interactions. We draw on ideas from optimum control and game theory to provide a ‘game theory of mind’. First, we consider the representations of goals in terms of value functions that are prescribed by utility or rewards. Critically, the joint value functions and ensuing behaviour are optimised recursively, under the assumption that I represent your value function, your representation of mine, your representation of my representation of yours, and so on ad infinitum. However, if we assume that the degree of recursion is bounded, then players need to estimate the opponent's degree of recursion (i.e., sophistication) to respond optimally. This induces a problem of inferring the opponent's sophistication, given behavioural exchanges. We show it is possible to deduce whether players make inferences about each other and quantify their sophistication on the basis of choices in sequential games. This rests on comparing generative models of choices with, and without, inference. Model comparison is demonstrated using simulated and real data from a ‘stag-hunt’. Finally, we note that exactly the same sophisticated behaviour can be achieved by optimising the utility function itself (through prosocial utility), producing unsophisticated but apparently altruistic agents. This may be relevant ethologically in hierarchal game theory and coevolution

Approximating n-player behavioural strategy nash equilibria using coevolution

Coevolutionary algorithms are plagued with a set of problems related to intransitivity that make it questionable what the end product of a coevolutionary run can achieve. With the introduction of solution concepts into coevolution, part of the issue was alleviated, however efficiently representing and achieving game theoretic solution concepts is still not a trivial task. In this paper we propose a coevolutionary algorithm that approximates behavioural strategy Nash equilibria in n-player zero sum games, by exploiting the minimax solution concept. In order to support our case we provide a set of experiments in both games of known and unknown equilibria. In the case of known equilibria, we can confirm our algorithm converges to the known solution, while in the case of unknown equilibria we can see a steady progress towards Nash. Copyright 2011 ACM

Walrasian Economics in Retrospect

Two basic tenets of theWalrasian model, behavior based on self-interested exogenous preferences and complete and costless contracting have recently come under critical scrutiny. First, social norms and psychological dispositions extending beyond the selfish motives of Homo economicus may have an important bearing on outcomes, even in competitive markets. Second, market outcomes depend on strategic interactions in which power in the political sense is exercised. It follows that economics must become more behavioral and more institutional. We can return to these themes of the classical tradition, now equipped with more the powerful mathematical tools developed over the past century.

Deep Reinforcement Learning from Self-Play in Imperfect-Information Games

Many real-world applications can be described as large-scale games of imperfect information. To deal with these challenging domains, prior work has focused on computing Nash equilibria in a handcrafted abstraction of the domain. In this paper we introduce the first scalable end-to-end approach to learning approximate Nash equilibria without prior domain knowledge. Our method combines fictitious self-play with deep reinforcement learning. When applied to Leduc poker, Neural Fictitious Self-Play (NFSP) approached a Nash equilibrium, whereas common reinforcement learning methods diverged. In Limit Texas Holdem, a poker game of real-world scale, NFSP learnt a strategy that approached the performance of state-of-the-art, superhuman algorithms based on significant domain expertise.Comment: updated version, incorporating conference feedbac

Extended Inclusive Fitness Theory bridges Economics and Biology through a common understanding of Social Synergy

Inclusive Fitness Theory (IFT) was proposed half a century ago by W.D. Hamilton to explain the emergence and maintenance of cooperation between individuals that allows the existence of society. Contemporary evolutionary ecology identified several factors that increase inclusive fitness, in addition to kin-selection, such as assortation or homophily, and social synergies triggered by cooperation. Here we propose an Extend Inclusive Fitness Theory (EIFT) that includes in the fitness calculation all direct and indirect benefits an agent obtains by its own actions, and through interactions with kin and with genetically unrelated individuals. This formulation focuses on the sustainable cost/benefit threshold ratio of cooperation and on the probability of agents sharing mutually compatible memes or genes. This broader description of the nature of social dynamics allows to compare the evolution of cooperation among kin and non-kin, intra- and inter-specific cooperation, co-evolution, the emergence of symbioses, of social synergies, and the emergence of division of labor. EIFT promotes interdisciplinary cross fertilization of ideas by allowing to describe the role for division of labor in the emergence of social synergies, providing an integrated framework for the study of both, biological evolution of social behavior and economic market dynamics.Comment: Bioeconomics, Synergy, Complexit