A Study of AI Population Dynamics with Million-agent Reinforcement Learning

We conduct an empirical study on discovering the ordered collective dynamics obtained by a population of intelligence agents, driven by million-agent reinforcement learning. Our intention is to put intelligent agents into a simulated natural context and verify if the principles developed in the real world could also be used in understanding an artificially-created intelligent population. To achieve this, we simulate a large-scale predator-prey world, where the laws of the world are designed by only the findings or logical equivalence that have been discovered in nature. We endow the agents with the intelligence based on deep reinforcement learning (DRL). In order to scale the population size up to millions agents, a large-scale DRL training platform with redesigned experience buffer is proposed. Our results show that the population dynamics of AI agents, driven only by each agent's individual self-interest, reveals an ordered pattern that is similar to the Lotka-Volterra model studied in population biology. We further discover the emergent behaviors of collective adaptations in studying how the agents' grouping behaviors will change with the environmental resources. Both of the two findings could be explained by the self-organization theory in nature.Comment: Full version of the paper presented at AAMAS 2018 (International Conference on Autonomous Agents and Multiagent Systems

Evolutionary games on graphs

Game theory is one of the key paradigms behind many scientific disciplines from biology to behavioral sciences to economics. In its evolutionary form and especially when the interacting agents are linked in a specific social network the underlying solution concepts and methods are very similar to those applied in non-equilibrium statistical physics. This review gives a tutorial-type overview of the field for physicists. The first three sections introduce the necessary background in classical and evolutionary game theory from the basic definitions to the most important results. The fourth section surveys the topological complications implied by non-mean-field-type social network structures in general. The last three sections discuss in detail the dynamic behavior of three prominent classes of models: the Prisoner's Dilemma, the Rock-Scissors-Paper game, and Competing Associations. The major theme of the review is in what sense and how the graph structure of interactions can modify and enrich the picture of long term behavioral patterns emerging in evolutionary games.Comment: Review, final version, 133 pages, 65 figure

Emergence of Self-Organized Symbol-Based Communication \ud in Artificial Creatures

In this paper, we describe a digital scenario where we simulated the emergence of self-organized symbol-based communication among artificial creatures inhabiting a \ud virtual world of unpredictable predatory events. In our experiment, creatures are autonomous agents that learn symbolic relations in an unsupervised manner, with no explicit feedback, and are able to engage in dynamical and autonomous communicative interactions with other creatures, even simultaneously. In order to synthesize a behavioral ecology and infer the minimum organizational constraints for the design of our creatures, \ud we examined the well-studied case of communication in vervet monkeys. Our results show that the creatures, assuming the role of sign users and learners, behave collectively as a complex adaptive system, where self-organized communicative interactions play a \ud major role in the emergence of symbol-based communication. We also strive in this paper for a careful use of the theoretical concepts involved, including the concepts of symbol and emergence, and we make use of a multi-level model for explaining the emergence of symbols in semiotic systems as a basis for the interpretation of inter-level relationships in the semiotic processes we are studying

The emergence of reciprocally beneficial cooperation

This paper offers a new and robust model of the emergence and persistence of cooperation. In the model, interactions are anonymous, the population is well-mixed, and the evolutionary process selects strategies according to material payoffs. The cooperation problem is modelled as a game similar to Prisoner’s Dilemma, but there is an outside option of nonparticipation and the payoff to mutual cooperation is stochastic; with positive probability, this payoff exceeds that from cheating against a cooperator. Under mild conditions, mutually beneficial cooperation occurs in equilibrium. This is possible because the non-participation option holds down the equilibrium frequency of cheating.Cooperation; voluntary participation; random payoffs.

Chris Cannings: A Life in Games

Chris Cannings was one of the pioneers of evolutionary game theory. His early work was inspired by the formulations of John Maynard Smith, Geoff Parker and Geoff Price; Chris recognized the need for a strong mathematical foundation both to validate stated results and to give a basis for extensions of the models. He was responsible for fundamental results on matrix games, as well as much of the theory of the important war of attrition game, patterns of evolutionarily stable strategies, multiplayer games and games on networks. In this paper we describe his work, key insights and their influence on research by others in this increasingly important field. Chris made substantial contributions to other areas such as population genetics and segregation analysis, but it was to games that he always returned. This review is written by three of his students from different stages of his career