A Study of AI Population Dynamics with Million-agent Reinforcement Learning

We conduct an empirical study on discovering the ordered collective dynamics obtained by a population of intelligence agents, driven by million-agent reinforcement learning. Our intention is to put intelligent agents into a simulated natural context and verify if the principles developed in the real world could also be used in understanding an artificially-created intelligent population. To achieve this, we simulate a large-scale predator-prey world, where the laws of the world are designed by only the findings or logical equivalence that have been discovered in nature. We endow the agents with the intelligence based on deep reinforcement learning (DRL). In order to scale the population size up to millions agents, a large-scale DRL training platform with redesigned experience buffer is proposed. Our results show that the population dynamics of AI agents, driven only by each agent's individual self-interest, reveals an ordered pattern that is similar to the Lotka-Volterra model studied in population biology. We further discover the emergent behaviors of collective adaptations in studying how the agents' grouping behaviors will change with the environmental resources. Both of the two findings could be explained by the self-organization theory in nature.Comment: Full version of the paper presented at AAMAS 2018 (International Conference on Autonomous Agents and Multiagent Systems

Extended Inclusive Fitness Theory bridges Economics and Biology through a common understanding of Social Synergy

Inclusive Fitness Theory (IFT) was proposed half a century ago by W.D. Hamilton to explain the emergence and maintenance of cooperation between individuals that allows the existence of society. Contemporary evolutionary ecology identified several factors that increase inclusive fitness, in addition to kin-selection, such as assortation or homophily, and social synergies triggered by cooperation. Here we propose an Extend Inclusive Fitness Theory (EIFT) that includes in the fitness calculation all direct and indirect benefits an agent obtains by its own actions, and through interactions with kin and with genetically unrelated individuals. This formulation focuses on the sustainable cost/benefit threshold ratio of cooperation and on the probability of agents sharing mutually compatible memes or genes. This broader description of the nature of social dynamics allows to compare the evolution of cooperation among kin and non-kin, intra- and inter-specific cooperation, co-evolution, the emergence of symbioses, of social synergies, and the emergence of division of labor. EIFT promotes interdisciplinary cross fertilization of ideas by allowing to describe the role for division of labor in the emergence of social synergies, providing an integrated framework for the study of both, biological evolution of social behavior and economic market dynamics.Comment: Bioeconomics, Synergy, Complexit

Agent-based pedestrian modelling

When the focus of interest in geographical systems is at the very fine scale, at the level of streets and buildings for example, movement becomes central to simulations of how spatial activities are used and develop. Recent advances in computing power and the acquisition of fine scale digital data now mean that we are able to attempt to understand and predict such phenomena with the focus in spatial modelling changing to dynamic simulations of the individual and collective behaviour of individual decision-making at such scales. In this Chapter, we develop ideas about how such phenomena can be modelled showing first how randomness and geometry are all important to local movement and how ordered spatial structures emerge from such actions. We focus on developing these ideas for pedestrians showing how random walks constrained by geometry but aided by what agents can see, determine how individuals respond to locational patterns. We illustrate these ideas with three types of example: first for local scale street scenes where congestion and flocking is all important, second for coarser scale shopping centres such as malls where economic preference interferes much more with local geometry, and finally for semi-organised street festivals where management and control by police and related authorities is integral to the way crowds move

The roundtable: an abstract model of conversation dynamics

Is it possible to abstract a formal mechanism originating schisms and governing the size evolution of social conversations? In this work a constructive solution to such problem is proposed: an abstract model of a generic N-party turn-taking conversation. The model develops from simple yet realistic assumptions derived from experimental evidence, abstracts from conversation content and semantics while including topological information, and is driven by stochastic dynamics. We find that a single mechanism - namely the dynamics of conversational party's individual fitness, as related to conversation size - controls the development of the self-organized schisming phenomenon. Potential generalizations of the model - including individual traits and preferences, memory effects and more elaborated conversational topologies - may find important applications also in other fields of research, where dynamically-interacting and networked agents play a fundamental role.Comment: 18 pages, 4 figures, to be published in Journal of Artificial Societies and Social Simulatio