A Study of AI Population Dynamics with Million-agent Reinforcement Learning

We conduct an empirical study on discovering the ordered collective dynamics obtained by a population of intelligence agents, driven by million-agent reinforcement learning. Our intention is to put intelligent agents into a simulated natural context and verify if the principles developed in the real world could also be used in understanding an artificially-created intelligent population. To achieve this, we simulate a large-scale predator-prey world, where the laws of the world are designed by only the findings or logical equivalence that have been discovered in nature. We endow the agents with the intelligence based on deep reinforcement learning (DRL). In order to scale the population size up to millions agents, a large-scale DRL training platform with redesigned experience buffer is proposed. Our results show that the population dynamics of AI agents, driven only by each agent's individual self-interest, reveals an ordered pattern that is similar to the Lotka-Volterra model studied in population biology. We further discover the emergent behaviors of collective adaptations in studying how the agents' grouping behaviors will change with the environmental resources. Both of the two findings could be explained by the self-organization theory in nature.Comment: Full version of the paper presented at AAMAS 2018 (International Conference on Autonomous Agents and Multiagent Systems

Models for an Ecosystem Approach to Fisheries

This document is one outcome from a workshop held in Gizo in October 2010 attended by 82 representatives from government, NGO's private sector, and communities. The target audience for the document is primarily organizations planning to work with coastal communities of Solomon Islands to implement Community-Based Resource Management (CBRM). It is however also envisaged that the document will serve as a reference for communities to better understand what to expect from their partners and also for donors, to be informed about agreed approaches amongst Solomon Islands stakeholders. This document does not attempt to summarize all the outcomes of the workshop; rather it focuses on the Solomon Islands Coral Triangle Initiative (CTI) National Plan of Action (NPoA): Theme 1: Support and implementation of CBRM and specifically, the scaling up of CBRM in Solomon Islands. Most of the principles given in this document are derived from experiences in coastal communities and ecosystems as, until relatively recently, these have received most attention in Solomon Islands resource management. It is recognized however that the majority of these principles will be applicable to both coastal and terrestrial initiatives. This document synthesizes information provided by stakeholders at the October 2010 workshop and covers some basic principles of engagement and implementation that have been learned over more than twenty years of activities by the stakeholder partners in Solomon Islands. The document updates and expands on a summary of guiding principles for CBRM which was originally prepared by the Solomon Islands Locally Managed Marine Area Network (SILMMA) in 2007

How predation and landscape fragmentation affect vole population dynamics

Background: Microtine species in Fennoscandia display a distinct north-south gradient from regular cycles to stable populations. The gradient has often been attributed to changes in the interactions between microtines and their predators. Although the spatial structure of the environment is known to influence predator-prey dynamics of a wide range of species, it has scarcely been considered in relation to the Fennoscandian gradient. Furthermore, the length of microtine breeding season also displays a north-south gradient. However, little consideration has been given to its role in shaping or generating population cycles. Because these factors covary along the gradient it is difficult to distinguish their effects experimentally in the field. The distinction is here attempted using realistic agent-based modelling. Methodology/Principal Findings: By using a spatially explicit computer simulation model based on behavioural and ecological data from the field vole (Microtus agrestis), we generated a number of repeated time series of vole densities whose mean population size and amplitude were measured. Subsequently, these time series were subjected to statistical autoregressive modelling, to investigate the effects on vole population dynamics of making predators more specialised, of altering the breeding season, and increasing the level of habitat fragmentation. We found that fragmentation as well as the presence of specialist predators are necessary for the occurrence of population cycles. Habitat fragmentation and predator assembly jointly determined cycle length and amplitude. Length of vole breeding season had little impact on the oscillations. Significance: There is good agreement between our results and the experimental work from Fennoscandia, but our results allow distinction of causation that is hard to unravel in field experiments. We hope our results will help understand the reasons for cycle gradients observed in other areas. Our results clearly demonstrate the importance of landscape fragmentation for population cycling and we recommend that the degree of fragmentation be more fully considered in future analyses of vole dynamics

Emergence of Self-Organized Symbol-Based Communication \ud in Artificial Creatures

In this paper, we describe a digital scenario where we simulated the emergence of self-organized symbol-based communication among artificial creatures inhabiting a \ud virtual world of unpredictable predatory events. In our experiment, creatures are autonomous agents that learn symbolic relations in an unsupervised manner, with no explicit feedback, and are able to engage in dynamical and autonomous communicative interactions with other creatures, even simultaneously. In order to synthesize a behavioral ecology and infer the minimum organizational constraints for the design of our creatures, \ud we examined the well-studied case of communication in vervet monkeys. Our results show that the creatures, assuming the role of sign users and learners, behave collectively as a complex adaptive system, where self-organized communicative interactions play a \ud major role in the emergence of symbol-based communication. We also strive in this paper for a careful use of the theoretical concepts involved, including the concepts of symbol and emergence, and we make use of a multi-level model for explaining the emergence of symbols in semiotic systems as a basis for the interpretation of inter-level relationships in the semiotic processes we are studying

Talking Helps: Evolving Communicating Agents for the Predator-Prey Pursuit Problem

We analyze a general model of multi-agent communication in which all agents communicate simultaneously to a message board. A genetic algorithm is used to evolve multi-agent languages for the predator agents in a version of the predator-prey pursuit problem. We show that the resulting behavior of the communicating multi-agent system is equivalent to that of a Mealy finite state machine whose states are determined by the agents’ usage of the evolved language. Simulations show that the evolution of a communication language improves the performance of the predators. Increasing the language size (and thus increasing the number of possible states in the Mealy machine) improves the performance even further. Furthermore, the evolved communicating predators perform significantly better than all previous work on similar preys. We introduce a method for incrementally increasing the language size which results in an effective coarse-to-fine search that significantly reduces the evolution time required to find a solution. We present some observations on the effects of language size, experimental setup, and prey difficulty on the evolved Mealy machines. In particular, we observe that the start state is often revisited, and incrementally increasing the language size results in smaller Mealy machines. Finally, a simple rule is derived that provides a pessimistic estimate on the minimum language size that should be used for any multi-agent problem

Coevolutionary dynamics of a variant of the cyclic Lotka-Volterra model with three-agent interactions

We study a variant of the cyclic Lotka-Volterra model with three-agent interactions. Inspired by a multiplayer variation of the Rock-Paper-Scissors game, the model describes an ideal ecosystem in which cyclic competition among three species develops through cooperative predation. Its rate equations in a well-mixed environment display a degenerate Hopf bifurcation, occurring as reactions involving two predators plus one prey have the same rate as reactions involving two preys plus one predator. We estimate the magnitude of the stochastic noise at the bifurcation point, where finite size effects turn neutrally stable orbits into erratically diverging trajectories. In particular, we compare analytic predictions for the extinction probability, derived in the Fokker-Planck approximation, with numerical simulations based on the Gillespie stochastic algorithm. We then extend the analysis of the phase portrait to heterogeneous rates. In a well-mixed environment, we observe a continuum of degenerate Hopf bifurcations, generalizing the above one. Neutral stability ensues from a complex equilibrium between different reactions. Remarkably, on a two-dimensional lattice, all bifurcations disappear as a consequence of the spatial locality of the interactions. In the second part of the paper, we investigate the effects of mobility in a lattice metapopulation model with patches hosting several agents. We find that strategies propagate along the arms of rotating spirals, as they usually do in models of cyclic dominance. We observe propagation instabilities in the regime of large wavelengths. We also examine three-agent interactions inducing nonlinear diffusion.Comment: 22 pages, 13 figures. v2: version accepted for publication in EPJ

Species abundance patterns in an ecosystem simulation studied through Fisher's logseries

We have developed an individual-based evolving predator-prey ecosystem simulation that integrates, for the first time, a complex individual behaviour model, an evolutionary mechanism and a speciation process, at an acceptable computational cost. In this article, we analyse the species abundance patterns observed in the communities generated by our simulation, based on Fisher's logseries. We propose a rigorous methodology for testing abundance data against the logseries. We show that our simulation produces coherent results, in terms of relative species abundance, when compared to classical ecological patterns. Some preliminary results are also provided about how our simulation is supporting ecological field results