A Spatial Agent-Based Model of N-Person Prisoner's Dilemma Cooperation in a Socio-Geographic Community

The purpose of this paper is to present a spatial agent-based model of N-person prisoner's dilemma that is designed to simulate the collective communication and cooperation within a socio-geographic community. Based on a tight coupling of REPAST and a vector Geographic Information System, the model simulates the emergence of cooperation from the mobility behaviors and interaction strategies of citizen agents. To approximate human behavior, the agents are set as stochastic learning automata with Pavlovian personalities and attitudes. A review of the theory of the standard prisoner's dilemma, the iterated prisoner's dilemma, and the N-person prisoner's dilemma is given as well as an overview of the generic architecture of the agent-based model. The capabilities of the spatial N-person prisoner's dilemma component are demonstrated with several scenario simulation runs for varied initial cooperation percentages and mobility dynamics. Experimental results revealed that agent mobility and context preservation bring qualitatively different effects to the evolution of cooperative behavior in an analyzed spatial environment.Agent Based Modeling, Cooperation, Prisoners Dilemma, Spatial Interaction Model, Spatially Structured Social Dilemma, Geographic Information Systems

Reinforcement Learning Dynamics in Social Dilemmas

In this paper we replicate and advance Macy and Flache\'s (2002; Proc. Natl. Acad. Sci. USA, 99, 7229–7236) work on the dynamics of reinforcement learning in 2�2 (2-player 2-strategy) social dilemmas. In particular, we provide further insight into the solution concepts that they describe, illustrate some recent analytical results on the dynamics of their model, and discuss the robustness of such results to occasional mistakes made by players in choosing their actions (i.e. trembling hands). It is shown here that the dynamics of their model are strongly dependent on the speed at which players learn. With high learning rates the system quickly reaches its asymptotic behaviour; on the other hand, when learning rates are low, two distinctively different transient regimes can be clearly observed. It is shown that the inclusion of small quantities of randomness in players\' decisions can change the dynamics of the model dramatically.Reinforcement Learning; Replication; Game Theory; Social Dilemmas; Agent-Based; Slow Learning

The Evolution of Society

We re-examine the evolutionary stability of the tit-for-tat (tft) strategy in the context of the iterated prisoner's dilemma, as introduced by Axelrod and Hamilton. This environment involves a mixture of populations of "organisms" which interact with each other according to the rules of the prisoner's dilemma, from game theory. The tft strategy is nice, retaliatory and forgiving, and these properties contributed to the success of the strategy in the earlier experiments. However, it turns out that the property of being nice represents a weakness, when competing with an insular strategy, but the reverse is also true, which means that tft is not an evolutionarily stable strategy. In fact, insular strategies prove to be better at resisting incursion. Finally, we consider the implications of this result, in terms of naturally occurring societies.MIT Artificial Intelligence Laborator

Uncertainty and Cooperation: Analytical Results and a Simulated Agent Society

Uncertainty is an important factor that influences social evolution in natural and artificial environments. Here we distinguish between three aspects of uncertainty. Environmental uncertainty is the variance of resources in the environment, perceived uncertainty is the variance of the resource distribution as perceived by the organism and effective uncertainty is the variance of resources effectively enjoyed by individuals. We show analytically that perceived uncertainty is larger than environmental uncertainty and that effective uncertainty is smaller than perceived uncertainty, when cooperation is present. We use an agent society simulation in a two dimensional world for the generation of simulation data as one realisation of the analytical results. Together with our earlier theoretical work, results here show that cooperation can buffer the detrimental effects of uncertainty on the organism. The proposed conceptualisation of uncertainty can help in understanding its effects on social evolution and in designing artificial social environments.Agent-Based Modelling, Cooperation, Social Interaction Simulation, Uncertainty

AN EXAMINATION OF THE STABILITY OF COOPERATION IN A VOLUNTARY COLLECTIVE ACTION: THE CASE OF NONPOINT-SOURCE POLLUTION IN AN AGRICULTURAL WATERSHED

This paper addresses the collective action problem of nonpoint-source pollution control in a small agricultural watershed. At issue is the stability of cooperative behavior among a group of farmers, who have voluntarily agreed to discontinue their use of the herbicide atrazine due to high concentrations of the herbicide in a local water supply. Continued cooperation among the group is threatened by the unexpected cancellation of cyanazine, an inexpensive and widely used alternative to atrazine. With cyanazine no longer available, the farmers will face a significant increase in weed control costs if they continue to use products that do not contain atrazine. Is cooperation among the farmers still possible despite the increased cost of cooperating? This research explores the economic and behavioral factors that influence the collective outcome of this social dilemma. The collective action is modeled as a recurrent coordination problem. The producers (farmers) are engaged in a repeated assurance game with imperfect public information, where producers' choices are driven by the desire to coordinate their actions with the others in the group. A producer's decision to cooperate or defect is based on a threshold approach; if the number of others believed to be cooperating exceeds the level of cooperation required to make cooperation beneficial, then the producer will choose to cooperate. Otherwise, the producer will defect. Since producers are unable to directly observe the choices of the others in the group, each producer must rely on a subjective assessment of the group's behavior based on the realization of the public outcome, the concentration of atrazine in the lake. Producers use a naive Bayesian learning process to update their beliefs about the joint actions of the group. The formal learning process is modeled using a sequential quasi-Bayesian procedure that is consistent with the fictitious play model of learning. The interaction between the producers and the impact of their collective behavior on the levels of atrazine in the lake is formulated as a computational multi-agent system (MAS). The MAS is an artificial representation of the collective action problem that integrates the economic, behavioral and environmental factors that influence the decision-making process of producers. The MAS is used to simulate the evolution of collective behavior among the group and to evaluate the effectiveness of selected incentive mechanisms in preventing the collapse of joint cooperation. The results suggest that without additional incentives, farmers are likely to abandon their voluntary agreement and resume their use of atrazine within the watershed. It is then demonstrated how a combination of policy instruments can be used to alter the underlying game configuration of the collective action problem, resulting in cooperative outcomes. An ambient-based penalty, when used in conjunction with a subsidy payment, is shown to produce divergent incentive structures that shift the classification of the collective action away from a coordination problem with two equilibria to a mixed configuration containing several different game structures and many possible equilibria. This result has important consequences in terms of the evolution of producer behavior and the set of possible collective outcomes. The analysis concludes with an example, which demonstrates that when a mixture of game structures characterizes the collective action, joint cooperation is not a prerequisite to the realization of socially desirable outcomes. By carefully selecting the combination of subsidy payment and ambient penalty, a policy maker can manipulate the underlying structure of the collective action, whereby producers with the smallest impact on water quality choose to defect while all others cooperate.Environmental Economics and Policy,