Learning and innovative elements of strategy adoption rules expand cooperative network topologies

Cooperation plays a key role in the evolution of complex systems. However, the level of cooperation extensively varies with the topology of agent networks in the widely used models of repeated games. Here we show that cooperation remains rather stable by applying the reinforcement learning strategy adoption rule, Q-learning on a variety of random, regular, small-word, scale-free and modular network models in repeated, multi-agent Prisoners Dilemma and Hawk-Dove games. Furthermore, we found that using the above model systems other long-term learning strategy adoption rules also promote cooperation, while introducing a low level of noise (as a model of innovation) to the strategy adoption rules makes the level of cooperation less dependent on the actual network topology. Our results demonstrate that long-term learning and random elements in the strategy adoption rules, when acting together, extend the range of network topologies enabling the development of cooperation at a wider range of costs and temptations. These results suggest that a balanced duo of learning and innovation may help to preserve cooperation during the re-organization of real-world networks, and may play a prominent role in the evolution of self-organizing, complex systems.Comment: 14 pages, 3 Figures + a Supplementary Material with 25 pages, 3 Tables, 12 Figures and 116 reference

Discriminating to learn to discriminate.

Experiments in which subjects play simultaneously several finite prisoner's dilemma supergames reveal that many hypotheses used in the literature to explain cooperation are wrong. In particular the existence of player types is rejected as well as over-simplified behavioural postulates which allow for the existence of agents who make consistent errors. Experimental subjects turn out to permanently search for a better strategy. It is further suggested that the freedom to choose whether or not to play the prisoner's dilemma might be a key element in explaining observed cooperation levels in real dataPrisoner´s dilemma; Cooperation; Exit; Experiments; Learning;

Agent-Based Computational Economics

Agent-based computational economics (ACE) is the computational study of economies modeled as evolving systems of autonomous interacting agents. Starting from initial conditions, specified by the modeler, the computational economy evolves over time as its constituent agents repeatedly interact with each other and learn from these interactions. ACE is therefore a bottom-up culture-dish approach to the study of economic systems. This study discusses the key characteristics and goals of the ACE methodology. Eight currently active research areas are highlighted for concrete illustration. Potential advantages and disadvantages of the ACE methodology are considered, along with open questions and possible directions for future research.Agent-based computational economics; Autonomous agents; Interaction networks; Learning; Evolution; Mechanism design; Computational economics; Object-oriented programming.

Stubborn Learning

The paper studies a specific reinforcement learning rule in two-player games when each player faces a unidimensional strategy set. The essential feature of the rule is that a player keeps on incrementing her strategy in the same direction if and only if her utility increases. The paper concentrates on games on the square [0; 1] x [0; 1] with bilinear payoff functions such as the mixed extensions of 2 x 2 games. It studies the behavior of the system in the interior as well as on the borders of the strategy space. It precisely exhibits the trajectories of the system and the asymptotic states for symmetric, zero-sum, and twin games.

INVESTIGATIONS INTO THE COGNITIVE ABILITIES OF ALTERNATE LEARNING CLASSIFIER SYSTEM ARCHITECTURES

The Learning Classifier System (LCS) and its descendant, XCS, are promising paradigms for machine learning design and implementation. Whereas LCS allows classifier payoff predictions to guide system performance, XCS focuses on payoff-prediction accuracy instead, allowing it to evolve optimal classifier sets in particular applications requiring rational thought. This research examines LCS and XCS performance in artificial situations with broad social/commercial parallels, created using the non-Markov Iterated Prisoner\u27s Dilemma (IPD) game-playing scenario, where the setting is sometimes asymmetric and where irrationality sometimes pays. This research systematically perturbs a conventional IPD-playing LCS-based agent until it results in a full-fledged XCS-based agent, contrasting the simulated behavior of each LCS variant in terms of a number of performance measures. The intent is to examine the XCS paradigm to understand how it better copes with a given situation (if it does) than the LCS perturbations studied.Experiment results indicate that the majority of the architectural differences do have a significant effect on the agents\u27 performance with respect to the performance measures used in this research. The results of these competitions indicate that while each architectural difference significantly affected its agent\u27s performance, no single architectural difference could be credited as causing XCS\u27s demonstrated superiority in evolving optimal populations. Instead, the data suggests that XCS\u27s ability to evolve optimal populations in the multiplexer and IPD problem domains result from the combined and synergistic effects of multiple architectural differences.In addition, it is demonstrated that XCS is able to reliably evolve the Optimal Population [O] against the TFT opponent. This result supports Kovacs\u27 Optimality Hypothesis in the IPD environment and is significant because it is the first demonstrated occurrence of this ability in an environment other than the multiplexer and Woods problem domains.It is therefore apparent that while XCS performs better than its LCS-based counterparts, its demonstrated superiority may not be attributed to a single architectural characteristic. Instead, XCS\u27s ability to evolve optimal classifier populations in the multiplexer problem domain and in the IPD problem domain studied in this research results from the combined and synergistic effects of multiple architectural differences

Discriminating to learn to discriminate

Experiments in which subjects play simultaneously several finite prisoner's dilemma supergames reveal that many hypotheses used in the literature to explain cooperation are wrong. In particular the existence of player types is rejected as well as over-simplified behavioural postulates which allow for the existence of agents who make consistent errors. Experimental subjects turn out to permanently search for a better strategy. It is further suggested that the freedom to choose whether or not to play the prisoner's dilemma might be a key element in explaining observed cooperation levels in real dat

One Step at a Time: Does Gradualism Build Coordination?

We study how gradualism -- increasing required levels (“thresholds”) of contributions slowly over time rather than requiring a high level of contribution immediately -- affects individuals’ decisions to contribute to a public project. Using a laboratory binary choice minimum-effort coordination game, we randomly assign participants to three treatments: starting and continuing at a high threshold, starting at a low threshold but jumping to a high threshold after a few periods, and starting at a low threshold and gradually increasing the threshold over time (the “gradualism” treatment). We find that individuals coordinate most successfully at the high threshold in the gradualism treatment relative to the other two groups. We propose a theory based on belief updating to explain why gradualism works. We also discuss alternative explanations such as reinforcement learning, conditional cooperation, inertia, preference for consistency, and limited attention. Our findings point to a simple, voluntary mechanism to promote successful coordination when the capacity to impose sanctions is limited.Gradualism; Coordination; Cooperation; Public Goods; Belief-based Learning; Laboratory Experiment