Coupled Replicator Equations for the Dynamics of Learning in Multiagent Systems

Starting with a group of reinforcement-learning agents we derive coupled replicator equations that describe the dynamics of collective learning in multiagent systems. We show that, although agents model their environment in a self-interested way without sharing knowledge, a game dynamics emerges naturally through environment-mediated interactions. An application to rock-scissors-paper game interactions shows that the collective learning dynamics exhibits a diversity of competitive and cooperative behaviors. These include quasiperiodicity, stable limit cycles, intermittency, and deterministic chaos--behaviors that should be expected in heterogeneous multiagent systems described by the general replicator equations we derive.Comment: 4 pages, 3 figures, http://www.santafe.edu/projects/CompMech/papers/credlmas.html; updated references, corrected typos, changed conten

An Application of Attribution Theory to Developing Self-Esteem in Learning Disabled Adolescents

This research was published by the KU Center for Research on Learning, formerly known as the University of Kansas Institute for Research in Learning Disabilities.The study found that LD adolescents did not differ significantly from non-LD adolescents in their esponses to general self esteem and attribution questionnaires. Effort attribution training brought no significant increase in effort attributions for the experimental group of LD students. LD students reported the effort was a factor that explained success or failure in achievement tasks, but also reported that factors other than effort explained their personal success or failure on a specific spelling task

The effects of changes in the order of verbal labels and numerical values on children's scores on attitude and rating scales

Research with adults has shown that variations in verbal labels and numerical scale values on rating scales can affect the responses given. However, few studies have been conducted with children. The study aimed to examine potential differences in children’s responses to Likert-type rating scales according to their anchor points and scale direction, and to see whether or not such differences were stable over time. 130 British children, aged 9 to 11, completed six sets of Likert-type rating scales, presented in four different ways varying the position of positive labels and numerical values. The results showed, both initially and 8-12 weeks later, that presenting a positive label or a high score on the left of a scale led to significantly higher mean scores than did the other variations. These findings indicate that different arrangements of rating scales can produce different results which has clear implications for the administration of scales with children

Scale-free memory model for multiagent reinforcement learning. Mean field approximation and rock-paper-scissors dynamics

A continuous time model for multiagent systems governed by reinforcement learning with scale-free memory is developed. The agents are assumed to act independently of one another in optimizing their choice of possible actions via trial-and-error search. To gain awareness about the action value the agents accumulate in their memory the rewards obtained from taking a specific action at each moment of time. The contribution of the rewards in the past to the agent current perception of action value is described by an integral operator with a power-law kernel. Finally a fractional differential equation governing the system dynamics is obtained. The agents are considered to interact with one another implicitly via the reward of one agent depending on the choice of the other agents. The pairwise interaction model is adopted to describe this effect. As a specific example of systems with non-transitive interactions, a two agent and three agent systems of the rock-paper-scissors type are analyzed in detail, including the stability analysis and numerical simulation. Scale-free memory is demonstrated to cause complex dynamics of the systems at hand. In particular, it is shown that there can be simultaneously two modes of the system instability undergoing subcritical and supercritical bifurcation, with the latter one exhibiting anomalous oscillations with the amplitude and period growing with time. Besides, the instability onset via this supercritical mode may be regarded as "altruism self-organization". For the three agent system the instability dynamics is found to be rather irregular and can be composed of alternate fragments of oscillations different in their properties.Comment: 17 pages, 7 figur