Scale-free memory model for multiagent reinforcement learning. Mean field approximation and rock-paper-scissors dynamics

A continuous time model for multiagent systems governed by reinforcement learning with scale-free memory is developed. The agents are assumed to act independently of one another in optimizing their choice of possible actions via trial-and-error search. To gain awareness about the action value the agents accumulate in their memory the rewards obtained from taking a specific action at each moment of time. The contribution of the rewards in the past to the agent current perception of action value is described by an integral operator with a power-law kernel. Finally a fractional differential equation governing the system dynamics is obtained. The agents are considered to interact with one another implicitly via the reward of one agent depending on the choice of the other agents. The pairwise interaction model is adopted to describe this effect. As a specific example of systems with non-transitive interactions, a two agent and three agent systems of the rock-paper-scissors type are analyzed in detail, including the stability analysis and numerical simulation. Scale-free memory is demonstrated to cause complex dynamics of the systems at hand. In particular, it is shown that there can be simultaneously two modes of the system instability undergoing subcritical and supercritical bifurcation, with the latter one exhibiting anomalous oscillations with the amplitude and period growing with time. Besides, the instability onset via this supercritical mode may be regarded as "altruism self-organization". For the three agent system the instability dynamics is found to be rather irregular and can be composed of alternate fragments of oscillations different in their properties.Comment: 17 pages, 7 figur

Escherichia coli Bacteriocins: Antimicrobial Efficacy and Prevalence among Isolates from Patients with Bacteraemia

Bacteriocins are antimicrobial peptides generally active against bacteria closely related to the producer. Escherichia coli produces two types of bacteriocins, colicins and microcins. The in vitro efficacy of isolated colicins E1, E6, E7, K and M, was assessed against Escherichia coli strains from patients with bacteraemia of urinary tract origin. Colicin E7 was most effective, as only 13% of the tested strains were resistant. On the other hand, 32%, 33%, 43% and 53% of the tested strains exhibited resistance to colicins E6, K, M and E1. Moreover, the inhibitory activity of individual colicins E1, E6, E7, K and M and combinations of colicins K, M, E7 and E1, E6, E7, K, M were followed in liquid broth for 24 hours. Resistance against individual colicins developed after 9 hours of treatment. On the contrary, resistance development against the combined action of 5 colicins was not observed. One hundred and five E. coli strains from patients with bacteraemia were screened by PCR for the presence of 5 colicins and 7 microcins. Sixty-six percent of the strains encoded at least one bacteriocin, 43% one or more colicins, and 54% one or more microcins. Microcins were found to co-occur with toxins, siderophores, adhesins and with the Toll/Interleukin-1 receptor domain-containing protein involved in suppression of innate immunity, and were significantly more prevalent among strains from non-immunocompromised patients. In addition, microcins were highly prevalent among non-multidrug-resistant strains compared to multidrug-resistant strains. Our results indicate that microcins contribute to virulence of E. coli instigating bacteraemia of urinary tract origin

Mapping Statistics.

<p>These summary statistics list the number of molecules assembled to maps and the number of map assemblies generated during the processing of data from a single mapcard. The mixed mapcard bore DNA from both <i>Escherichia</i> and <i>Shigella</i>. The maps which were generated were circular and passed standard quality control.</p

<i>Shigella</i> Genomes.

<p>Complete circularized <i>S. dysenteriae</i> genomes from mixed DNA mapcard run aligned to genome from individual run.</p

<i>E. coli</i> Genomes.

<p>Complete circularized <i>E. coli</i> genome maps created as a result of a mixed DNA mapcard run aligned to an identical map from individual run.</p

Assembly Statistics.

<p>The columns display the values for the assembly statistics for E. coli and S. dysenteriae genomic maps from the combined run and individual runs. All genomes were circularized and passed standard QC.</p

Multiple Surfaces.

<p>A surface subdivided in two permits parallel mapping efforts during a single run with a single set of consumables.</p