Scale-free memory model for multiagent reinforcement learning. Mean field approximation and rock-paper-scissors dynamics

A continuous time model for multiagent systems governed by reinforcement learning with scale-free memory is developed. The agents are assumed to act independently of one another in optimizing their choice of possible actions via trial-and-error search. To gain awareness about the action value the agents accumulate in their memory the rewards obtained from taking a specific action at each moment of time. The contribution of the rewards in the past to the agent current perception of action value is described by an integral operator with a power-law kernel. Finally a fractional differential equation governing the system dynamics is obtained. The agents are considered to interact with one another implicitly via the reward of one agent depending on the choice of the other agents. The pairwise interaction model is adopted to describe this effect. As a specific example of systems with non-transitive interactions, a two agent and three agent systems of the rock-paper-scissors type are analyzed in detail, including the stability analysis and numerical simulation. Scale-free memory is demonstrated to cause complex dynamics of the systems at hand. In particular, it is shown that there can be simultaneously two modes of the system instability undergoing subcritical and supercritical bifurcation, with the latter one exhibiting anomalous oscillations with the amplitude and period growing with time. Besides, the instability onset via this supercritical mode may be regarded as "altruism self-organization". For the three agent system the instability dynamics is found to be rather irregular and can be composed of alternate fragments of oscillations different in their properties.Comment: 17 pages, 7 figur

Modulation of immune responses by targeting CD169/Siglec-1 with the glycan ligand

A fundamental role in the plant-bacterium interaction for Gram-negative phytopathogenic bacteria is played by membrane constituents, such as proteins, lipopoly- or lipooligosaccharides (LPS, LOS) and Capsule Polysaccharides (CPS). In the frame of the understanding the molecular basis of plant bacterium interaction, the Gram-negative bacterium Agrobacterium vitis was selected in this study. It is a phytopathogenic member of the Rhizobiaceae family and it induces the crown gall disease selectively on grapevines (Vitis vinifera). A. vitis wild type strain F2/5, and its mutant in the quorum sensing gene ΔaviR, were studied. The wild type produces biosurfactants; it is considered a model to study surface motility, and it causes necrosis on grapevine roots and HR (Hypersensitive Response) on tobacco. Conversely, the mutant does not show any surface motility and does not produce any surfactant material; additionally, it induces neither necrosis on grape, nor HR on tobacco. Therefore, the two strains were analyzed to shed some light on the QS regulation of LOS structure and the consequent variation, if any, on HR response. LOS from both strains were isolated and characterized: the two LOS structures maintained several common features and differed for few others. With regards to the common patterns, firstly: the Lipid A region was not phosphorylated at C4 of the non reducing glucosamine but glycosylated by an uronic acid (GalA) unit, secondly: a third Kdo and the rare Dha (3-deoxy-lyxo-2-heptulosaric acid) moiety was present. Importantly, the third Kdo and the Dha residues were substituted by rhamnose in a not stoichiometric fashion, giving four different oligosaccharide species. The proportions among these four species, is the key difference between the LOSs from both the two bacteria. LOS from both strains and Lipid A from wild type A. vitis are now examined for their HR potential in tobacco leaves and grapevine roots