Scale-free memory model for multiagent reinforcement learning. Mean field approximation and rock-paper-scissors dynamics

A continuous time model for multiagent systems governed by reinforcement learning with scale-free memory is developed. The agents are assumed to act independently of one another in optimizing their choice of possible actions via trial-and-error search. To gain awareness about the action value the agents accumulate in their memory the rewards obtained from taking a specific action at each moment of time. The contribution of the rewards in the past to the agent current perception of action value is described by an integral operator with a power-law kernel. Finally a fractional differential equation governing the system dynamics is obtained. The agents are considered to interact with one another implicitly via the reward of one agent depending on the choice of the other agents. The pairwise interaction model is adopted to describe this effect. As a specific example of systems with non-transitive interactions, a two agent and three agent systems of the rock-paper-scissors type are analyzed in detail, including the stability analysis and numerical simulation. Scale-free memory is demonstrated to cause complex dynamics of the systems at hand. In particular, it is shown that there can be simultaneously two modes of the system instability undergoing subcritical and supercritical bifurcation, with the latter one exhibiting anomalous oscillations with the amplitude and period growing with time. Besides, the instability onset via this supercritical mode may be regarded as "altruism self-organization". For the three agent system the instability dynamics is found to be rather irregular and can be composed of alternate fragments of oscillations different in their properties.Comment: 17 pages, 7 figur

Evolutionary instability of Zero Determinant strategies demonstrates that winning isn't everything

Zero Determinant (ZD) strategies are a new class of probabilistic and conditional strategies that are able to unilaterally set the expected payoff of an opponent in iterated plays of the Prisoner's Dilemma irrespective of the opponent's strategy, or else to set the ratio between a ZD player's and their opponent's expected payoff. Here we show that while ZD strategies are weakly dominant, they are not evolutionarily stable and will instead evolve into less coercive strategies. We show that ZD strategies with an informational advantage over other players that allows them to recognize other ZD strategies can be evolutionarily stable (and able to exploit other players). However, such an advantage is bound to be short-lived as opposing strategies evolve to counteract the recognition.Comment: 14 pages, 4 figures. Change in title (again!) to comply with Nature Communications requirements. To appear in Nature Communication

Mobility promotes and jeopardizes biodiversity in rock-paper-scissors games

Biodiversity is essential to the viability of ecological systems. Species diversity in ecosystems is promoted by cyclic, non-hierarchical interactions among competing populations. Such non-transitive relations lead to an evolution with central features represented by the `rock-paper-scissors' game, where rock crushes scissors, scissors cut paper, and paper wraps rock. In combination with spatial dispersal of static populations, this type of competition results in the stable coexistence of all species and the long-term maintenance of biodiversity. However, population mobility is a central feature of real ecosystems: animals migrate, bacteria run and tumble. Here, we observe a critical influence of mobility on species diversity. When mobility exceeds a certain value, biodiversity is jeopardized and lost. In contrast, below this critical threshold all subpopulations coexist and an entanglement of travelling spiral waves forms in the course of temporal evolution. We establish that this phenomenon is robust, it does not depend on the details of cyclic competition or spatial environment. These findings have important implications for maintenance and evolution of ecological systems and are relevant for the formation and propagation of patterns in excitable media, such as chemical kinetics or epidemic outbreaks.Comment: Final submitted version; the printed version can be found at http://dx.doi.org/10.1038/nature06095 Supplementary movies are available at http://www.theorie.physik.uni-muenchen.de/lsfrey/images_content/movie1.AVI and http://www.theorie.physik.uni-muenchen.de/lsfrey/images_content/movie2.AV

Top Quark Physics at the Tevatron

We review the field of top-quark physics with an emphasis on experimental techniques. The role of the top quark in the Standard Model of particle physics is summarized and the basic phenomenology of top-quark production and decay is introduced. We discuss how contributions from physics beyond the Standard Model could affect top-quark properties or event samples. The many measurements made at the Fermilab Tevatron, which test the Standard Model predictions or probe for direct evidence of new physics using the top-quark event samples, are reviewed here.Comment: 50 pages, 17 figures, 2 tables; version accepted by Review of Modern Physic

Sociobiological Control of Plasmid copy number

Background:
All known mechanisms and genes responsible for the regulation of plasmid replication lie with the plasmid rather than the chromosome. It is possible therefore that there can be copy-up mutants. Copy-up mutants will have within host selective advantage. This would eventually result into instability of bacteria-plasmid association. In spite of this possibility low copy number plasmids appear to exist stably in host populations. We examined this paradox using a computer simulation model.

Model:
Our multilevel selection model assumes a wild type with tightly regulated replication to ensure low copy number. A mutant with slightly relaxed replication regulation can act as a “cheater” or “selfish” plasmid and can enjoy a greater within-host-fitness. However the host of a cheater plasmid has to pay a greater cost. As a result, in host level competition, host cell with low copy number plasmid has a greater fitness. Furthermore, another mutant that has lost the genes required for conjugation was introduced in the model. The non-conjugal mutant was assumed to undergo conjugal transfer in the presence of another conjugal plasmid in the host cell.

Results:
The simulatons showed that if the cost of carrying a plasmid was low, the copy-up mutant could drive the wild type to extinction or very low frequencies. Consequently, another mutant with a higher copy number could invade the first invader. This process could result into an increasing copy number. However above a certain copy number within-host selection was overcompensated by host level selection leading to a rock-paper-scissor (RPS) like situation. The RPS situation allowed the coexistence of high and low copy number plasmids. The non-conjugal “hypercheaters” could further arrest the copy numbers to a substantially lower level.

Conclusions:
These sociobiological interactions might explain the stability of copy numbers better than molecular mechanisms of replication regulation alone

Defensive alliances in spatial models of cyclical population interactions

As a generalization of the 3-strategy Rock-Scissors-Paper game dynamics in space, cyclical interaction models of six mutating species are studied on a square lattice, in which each species is supposed to have two dominant, two subordinated and a neutral interacting partner. Depending on their interaction topologies, these systems can be classified into four (isomorphic) groups exhibiting significantly different behaviors as a function of mutation rate. On three out of four cases three (or four) species form defensive alliances which maintain themselves in a self-organizing polydomain structure via cyclic invasions. Varying the mutation rate this mechanism results in an ordering phenomenon analogous to that of magnetic Ising model.Comment: 4 pages, 3 figure

Phase transition in a spatial Lotka-Volterra model

Spatial evolution is investigated in a simulated system of nine competing and mutating bacterium strains, which mimics the biochemical war among bacteria capable of producing two different bacteriocins (toxins) at most. Random sequential dynamics on a square lattice is governed by very symmetrical transition rules for neighborhood invasion of sensitive strains by killers, killers by resistants, and resistants by by sensitives. The community of the nine possible toxicity/resistance types undergoes a critical phase transition as the uniform transmutation rates between the types decreases below a critical value $P_c$ above which all the nine types of strain coexist with equal frequencies. Passing the critical mutation rate from above, the system collapses into one of the three topologically identical states, each consisting of three strain types. Of the three final states each accrues with equal probability and all three maintain themselves in a self-organizing polydomain structure via cyclic invasions. Our Monte Carlo simulations support that this symmetry breaking transition belongs to the universality class of the three-state Potts model.Comment: 4 page

Measurement of the ttbar Production Cross Section in ppbar Collisions at sqrt(s)=1.96 TeV using Lepton + Jets Events with Lifetime b-tagging

We present a measurement of the top quark pair ($t\bar{t}$) production cross section ($\sigma_{t\bar{t}}$) in $p\bar{p}$ collisions at $\sqrt{s}=1.96$ TeV using 230 pb$^{-1}$ of data collected by the D0 experiment at the Fermilab Tevatron Collider. We select events with one charged lepton (electron or muon), missing transverse energy, and jets in the final state. We employ lifetime-based b-jet identification techniques to further enhance the $t\bar{t}$ purity of the selected sample. For a top quark mass of 175 GeV, we measure $\sigma_{t\bar{t}}=8.6^{+1.6}_{-1.5}(stat.+syst.)\pm 0.6(lumi.)$ pb, in agreement with the standard model expectation.Comment: 7 pages, 2 figures, 3 tables Submitted to Phys.Rev.Let

Search for squarks and gluinos with the ATLAS detector in final states with jets and missing transverse momentum using √s=8 TeV proton-proton collision data

A search for squarks and gluinos in final states containing high-p T jets, missing transverse momentum and no electrons or muons is presented. The data were recorded in 2012 by the ATLAS experiment in s√=8 TeV proton-proton collisions at the Large Hadron Collider, with a total integrated luminosity of 20.3 fb−1. Results are interpreted in a variety of simplified and specific supersymmetry-breaking models assuming that R-parity is conserved and that the lightest neutralino is the lightest supersymmetric particle. An exclusion limit at the 95% confidence level on the mass of the gluino is set at 1330 GeV for a simplified model incorporating only a gluino and the lightest neutralino. For a simplified model involving the strong production of first- and second-generation squarks, squark masses below 850 GeV (440 GeV) are excluded for a massless lightest neutralino, assuming mass degenerate (single light-flavour) squarks. In mSUGRA/CMSSM models with tan β = 30, A 0 = −2m 0 and μ > 0, squarks and gluinos of equal mass are excluded for masses below 1700 GeV. Additional limits are set for non-universal Higgs mass models with gaugino mediation and for simplified models involving the pair production of gluinos, each decaying to a top squark and a top quark, with the top squark decaying to a charm quark and a neutralino. These limits extend the region of supersymmetric parameter space excluded by previous searches with the ATLAS detector

Single hadron response measurement and calorimeter jet energy scale uncertainty with the ATLAS detector at the LHC

The uncertainty on the calorimeter energy response to jets of particles is derived for the ATLAS experiment at the Large Hadron Collider (LHC). First, the calorimeter response to single isolated charged hadrons is measured and compared to the Monte Carlo simulation using proton-proton collisions at centre-of-mass energies of sqrt(s) = 900 GeV and 7 TeV collected during 2009 and 2010. Then, using the decay of K_s and Lambda particles, the calorimeter response to specific types of particles (positively and negatively charged pions, protons, and anti-protons) is measured and compared to the Monte Carlo predictions. Finally, the jet energy scale uncertainty is determined by propagating the response uncertainty for single charged and neutral particles to jets. The response uncertainty is 2-5% for central isolated hadrons and 1-3% for the final calorimeter jet energy scale.Comment: 24 pages plus author list (36 pages total), 23 figures, 1 table, submitted to European Physical Journal