Scale-free memory model for multiagent reinforcement learning. Mean field approximation and rock-paper-scissors dynamics

A continuous time model for multiagent systems governed by reinforcement learning with scale-free memory is developed. The agents are assumed to act independently of one another in optimizing their choice of possible actions via trial-and-error search. To gain awareness about the action value the agents accumulate in their memory the rewards obtained from taking a specific action at each moment of time. The contribution of the rewards in the past to the agent current perception of action value is described by an integral operator with a power-law kernel. Finally a fractional differential equation governing the system dynamics is obtained. The agents are considered to interact with one another implicitly via the reward of one agent depending on the choice of the other agents. The pairwise interaction model is adopted to describe this effect. As a specific example of systems with non-transitive interactions, a two agent and three agent systems of the rock-paper-scissors type are analyzed in detail, including the stability analysis and numerical simulation. Scale-free memory is demonstrated to cause complex dynamics of the systems at hand. In particular, it is shown that there can be simultaneously two modes of the system instability undergoing subcritical and supercritical bifurcation, with the latter one exhibiting anomalous oscillations with the amplitude and period growing with time. Besides, the instability onset via this supercritical mode may be regarded as "altruism self-organization". For the three agent system the instability dynamics is found to be rather irregular and can be composed of alternate fragments of oscillations different in their properties.Comment: 17 pages, 7 figur

Size Doesn't Matter: Towards a More Inclusive Philosophy of Biology

notes: As the primary author, O’Malley drafted the paper, and gathered and analysed data (scientific papers and talks). Conceptual analysis was conducted by both authors.publication-status: Publishedtypes: ArticlePhilosophers of biology, along with everyone else, generally perceive life to fall into two broad categories, the microbes and macrobes, and then pay most of their attention to the latter. ‘Macrobe’ is the word we propose for larger life forms, and we use it as part of an argument for microbial equality. We suggest that taking more notice of microbes – the dominant life form on the planet, both now and throughout evolutionary history – will transform some of the philosophy of biology’s standard ideas on ontology, evolution, taxonomy and biodiversity. We set out a number of recent developments in microbiology – including biofilm formation, chemotaxis, quorum sensing and gene transfer – that highlight microbial capacities for cooperation and communication and break down conventional thinking that microbes are solely or primarily single-celled organisms. These insights also bring new perspectives to the levels of selection debate, as well as to discussions of the evolution and nature of multicellularity, and to neo-Darwinian understandings of evolutionary mechanisms. We show how these revisions lead to further complications for microbial classification and the philosophies of systematics and biodiversity. Incorporating microbial insights into the philosophy of biology will challenge many of its assumptions, but also give greater scope and depth to its investigations

Integrated analysis of environmental and genetic influences on cord blood DNA methylation in new-borns

Epigenetic processes, including DNA methylation (DNAm), are among the mechanisms allowing integration of genetic and environmental factors to shape cellular function. While many studies have investigated either environmental or genetic contributions to DNAm, few have assessed their integrated effects. Here we examine the relative contributions of prenatal environmental factors and genotype on DNA methylation in neonatal blood at variably methylated regions (VMRs) in 4 independent cohorts (overall n = 2365). We use Akaike’s information criterion to test which factors best explain variability of methylation in the cohort-specific VMRs: several prenatal environmental factors (E), genotypes in cis (G), or their additive (G + E) or interaction (GxE) effects. Genetic and environmental factors in combination best explain DNAm at the majority of VMRs. The CpGs best explained by either G, G + E or GxE are functionally distinct. The enrichment of genetic variants from GxE models in GWAS for complex disorders supports their importance for disease risk

The political psychology of reproductive strategies

We argue that several reproductive strategies that parental investment theory suggests are adaptive for men and for women are in fact most adaptive for individuals able and willing to use power in dominance relationships. We tested whether people's support for social dominance mediates their willingness to use these reproductive strategies. We hypothesized that the strategies of multiple simultaneous mating, resistance to caring for children as one's own, and sexual jealousy will be appealing especially to men who approve of social dominance, and that finding a high-status, high-earning mate will appeal especially to women who approve of social dominance. Support for the hypotheses using multiple samples and multiple measures was found, and theoretical issues are discussed