Scale-free memory model for multiagent reinforcement learning. Mean field approximation and rock-paper-scissors dynamics

A continuous time model for multiagent systems governed by reinforcement learning with scale-free memory is developed. The agents are assumed to act independently of one another in optimizing their choice of possible actions via trial-and-error search. To gain awareness about the action value the agents accumulate in their memory the rewards obtained from taking a specific action at each moment of time. The contribution of the rewards in the past to the agent current perception of action value is described by an integral operator with a power-law kernel. Finally a fractional differential equation governing the system dynamics is obtained. The agents are considered to interact with one another implicitly via the reward of one agent depending on the choice of the other agents. The pairwise interaction model is adopted to describe this effect. As a specific example of systems with non-transitive interactions, a two agent and three agent systems of the rock-paper-scissors type are analyzed in detail, including the stability analysis and numerical simulation. Scale-free memory is demonstrated to cause complex dynamics of the systems at hand. In particular, it is shown that there can be simultaneously two modes of the system instability undergoing subcritical and supercritical bifurcation, with the latter one exhibiting anomalous oscillations with the amplitude and period growing with time. Besides, the instability onset via this supercritical mode may be regarded as "altruism self-organization". For the three agent system the instability dynamics is found to be rather irregular and can be composed of alternate fragments of oscillations different in their properties.Comment: 17 pages, 7 figur

The assessment of circulating volume using inferior vena cava collapse index and carotid Doppler velocity time integral in healthy volunteers: a pilot study

International audienceBackground: Assessment of circulating volume and the requirement for fluid replacement are fundamental to resuscitation but remain largely empirical. Passive leg raise (PLR) may determine fluid responders while avoiding potential fluid overload. We hypothesised that inferior vena cava collapse index (IVCCI) and carotid artery blood flow would change predictably in response to PLR, potentially providing a non-invasive tool to assess circulating volume and identifying fluid responsive patients.Methods: We conducted a prospective proof of concept pilot study on fasted healthy volunteers. One operator measured IVC diameter during quiet respiration and sniff, and carotid artery flow. Stroke volume (SV) was also measured using suprasternal Doppler. Our primary endpoint was change in IVCCI after PLR. We also studied changes in IVCCI after “sniff”, and correlation between carotid artery flow and SV.Results: Passive leg raise was associated with significant reduction in the mean inferior vena cava collapsibility index from 0.24 to 0.17 (p < 0.01). Mean stroke volume increased from 56.0 to 69.2 mL (p < 0.01). There was no significant change in common carotid artery blood flow. Changes in physiology consequent upon passive leg raise normalised rapidly.Discussion: Passive leg raise is associated with a decrease of IVCCI and increase in stroke volume. However, the wide range of values observed suggests that factors other than circulating volume predominate in determining the proportion of collapse with respiration.Conclusion: In contrast to other studies, we did not find that carotid blood flow increased with passive leg raise. Rapid normalisation of post-PLR physiology may account for this

Biogeographical Survey Identifies Consistent Alternative Physiological Optima and a Minor Role for Environmental Drivers in Maintaining a Polymorphism

The contribution of adaptive mechanisms in maintaining genetic polymorphisms is still debated in many systems. To understand the contribution of selective factors in maintaining polymorphism, we investigated large-scale (>1000 km) geographic variation in morph frequencies and fitness-related physiological traits in the damselfly Nehalennia irene. As fitness-related physiological traits, we investigated investment in immune function (phenoloxidase activity), energy storage and fecundity (abdomen protein and lipid content), and flight muscles (thorax protein content). In the first part of the study, our aim was to identify selective agents maintaining the large-scale spatial variation in morph frequencies. Morph frequencies varied considerably among populations, but, in contrast to expectation, in a geographically unstructured way. Furthermore, frequencies co-varied only weakly with the numerous investigated ecological parameters. This suggests that spatial frequency patterns are driven by stochastic processes, or alternatively, are consequence of highly variable and currently unidentified ecological conditions. In line with this, the investigated ecological parameters did not affect the fitness-related physiological traits differently in both morphs. In the second part of the study, we aimed at identifying trade-offs between fitness-related physiological traits that may contribute to the local maintenance of both colour morphs by defining alternative phenotypic optima, and test the spatial consistency of such trade-off patterns. The female morph with higher levels of phenoloxidase activity had a lower thorax protein content, and vice versa, suggesting a trade-off between investments in immune function and in flight muscles. This physiological trade-off was consistent across the geographical scale studied and supports widespread correlational selection, possibly driven by male harassment, favouring alternative trait combinations in both female morphs

l-Arginine stimulates proliferation and prevents endotoxin-induced death of intestinal cells

This study tested the hypothesis that l-arginine (Arg) may stimulate cell proliferation and prevent lipopolysaccharide (LPS)-induced death of intestinal cells. Intestinal porcine epithelial cells (IPEC-1) were cultured for 4 days in Arg-free Dulbecco’s modified Eagle’s-F12 Ham medium (DMEM-F12) containing 10, 100 or 350 μM Arg and 0 or 20 ng/ml LPS. Cell numbers, protein concentrations, protein synthesis and degradation, as well as mammalian target of rapamycin (mTOR) and Toll-like receptor 4 (TLR4) signaling pathways were determined. Without LPS, IPEC-1 cells exhibited time- and Arg-dependent growth curves. LPS treatment increased cell death and reduced protein concentrations in IPEC-1 cells. Addition of 100 and 350 μM Arg to culture medium dose-dependently attenuated LPS-induced cell death and reduction of protein concentrations, in comparison with the basal medium containing 10 μM Arg. Furthermore, supplementation of 100 and 350 μM Arg increased protein synthesis and reduced protein degradation in both control and LPS-treated IPEC-1 cells. Consistent with the data on cell growth and protein turnover, addition of 100 or 350 μM Arg to culture medium increased relative protein levels for phosphorylated mTOR and phosphorylated ribosomal protein S6 kinase-1, while reducing the relative levels of TLR4 and phosphorylated levels of nuclear factor-κB in LPS-treated IPEC-1 cells. These results demonstrate a protective effect of Arg against LPS-induced enterocyte damage through mechanisms involving mTOR and TLR4 signaling pathways, as well as intracellular protein turnover

MAP4K3 Is a Component of the TORC1 Signalling Complex that Modulates Cell Growth and Viability in Drosophila melanogaster

Background: MAP4K3 is a conserved Ser/Thr kinase that has being found in connection with several signalling pathways, including the Imd, EGFR, TORC1 and JNK modules, in different organisms and experimental assays. We have analyzed the consequences of changing the levels of MAP4K3 expression in the development of the Drosophila wing, a convenient model system to characterize gene function during epithelial development. Methodology and Principal Findings: Using loss-of-function mutants and over-expression conditions we find that MAP4K3 activity affects cell growth and viability in the Drosophila wing. These requirements are related to the modulation of the TORC1 and JNK signalling pathways, and are best detected when the larvae grow in a medium with low protein concentration (TORC1) or are exposed to irradiation (JNK). We also show that MAP4K3 display strong genetic interactions with different components of the InR/Tor signalling pathway, and can interact directly with the GTPases RagA and RagC and with the multi-domain kinase Tor. Conclusions and Significance: We suggest that MAP4K3 has two independent functions during wing development, one related to the activation of the JNK pathway in response to stress and other in the assembling or activation of the TORC1 complex, being critical to modulate cellular responses to changes in nutrient availability

Conversion of t11t13 CLA into c9t11 CLA in Caco-2 Cells and Inhibition by Sterculic Oil

Background : Conjugated linoleic acids (CLA), and principally c9t11 CLA, are suspected to have numerous preventive properties regarding non-infectious pathologies such as inflammatory diseases, atherosclerosis and several types of cancer. C9t11 CLA is produced in the rumen during biohydrogenation of linoleic acid, but can also be synthesized in mammalian tissues from trans-vaccenic acid (C18:1 t11) through the action of delta-9 desaturase (D9D). For several years, it is also known that c9t11 CLA can be synthesized from conjugated linolenic acids (CLnA), i.e. c9t11c13 CLnA and c9t11t13 CLnA. This study aimed at investigating to which extent and by which route c9t11 CLA can be produced from another isomer of CLA, the t11t13 CLA that is structurally very similar to c9t11t13 CLnA, in Caco-2 cells

Cis-regulatory evolution spotlights species differences in the adaptive potential of gene expression plasticity

Plasticity allows organisms to respond to environmental change. Here the authors compare the distribution of cis-regulatory variants in the transcriptomes of Arabidopsis lyrata and A. halleriafter exposure to stress, to trace the role of polygenic selection in the evolution of gene expression plasticity

An Evolutionary Conserved Role for Anaplastic Lymphoma Kinase in Behavioral Responses to Ethanol

Anaplastic lymphoma kinase (Alk) is a gene expressed in the nervous system that encodes a receptor tyrosine kinase commonly known for its oncogenic function in various human cancers. We have determined that Alk is associated with altered behavioral responses to ethanol in the fruit fly Drosophila melanogaster, in mice, and in humans. Mutant flies containing transposon insertions in dAlk demonstrate increased resistance to the sedating effect of ethanol. Database analyses revealed that Alk expression levels in the brains of recombinant inbred mice are negatively correlated with ethanol-induced ataxia and ethanol consumption. We therefore tested Alk gene knockout mice and found that they sedate longer in response to high doses of ethanol and consume more ethanol than wild-type mice. Finally, sequencing of human ALK led to the discovery of four polymorphisms associated with a low level of response to ethanol, an intermediate phenotype that is predictive of future alcohol use disorders (AUDs). These results suggest that Alk plays an evolutionary conserved role in ethanol-related behaviors. Moreover, ALK may be a novel candidate gene conferring risk for AUDs as well as a potential target for pharmacological intervention

The Germinal Center Kinase GCK-1 Is a Negative Regulator of MAP Kinase Activation and Apoptosis in the C. elegans Germline

The germinal center kinases (GCK) constitute a large, highly conserved family of proteins that has been implicated in a wide variety of cellular processes including cell growth and proliferation, polarity, migration, and stress responses. Although diverse, these functions have been attributed to an evolutionarily conserved role for GCKs in the activation of ERK, JNK, and p38 MAP kinase pathways. In addition, multiple GCKs from different species promote apoptotic cell death. In contrast to these paradigms, we found that a C. elegans GCK, GCK-1, functions to inhibit MAP kinase activation and apoptosis in the C. elegans germline. In the absence of GCK-1, a specific MAP kinase isoform is ectopically activated and oocytes undergo abnormal development. Moreover, GCK-1- deficient animals display a significant increase in germ cell death. Our results suggest that individual germinal center kinases act in mechanistically distinct ways and that these functions are likely to depend on organ- and developmental-specific contexts

Down-Regulation of Honey Bee IRS Gene Biases Behavior toward Food Rich in Protein

Food choice and eating behavior affect health and longevity. Large-scale research efforts aim to understand the molecular and social/behavioral mechanisms of energy homeostasis, body weight, and food intake. Honey bees (Apis mellifera) could provide a model for these studies since individuals vary in food-related behavior and social factors can be controlled. Here, we examine a potential role of peripheral insulin receptor substrate (IRS) expression in honey bee foraging behavior. IRS is central to cellular nutrient sensing through transduction of insulin/insulin-like signals (IIS). By reducing peripheral IRS gene expression and IRS protein amount with the use of RNA interference (RNAi), we demonstrate that IRS influences foraging choice in two standard strains selected for different food-hoarding behavior. Compared with controls, IRS knockdowns bias their foraging effort toward protein (pollen) rather than toward carbohydrate (nectar) sources. Through control experiments, we establish that IRS does not influence the bees' sucrose sensory response, a modality that is generally associated with food-related behavior and specifically correlated with the foraging preference of honey bees. These results reveal a new affector pathway of honey bee social foraging, and suggest that IRS expressed in peripheral tissue can modulate an insect's foraging choice between protein and carbohydrate sources