Scale-free memory model for multiagent reinforcement learning. Mean field approximation and rock-paper-scissors dynamics

A continuous time model for multiagent systems governed by reinforcement learning with scale-free memory is developed. The agents are assumed to act independently of one another in optimizing their choice of possible actions via trial-and-error search. To gain awareness about the action value the agents accumulate in their memory the rewards obtained from taking a specific action at each moment of time. The contribution of the rewards in the past to the agent current perception of action value is described by an integral operator with a power-law kernel. Finally a fractional differential equation governing the system dynamics is obtained. The agents are considered to interact with one another implicitly via the reward of one agent depending on the choice of the other agents. The pairwise interaction model is adopted to describe this effect. As a specific example of systems with non-transitive interactions, a two agent and three agent systems of the rock-paper-scissors type are analyzed in detail, including the stability analysis and numerical simulation. Scale-free memory is demonstrated to cause complex dynamics of the systems at hand. In particular, it is shown that there can be simultaneously two modes of the system instability undergoing subcritical and supercritical bifurcation, with the latter one exhibiting anomalous oscillations with the amplitude and period growing with time. Besides, the instability onset via this supercritical mode may be regarded as "altruism self-organization". For the three agent system the instability dynamics is found to be rather irregular and can be composed of alternate fragments of oscillations different in their properties.Comment: 17 pages, 7 figur

Future response of global coastal wetlands to sea-level rise.

The response of coastal wetlands to sea-level rise during the twenty-first century remains uncertain. Global-scale projections suggest that between 20 and 90 per cent (for low and high sea-level rise scenarios, respectively) of the present-day coastal wetland area will be lost, which will in turn result in the loss of biodiversity and highly valued ecosystem services1-3. These projections do not necessarily take into account all essential geomorphological4-7 and socio-economic system feedbacks8. Here we present an integrated global modelling approach that considers both the ability of coastal wetlands to build up vertically by sediment accretion, and the accommodation space, namely, the vertical and lateral space available for fine sediments to accumulate and be colonized by wetland vegetation. We use this approach to assess global-scale changes in coastal wetland area in response to global sea-level rise and anthropogenic coastal occupation during the twenty-first century. On the basis of our simulations, we find that, globally, rather than losses, wetland gains of up to 60 per cent of the current area are possible, if more than 37 per cent (our upper estimate for current accommodation space) of coastal wetlands have sufficient accommodation space, and sediment supply remains at present levels. In contrast to previous studies1-3, we project that until 2100, the loss of global coastal wetland area will range between 0 and 30 per cent, assuming no further accommodation space in addition to current levels. Our simulations suggest that the resilience of global wetlands is primarily driven by the availability of accommodation space, which is strongly influenced by the building of anthropogenic infrastructure in the coastal zone and such infrastructure is expected to change over the twenty-first century. Rather than being an inevitable consequence of global sea-level rise, our findings indicate that large-scale loss of coastal wetlands might be avoidable, if sufficient additional accommodation space can be created through careful nature-based adaptation solutions to coastal management.Personal research fellowship of Mark Schuerch (Project Number 272052902) and by the Cambridge Coastal Research Unit (Visiting Scholar Programme). Furthermore, this work has partly been supported by the EU research project RISES-AM- (FP7-ENV-693396)

Dynamics of Cough Frequency in Adults Undergoing Treatment for Pulmonary Tuberculosis.

Background: Cough is the major determinant of tuberculosis transmission. Despite this, there is a paucity of information regarding characteristics of cough frequency throughout the day and in response to tuberculosis therapy. Here we evaluate the circadian cycle of cough, cough frequency risk factors, and the impact of appropriate treatment on cough and bacillary load. Methods: We prospectively evaluated human immunodeficiency virus-negative adults (n = 64) with a new diagnosis of culture-proven, drug-susceptible pulmonary tuberculosis immediately prior to treatment and repeatedly until treatment day 62. At each time point, participant cough was recorded (n = 670) and analyzed using the Cayetano Cough Monitor. Consecutive coughs at least 2 seconds apart were counted as separate cough episodes. Sputum samples (n = 426) were tested with microscopic-observation drug susceptibility broth culture, and in culture-positive samples (n = 252), the time to culture positivity was used to estimate bacillary load. Results: The highest cough frequency occurred from 1 pm to 2 pm, and the lowest from 1 am to 2 am (2.4 vs 1.1 cough episodes/hour, respectively). Cough frequency was higher among participants who had higher sputum bacillary load (P < .01). Pretreatment median cough episodes/hour was 2.3 (interquartile range [IQR], 1.2-4.1), which at 14 treatment days decreased to 0.48 (IQR, 0.0-1.4) and at the end of the study decreased to 0.18 (IQR, 0.0-0.59) (both reductions P < .001). By 14 treatment days, the probability of culture conversion was 29% (95% confidence interval, 19%-41%). Conclusions: Coughs were most frequent during daytime. Two weeks of appropriate treatment significantly reduced cough frequency and resulted in one-third of participants achieving culture conversion. Thus, treatment by 2 weeks considerably diminishes, but does not eliminate, the potential for airborne tuberculosis transmission

Unbounded violation of tripartite Bell inequalities

We prove that there are tripartite quantum states (constructed from random unitaries) that can lead to arbitrarily large violations of Bell inequalities for dichotomic observables. As a consequence these states can withstand an arbitrary amount of white noise before they admit a description within a local hidden variable model. This is in sharp contrast with the bipartite case, where all violations are bounded by Grothendieck's constant. We will discuss the possibility of determining the Hilbert space dimension from the obtained violation and comment on implications for communication complexity theory. Moreover, we show that the violation obtained from generalized GHZ states is always bounded so that, in contrast to many other contexts, GHZ states do in this case not lead to extremal quantum correlations. The results are based on tools from the theories of operator spaces and tensor norms which we exploit to prove the existence of bounded but not completely bounded trilinear forms from commutative C*-algebras.Comment: Substantial changes in the presentation to make the paper more accessible for a non-specialized reade

The OMERACT-OARSI Core Domain Set for Measurement in Clinical Trials of Hip and/or Knee Osteoarthritis

Objective: To update the 1997 OMERACT-OARSI (Outcome Measures in Rheumatology-Osteoarthritis Research Society International) core domain set for clinical trials in hip and/or knee osteoarthritis (OA). Methods: An initial review of the COMET database of core outcome sets (COS) was undertaken to identify all domains reported in previous COS including individuals with hip and/or knee OA. These were presented during 5 patient and health professionals/researcher meetings in 3 continents (Europe, Australasia, North America). A 3-round international Delphi survey was then undertaken among patients, healthcare professionals, researchers, and industry representatives to gain consensus on key domains to be included in a core domain set for hip and/or knee OA. Findings were presented and discussed in small groups at OMERACT 2018, where consensus was obtained in the final plenary. Results: Four previous COS were identified. Using these, and the patient and health professionals/researcher meetings, 50 potential domains formed the Delphi survey. There were 426 individuals from 25 different countries who contributed to the Delphi exercise. OMERACT 2018 delegates (n = 129) voted on candidate domains. Six domains gained agreement as mandatory to be measured and reported in all hip and/or knee OA clinical trials: pain, physical function, quality of life, and patient’s global assessment of the target joint, in addition to the mandated core domain of adverse events including mortality. Joint structure was agreed as mandatory in specific circumstances, i.e., depending on the intervention. Conclusion: The updated core domain set for hip and/or knee OA has been agreed upon. Work will commence to determine which outcome measurement instrument should be recommended to cover each core domain

Integrating education for sustainable development into a higher education institution: beginning the journey

Much of the current literature on integrating sustainability into HEIs is focussed on why HEIs should embrace sustainable development (SD) and what is still missing or hindering work and the integration of efforts. There is much less exploration of how SD has been interpreted at the individual HEI level and action taken as a result. This case study reflects on important elements of the journey Nottingham Trent University (NTU) in the UK has taken to integrate sustainability, focussing on key decisions and activity in 2009/10. In highlighting this, the authors seek to empower those looking to support and/or lead the embedding of Education for Sustainable Development (ESD), separately or as part of an integrated effort, in their own institution. Today in 2019, NTU is a global leader in integrating ESD as part of a wider SD agenda. The work which this paper presents, to understand and establish a baseline of key elements of NTU’s existing ESD activity and systems, was an important turning point. Activities undertaken to review and assess ‘where are we now?’, primarily through an institution-wide survey in 2009/10, led to important insights and supported dialogue, as well as the connection and underpinning of core administrative elements of the NTU SD framework and systems. Further recommendations are given in the final section of this paper on other drivers that can help to embed ESD within an HEI

Biofilm Development on Caenorhabditis elegans by Yersinia Is Facilitated by Quorum Sensing-Dependent Repression of Type III Secretion

Yersinia pseudotuberculosis forms biofilms on Caenorhabditis elegans which block nematode feeding. This genetically amenable host-pathogen model has important implications for biofilm development on living, motile surfaces. Here we show that Y. pseudotuberculosis biofilm development on C. elegans is governed by N-acylhomoserine lactone (AHL)-mediated quorum sensing (QS) since (i) AHLs are produced in nematode associated biofilms and (ii) Y. pseudotuberculosis strains expressing an AHL-degrading enzyme or in which the AHL synthase (ypsI and ytbI) or response regulator (ypsR and ytbR) genes have been mutated, are attenuated. Although biofilm formation is also attenuated in Y. pseudotuberculosis strains carrying mutations in the QS-controlled motility regulator genes, flhDC and fliA, and the flagellin export gene, flhA, flagella are not required since fliC mutants form normal biofilms. However, in contrast to the parent and fliC mutant, Yop virulon proteins are up-regulated in flhDC, fliA and flhA mutants in a temperature and calcium independent manner. Similar observations were found for the Y. pseudotuberculosis QS mutants, indicating that the Yop virulon is repressed by QS via the master motility regulator, flhDC. By curing the pYV virulence plasmid from the ypsI/ytbI mutant, by growing YpIII under conditions permissive for type III needle formation but not Yop secretion and by mutating the type III secretion apparatus gene, yscJ, we show that biofilm formation can be restored in flhDC and ypsI/ytbI mutants. These data demonstrate that type III secretion blocks biofilm formation and is reciprocally regulated with motility via QS

Erratum to: EuPRAXIA Conceptual Design Report – Eur. Phys. J. Special Topics 229, 3675-4284 (2020), https://doi.org/10.1140/epjst/e2020-000127-8

International audienceThe online version of the original article can be found at http://https://doi.org/10.1140/epjst/e2020-000127-8</A