Scale-free memory model for multiagent reinforcement learning. Mean field approximation and rock-paper-scissors dynamics

A continuous time model for multiagent systems governed by reinforcement learning with scale-free memory is developed. The agents are assumed to act independently of one another in optimizing their choice of possible actions via trial-and-error search. To gain awareness about the action value the agents accumulate in their memory the rewards obtained from taking a specific action at each moment of time. The contribution of the rewards in the past to the agent current perception of action value is described by an integral operator with a power-law kernel. Finally a fractional differential equation governing the system dynamics is obtained. The agents are considered to interact with one another implicitly via the reward of one agent depending on the choice of the other agents. The pairwise interaction model is adopted to describe this effect. As a specific example of systems with non-transitive interactions, a two agent and three agent systems of the rock-paper-scissors type are analyzed in detail, including the stability analysis and numerical simulation. Scale-free memory is demonstrated to cause complex dynamics of the systems at hand. In particular, it is shown that there can be simultaneously two modes of the system instability undergoing subcritical and supercritical bifurcation, with the latter one exhibiting anomalous oscillations with the amplitude and period growing with time. Besides, the instability onset via this supercritical mode may be regarded as "altruism self-organization". For the three agent system the instability dynamics is found to be rather irregular and can be composed of alternate fragments of oscillations different in their properties.Comment: 17 pages, 7 figur

Statistical Theory of Spin Relaxation and Diffusion in Solids

A comprehensive theoretical description is given for the spin relaxation and diffusion in solids. The formulation is made in a general statistical-mechanical way. The method of the nonequilibrium statistical operator (NSO) developed by D. N. Zubarev is employed to analyze a relaxation dynamics of a spin subsystem. Perturbation of this subsystem in solids may produce a nonequilibrium state which is then relaxed to an equilibrium state due to the interaction between the particles or with a thermal bath (lattice). The generalized kinetic equations were derived previously for a system weakly coupled to a thermal bath to elucidate the nature of transport and relaxation processes. In this paper, these results are used to describe the relaxation and diffusion of nuclear spins in solids. The aim is to formulate a successive and coherent microscopic description of the nuclear magnetic relaxation and diffusion in solids. The nuclear spin-lattice relaxation is considered and the Gorter relation is derived. As an example, a theory of spin diffusion of the nuclear magnetic moment in dilute alloys (like Cu-Mn) is developed. It is shown that due to the dipolar interaction between host nuclear spins and impurity spins, a nonuniform distribution in the host nuclear spin system will occur and consequently the macroscopic relaxation time will be strongly determined by the spin diffusion. The explicit expressions for the relaxation time in certain physically relevant cases are given.Comment: 41 pages, 119 Refs. Corrected typos, added reference

A function-based typology for Earth’s ecosystems

As the United Nations develops a post-2020 global biodiversity framework for the Convention on Biological Diversity, attention is focusing on how new goals and targets for ecosystem conservation might serve its vision of ‘living in harmony with nature’1,2. Advancing dual imperatives to conserve biodiversity and sustain ecosystem services requires reliable and resilient generalizations and predictions about ecosystem responses to environmental change and management3. Ecosystems vary in their biota4, service provision5 and relative exposure to risks6, yet there is no globally consistent classification of ecosystems that reflects functional responses to change and management. This hampers progress on developing conservation targets and sustainability goals. Here we present the International Union for Conservation of Nature (IUCN) Global Ecosystem Typology, a conceptually robust, scalable, spatially explicit approach for generalizations and predictions about functions, biota, risks and management remedies across the entire biosphere. The outcome of a major cross-disciplinary collaboration, this novel framework places all of Earth’s ecosystems into a unifying theoretical context to guide the transformation of ecosystem policy and management from global to local scales. This new information infrastructure will support knowledge transfer for ecosystem-specific management and restoration, globally standardized ecosystem risk assessments, natural capital accounting and progress on the post-2020 global biodiversity framework

Risk factors for SARS-CoV-2 seroprevalence following the first pandemic wave in UK healthcare workers in a large NHS Foundation Trust [version 1; peer review: awaiting peer review]

Background: We aimed to measure SARS-CoV-2 seroprevalence in a cohort of healthcare workers (HCWs) during the first UK wave of the COVID-19 pandemic, explore risk factors associated with infection, and investigate the impact of antibody titres on assay sensitivity. Methods: HCWs at Sheffield Teaching Hospitals NHS Foundation Trust were prospectively enrolled and sampled at two time points. SARS-CoV-2 antibodies were tested using an in-house assay for IgG and IgA reactivity against Spike and Nucleoprotein (sensitivity 99·47%, specificity 99·56%). Data were analysed using three statistical models: a seroprevalence model, an antibody kinetics model, and a heterogeneous sensitivity model. Results: As of 12th June 2020, 24·4% (n=311/1275) of HCWs were seropositive. Of these, 39·2% (n=122/311) were asymptomatic. The highest adjusted seroprevalence was measured in HCWs on the Acute Medical Unit (41·1%, 95% CrI 30·0–52·9) and in Physiotherapists and Occupational Therapists (39·2%, 95% CrI 24·4–56·5). Older age groups showed overall higher median antibody titres. Further modelling suggests that, for a serological assay with an overall sensitivity of 80%, antibody titres may be markedly affected by differences in age, with sensitivity estimates of 89% in those over 60 years but 61% in those ≤30 years. Conclusions: HCWs in acute medical units working closely with COVID-19 patients were at highest risk of infection, though whether these are infections acquired from patients or other staff is unknown. Current serological assays may underestimate seroprevalence in younger age groups if validated using sera from older and/or more symptomatic individuals

Implementable deep learning for multi-sequence proton MRI lung segmentation: a multi-center, multi-vendor, and multi-disease study

Background Recently, deep learning via convolutional neural networks (CNNs) has largely superseded conventional methods for proton (1H)-MRI lung segmentation. However, previous deep learning studies have utilized single-center data and limited acquisition parameters. Purpose Develop a generalizable CNN for lung segmentation in 1H-MRI, robust to pathology, acquisition protocol, vendor, and center. Study type Retrospective. Population A total of 809 1H-MRI scans from 258 participants with various pulmonary pathologies (median age (range): 57 (6–85); 42% females) and 31 healthy participants (median age (range): 34 (23–76); 34% females) that were split into training (593 scans (74%); 157 participants (55%)), testing (50 scans (6%); 50 participants (17%)) and external validation (164 scans (20%); 82 participants (28%)) sets. Field Strength/Sequence 1.5-T and 3-T/3D spoiled-gradient recalled and ultrashort echo-time 1H-MRI. Assessment 2D and 3D CNNs, trained on single-center, multi-sequence data, and the conventional spatial fuzzy c-means (SFCM) method were compared to manually delineated expert segmentations. Each method was validated on external data originating from several centers. Dice similarity coefficient (DSC), average boundary Hausdorff distance (Average HD), and relative error (XOR) metrics to assess segmentation performance. Statistical Tests Kruskal–Wallis tests assessed significances of differences between acquisitions in the testing set. Friedman tests with post hoc multiple comparisons assessed differences between the 2D CNN, 3D CNN, and SFCM. Bland–Altman analyses assessed agreement with manually derived lung volumes. A P value of <0.05 was considered statistically significant. Results The 3D CNN significantly outperformed its 2D analog and SFCM, yielding a median (range) DSC of 0.961 (0.880–0.987), Average HD of 1.63 mm (0.65–5.45) and XOR of 0.079 (0.025–0.240) on the testing set and a DSC of 0.973 (0.866–0.987), Average HD of 1.11 mm (0.47–8.13) and XOR of 0.054 (0.026–0.255) on external validation data. Data Conclusion The 3D CNN generated accurate 1H-MRI lung segmentations on a heterogenous dataset, demonstrating robustness to disease pathology, sequence, vendor, and center. Evidence Level 4. Technical Efficacy Stage 1

Sex Selective Abortion, Neoliberal Patriarchy and Structural Violence in India

This article explores sex selective abortion (SSA) as a form of structural violence within the broader notion of women’s ‘protection’ in contemporary India. While SSA tends to be framed more generally within ethical and choice-based frameworks around abortion access and reproductive ‘rights’, and specifically in India around preference for sons as a discriminatory, cultural, technological misogyny, this article argues that sex selective abortion in India needs to be understood as an outcome of broader systemic economic, political and social processes. The deepening of neoliberal values through state policies has impacted significantly on social relations, shaping SSA as a manifestation of structural violence. State-driven policies in India reflect a neoliberal governmentality through state patriarchy that is implicit within the neoliberal developmental, governmental and capitalist paradigm of contemporary India. This article argues that SSA is structurally produced and therefore cannot be remedied through awareness-raising strategies such as beti bachao or financial inclusion as a means to ‘protect’ or ‘save the girl child’. Indeed, it is neoliberal economic forces that actively, though seemingly inadvertently, promote anti-women, sex selective abortion as a reproductive strategy, which is then disciplined through neoliberal governmentality. This highlights SSA as a form of gendered and structural, rather than discriminatory, violence

A novel formulation of inhaled sodium cromoglicate (PA101) in idiopathic pulmonary fibrosis and chronic cough: a randomised, double-blind, proof-of-concept, phase 2 trial

Background Cough can be a debilitating symptom of idiopathic pulmonary fibrosis (IPF) and is difficult to treat. PA101 is a novel formulation of sodium cromoglicate delivered via a high-efficiency eFlow nebuliser that achieves significantly higher drug deposition in the lung compared with the existing formulations. We aimed to test the efficacy and safety of inhaled PA101 in patients with IPF and chronic cough and, to explore the antitussive mechanism of PA101, patients with chronic idiopathic cough (CIC) were also studied. Methods This pilot, proof-of-concept study consisted of a randomised, double-blind, placebo-controlled trial in patients with IPF and chronic cough and a parallel study of similar design in patients with CIC. Participants with IPF and chronic cough recruited from seven centres in the UK and the Netherlands were randomly assigned (1:1, using a computer-generated randomisation schedule) by site staff to receive PA101 (40 mg) or matching placebo three times a day via oral inhalation for 2 weeks, followed by a 2 week washout, and then crossed over to the other arm. Study participants, investigators, study staff, and the sponsor were masked to group assignment until all participants had completed the study. The primary efficacy endpoint was change from baseline in objective daytime cough frequency (from 24 h acoustic recording, Leicester Cough Monitor). The primary efficacy analysis included all participants who received at least one dose of study drug and had at least one post-baseline efficacy measurement. Safety analysis included all those who took at least one dose of study drug. In the second cohort, participants with CIC were randomly assigned in a study across four centres with similar design and endpoints. The study was registered with ClinicalTrials.gov (NCT02412020) and the EU Clinical Trials Register (EudraCT Number 2014-004025-40) and both cohorts are closed to new participants. Findings Between Feb 13, 2015, and Feb 2, 2016, 24 participants with IPF were randomly assigned to treatment groups. 28 participants with CIC were enrolled during the same period and 27 received study treatment. In patients with IPF, PA101 reduced daytime cough frequency by 31·1% at day 14 compared with placebo; daytime cough frequency decreased from a mean 55 (SD 55) coughs per h at baseline to 39 (29) coughs per h at day 14 following treatment with PA101, versus 51 (37) coughs per h at baseline to 52 (40) cough per h following placebo treatment (ratio of least-squares [LS] means 0·67, 95% CI 0·48–0·94, p=0·0241). By contrast, no treatment benefit for PA101 was observed in the CIC cohort; mean reduction of daytime cough frequency at day 14 for PA101 adjusted for placebo was 6·2% (ratio of LS means 1·27, 0·78–2·06, p=0·31). PA101 was well tolerated in both cohorts. The incidence of adverse events was similar between PA101 and placebo treatments, most adverse events were mild in severity, and no severe adverse events or serious adverse events were reported. Interpretation This study suggests that the mechanism of cough in IPF might be disease specific. Inhaled PA101 could be a treatment option for chronic cough in patients with IPF and warrants further investigation