Get It in Writing: Formal Contracts Mitigate Social Dilemmas in Multi-Agent RL

Multi-agent reinforcement learning (MARL) is a powerful tool for training automated systems acting independently in a common environment. However, it can lead to sub-optimal behavior when individual incentives and group incentives diverge. Humans are remarkably capable at solving these social dilemmas. It is an open problem in MARL to replicate such cooperative behaviors in selfish agents. In this work, we draw upon the idea of formal contracting from economics to overcome diverging incentives between agents in MARL. We propose an augmentation to a Markov game where agents voluntarily agree to binding state-dependent transfers of reward, under pre-specified conditions. Our contributions are theoretical and empirical. First, we show that this augmentation makes all subgame-perfect equilibria of all fully observed Markov games exhibit socially optimal behavior, given a sufficiently rich space of contracts. Next, we complement our game-theoretic analysis by showing that state-of-the-art RL algorithms learn socially optimal policies given our augmentation. Our experiments include classic static dilemmas like Stag Hunt, Prisoner's Dilemma and a public goods game, as well as dynamic interactions that simulate traffic, pollution management and common pool resource management.Comment: 12 pages, 8 figures, AAMAS 202

Stochastic optimization of a cold atom experiment using a genetic algorithm

We employ an evolutionary algorithm to automatically optimize different stages of a cold atom experiment without human intervention. This approach closes the loop between computer based experimental control systems and automatic real time analysis and can be applied to a wide range of experimental situations. The genetic algorithm quickly and reliably converges to the most performing parameter set independent of the starting population. Especially in many-dimensional or connected parameter spaces the automatic optimization outperforms a manual search.Comment: 4 pages, 3 figure

A Quantitative Theory of Mechanical Unfolding of a Homopolymer Globule

We propose the quantitative mean-field theory of mechanical unfolding of a globule formed by long flexible homopolymer chain collapsed in poor solvent and subjected to extensional deformation. We demonstrate that depending on the degree of polymerization and solvent quality (quantified by the Flory-Huggins $\chi$ parameter) the mechanical unfolding of the collapsed chain may either occur continuously (by passing a sequence of uniformly elongated configurations) or involves intra-molecular micro-phase coexistence of a collapsed and a stretched segment followed by an abrupt unraveling transition. The force-extension curves are obtained and quantitatively compared to our recent results of numerical self-consistent field (SCF) simulations. The phase diagrams for extended homopolymer chains in poor solvent comprising one- and two-phase regions are calculated for different chain length or/and solvent quality.Comment: 24 pages, 18 figure

Topotecan-vincristine-doxorubicin in stage 4 high risk neuroblastoma patients failing to achieve a complete metastatic response to rapid COJEC : a SIOPEN study

Purpose : Metastatic response to induction therapy for high-risk neuroblastoma is a prognostic factor. In the International Society of Paediatric Oncology Europe Neuroblastoma (SIOPEN) HR-NBL-1 protocol, only patients with metastatic complete response (CR) or partial response (PR) with <= three abnormal skeletal areas on iodine 123-metaiodobenzylguanidine ([I-123] mIBG) scintigraphy and no bone marrow disease proceed to high dose therapy (HDT). In this study, topotecan-vincristine-doxorubicin (TVD) was evaluated in patients failing to achieve these criteria, with the aim of improving the metastatic response rate. Materials and Methods : Patients with metastatic high-risk neuroblastoma who had not achieved the SIOPEN criteria for HDT after induction received two courses of topotecan 1.5 mg/m(2)/day for 5 days, followed by a 48-hour infusion of vincristine, 2 mg/m(2), and doxorubicin, 45 mg/m(2). Results : Sixty-three patients were eligible and evaluable. Following two courses of TVD, four (6.4%) patients had an overall CR, while 28 (44.4%) had a PR with a combined response rate of 50.8% (95% confidence interval [CI], 37.9 to 63.6). Of these, 23 patients achieved a metastatic CR or a PR with <= 3 mIBG skeletal areas and no bone marrow disease (36.5%; 95% CI, 24.7 to 49.6) and were eligible to receive HDT. Toxicity was mostly haematological, affecting 106 of the 126 courses (84.1%; 95% CI, 76.5 to 90.0), and dose reduction was necessary in six patients. Stomatitis was the second most common nonhematological toxicity, occurring in 20 patients (31.7%). Conclusion : TVD was effective in improving the response rate of high-risk neuroblastoma patients after induction with COJEC enabling them to proceed to HDT. However, the long-term benefits of TVD needs to be determined in randomized clinical trials

Constant-angle surfaces in liquid crystals

We discuss some properties of surfaces in R3 whose unit normal has constant angle with an assigned direction field. The constant angle condition can be rewritten as an Hamilton-Jacobi equation correlating the surface and the direction field. We focus on examples motivated by the physics of interfaces in liquid crystals and of layered fluids, and discuss the properties of the constant-angle surfaces when the direction field is singular along a line (disclination) or at a point (hedgehog defect

Icebergs in the North Atlantic: Modelling circulation changes and glacio-marine deposition

In order to investigate meltwater events in the North Atlantic, a simple iceberg generation, drift, and melting routine was implemented in a high-resolution OGCM. Starting from the modelled last glacial state, every 25th day cylindrical model icebergs 300 meters high were released at 32 specific points along the coasts. Icebergs launched at the Barents Shelf margin spread a light meltwater lid over the Norwegian and Greenland Seas, shutting down the deep convection and the anti-clockwise circulation in this area. Due to the constraining ocean circulation, the icebergs produce a tongue of relatively cold and fresh water extending eastward from Hudson Strait that must develop at this location, regardless of iceberg origin. From the total amount of freshwater inferred by the icebergs, the thickness of the deposited IRD could be calculated in dependance of iceberg sediment concentration. In this way, typical extent and thickness of Heinrich layers could be reproduced, running the model for 250 years of steady state with constant iceberg meltwater inflow

Oppida, agglomerations and suburbia: The Bibracte environs and new perspectives on Late Iron Age urbanism in central-eastern France

This paper explores the nature and chronology of La Tène and early Roman unenclosed agglomerations in central-eastern France. It has been prompted by the discovery of a c. 115 ha La Tène D2b/Augustan (c. 50 bc to ad 15) site close to Bibracte in the Morvan, located around the source of the River Yonne. This complex provides a new perspective on the chronology and role of Late La Tène and early Roman unenclosed settlements, adding further complexity to the story of the development of Late La Tène oppida. It indicates that these ‘agglomerations’ followed remarkably varied chronological trajectories, raising important issues concerning the nature of landscape and social change at the end of the Iron Age

Investigating a genetic link between Alzheimer’s Disease and CADASIL related Cerebral Small Vessel Disease

Monogenic forms of Alzheimer’s disease (AD) have been identified through mutations in genes such as APP, PSEN1, and PSEN2, whilst other genetic markers such as the APOE ε carrier allele status have been shown to increase the likelihood of having the disease. Mutations in these genes are not limited to AD, as APP mutations can also cause an amyloid form of cerebral small vessel disease (CSVD) known as cerebral amyloid angiopathy, whilst PSEN1 and PSEN2 are involved in NOTCH3 signalling, a process known to be dysregulated in the monogenic CSVD, cerebral autosomal dominant arteriopathy with subcortical infarcts and leukoencephalopathy (CADASIL). The overlap between AD genes and causes of CSVD led to the hypothesis that mutations in other genes within the PANTHER AD–presenilin pathway may be novel causes of CSVD in a cohort of clinically suspicious CADASIL patients without a pathogenic NOTCH3 mutation. To investigate this, whole exome sequencing was performed on 50 suspected CADASIL patients with no NOTCH3 mutations, and a targeted gene analysis was completed on the PANTHER. ERN1 was identified as a novel candidate CSVD gene following predicted pathogenic gene mutation analysis. Rare variant burden testing failed to identify an association with any gene; however, it did show a nominally significant link with ERN1 and TRPC3. This study provides evidence to support a genetic overlap between CSVD and Alzheimer’s disease.</p

Far-infrared absorption in parallel quantum wires with weak tunneling

We study collective and single-particle intersubband excitations in a system of quantum wires coupled via weak tunneling. For an isolated wire with parabolic confinement, the Kohn's theorem guarantees that the absorption spectrum represents a single sharp peak centered at the frequency given by the bare confining potential. We show that the effect of weak tunneling between two parabolic quantum wires is twofold: (i) additional peaks corresponding to single-particle excitations appear in the absorption spectrum, and (ii) the main absorption peak acquires a depolarization shift. We also show that the interplay between tunneling and weak perpendicular magnetic field drastically enhances the dispersion of single-particle excitations. The latter leads to a strong damping of the intersubband plasmon for magnetic fields exceeding a critical value.Comment: 18 pages + 6 postcript figure