On the Use and Misuse of Absorbing States in Multi-agent Reinforcement Learning

The creation and destruction of agents in cooperative multi-agent reinforcement learning (MARL) is a critically under-explored area of research. Current MARL algorithms often assume that the number of agents within a group remains fixed throughout an experiment. However, in many practical problems, an agent may terminate before their teammates. This early termination issue presents a challenge: the terminated agent must learn from the group's success or failure which occurs beyond its own existence. We refer to propagating value from rewards earned by remaining teammates to terminated agents as the Posthumous Credit Assignment problem. Current MARL methods handle this problem by placing these agents in an absorbing state until the entire group of agents reaches a termination condition. Although absorbing states enable existing algorithms and APIs to handle terminated agents without modification, practical training efficiency and resource use problems exist. In this work, we first demonstrate that sample complexity increases with the quantity of absorbing states in a toy supervised learning task for a fully connected network, while attention is more robust to variable size input. Then, we present a novel architecture for an existing state-of-the-art MARL algorithm which uses attention instead of a fully connected layer with absorbing states. Finally, we demonstrate that this novel architecture significantly outperforms the standard architecture on tasks in which agents are created or destroyed within episodes as well as standard multi-agent coordination tasks.Comment: RL in Games Workshop AAAI 202

A screen for downstream effectors of Neurogenin2 in the embryonic neocortex

Neurogenin (Ngn) 1 and Ngn2 encode basic-helix-loop-helix transcription factors expressed in the developing neocortex. Like other proneural genes, Ngns participate in the specification of neural fates and neuronal identities, but downstream effectors remain poorly defined. We set out to identify Ngn2 effectors in the cortex using a subtractive hybridization screen and identified several regionally expressed genes that were misregulated in Ngn2 and Ngn1; Ngn2 mutants. Included were genes down-regulated in germinal zone progenitors (e.g., Nlgn1, Unc5H4, and Dcc) and in postmitotic neurons in the cortical plate (e.g., Bhlhb5 and NFIB) and subplate (e.g., Mef2c, srGAP3, and protocadherin 9). Further analysis revealed that Ngn2 mutant subplate neurons were misspecified and that thalamocortical afferents (TCAs) that normally target this layer instead inappropriately projected towards the germinal zone. Strikingly, EphA5 and Sema3c, which encode repulsive guidance cues, were down-regulated in the Ngn2 and Ngn1; Ngn2 mutant germinal zones, providing a possible molecular basis for axonal targeting defects. Thus, we identified several new components of the differentiation cascade(s) activated downstream of Ngn1 and Ngn2 and provided novel insights into a new developmental process controlled by these proneural genes. Further analysis of the genes isolated in our screen should provide a fertile basis for understanding the molecular mechanisms underlying corticogenesis

GliaMorph: A modular image analysis toolkit to quantify Müller glial cell morphology

Cell morphology is critical for all cell functions. This is particularly true for glial cells as they rely on complex shape to contact and support neurons. However, methods to quantify complex glial cell shape accurately and reproducibly are lacking. To address this, we developed the image analysis pipeline "GliaMorph". GliaMorph is a modular analysis toolkit developed to perform (i) image pre-processing, (ii) semi-automatic region-of-interest (ROI) selection, (iii) apicobasal texture analysis, (iv) glia segmentation, and (v) cell feature quantification. Müller Glia (MG) have a stereotypic shape linked to their maturation and physiological status. We here characterized MG on three levels, including (a) global image-level, (b) apicobasal texture, and (c) regional apicobasal vertical-to-horizontal alignment. Using GliaMorph we quantified MG development on a global and single-cell level, showing increased feature elaboration and subcellular morphological rearrangement in the zebrafish retina. As proof-of-principle, we analysed expression changes in a mouse glaucoma model, identifying subcellular protein localization changes in MG. Together, GliaMorph enables an in-depth understanding of MG morphology in the developing and diseased retina

Antimicrobial resistance among migrants in Europe: a systematic review and meta-analysis

BACKGROUND: Rates of antimicrobial resistance (AMR) are rising globally and there is concern that increased migration is contributing to the burden of antibiotic resistance in Europe. However, the effect of migration on the burden of AMR in Europe has not yet been comprehensively examined. Therefore, we did a systematic review and meta-analysis to identify and synthesise data for AMR carriage or infection in migrants to Europe to examine differences in patterns of AMR across migrant groups and in different settings. METHODS: For this systematic review and meta-analysis, we searched MEDLINE, Embase, PubMed, and Scopus with no language restrictions from Jan 1, 2000, to Jan 18, 2017, for primary data from observational studies reporting antibacterial resistance in common bacterial pathogens among migrants to 21 European Union-15 and European Economic Area countries. To be eligible for inclusion, studies had to report data on carriage or infection with laboratory-confirmed antibiotic-resistant organisms in migrant populations. We extracted data from eligible studies and assessed quality using piloted, standardised forms. We did not examine drug resistance in tuberculosis and excluded articles solely reporting on this parameter. We also excluded articles in which migrant status was determined by ethnicity, country of birth of participants' parents, or was not defined, and articles in which data were not disaggregated by migrant status. Outcomes were carriage of or infection with antibiotic-resistant organisms. We used random-effects models to calculate the pooled prevalence of each outcome. The study protocol is registered with PROSPERO, number CRD42016043681. FINDINGS: We identified 2274 articles, of which 23 observational studies reporting on antibiotic resistance in 2319 migrants were included. The pooled prevalence of any AMR carriage or AMR infection in migrants was 25·4% (95% CI 19·1-31·8; I2 =98%), including meticillin-resistant Staphylococcus aureus (7·8%, 4·8-10·7; I2 =92%) and antibiotic-resistant Gram-negative bacteria (27·2%, 17·6-36·8; I2 =94%). The pooled prevalence of any AMR carriage or infection was higher in refugees and asylum seekers (33·0%, 18·3-47·6; I2 =98%) than in other migrant groups (6·6%, 1·8-11·3; I2 =92%). The pooled prevalence of antibiotic-resistant organisms was slightly higher in high-migrant community settings (33·1%, 11·1-55·1; I2 =96%) than in migrants in hospitals (24·3%, 16·1-32·6; I2 =98%). We did not find evidence of high rates of transmission of AMR from migrant to host populations. INTERPRETATION: Migrants are exposed to conditions favouring the emergence of drug resistance during transit and in host countries in Europe. Increased antibiotic resistance among refugees and asylum seekers and in high-migrant community settings (such as refugee camps and detention facilities) highlights the need for improved living conditions, access to health care, and initiatives to facilitate detection of and appropriate high-quality treatment for antibiotic-resistant infections during transit and in host countries. Protocols for the prevention and control of infection and for antibiotic surveillance need to be integrated in all aspects of health care, which should be accessible for all migrant groups, and should target determinants of AMR before, during, and after migration. FUNDING: UK National Institute for Health Research Imperial Biomedical Research Centre, Imperial College Healthcare Charity, the Wellcome Trust, and UK National Institute for Health Research Health Protection Research Unit in Healthcare-associated Infections and Antimictobial Resistance at Imperial College London

Neuromatch Academy: a 3-week, online summer school in computational neuroscience

Neuromatch Academy (https://academy.neuromatch.io; (van Viegen et al., 2021)) was designed as an online summer school to cover the basics of computational neuroscience in three weeks. The materials cover dominant and emerging computational neuroscience tools, how they complement one another, and specifically focus on how they can help us to better understand how the brain functions. An original component of the materials is its focus on modeling choices, i.e. how do we choose the right approach, how do we build models, and how can we evaluate models to determine if they provide real (meaningful) insight. This meta-modeling component of the instructional materials asks what questions can be answered by different techniques, and how to apply them meaningfully to get insight about brain function