MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning

This paper introduces MDP homomorphic networks for deep reinforcement learning. MDP homomorphic networks are neural networks that are equivariant under symmetries in the joint state-action space of an MDP. Current approaches to deep reinforcement learning do not usually exploit knowledge about such structure. By building this prior knowledge into policy and value networks using an equivariance constraint, we can reduce the size of the solution space. We specifically focus on group-structured symmetries (invertible transformations). Additionally, we introduce an easy method for constructing equivariant network layers numerically, so the system designer need not solve the constraints by hand, as is typically done. We construct MDP homomorphic MLPs and CNNs that are equivariant under either a group of reflections or rotations. We show that such networks converge faster than unstructured baselines on CartPole, a grid world and Pong

Immune monitoring and TCR sequencing of CD4 T cells in a long term responsive patient with metastasized pancreatic ductal carcinoma treated with individualized, neoepitope-derived multipeptide vaccines : a case report

Abstract Background Cancer vaccines can effectively establish clinically relevant tumor immunity. Novel sequencing approaches rapidly identify the mutational fingerprint of tumors, thus allowing to generate personalized tumor vaccines within a few weeks from diagnosis. Here, we report the case of a 62-year-old patient receiving a four-peptide-vaccine targeting the two sole mutations of his pancreatic tumor, identified via exome sequencing. Methods Vaccination started during chemotherapy in second complete remission and continued monthly thereafter. We tracked IFN-γ+ T cell responses against vaccine peptides in peripheral blood after 12, 17 and 34 vaccinations by analyzing T-cell receptor (TCR) repertoire diversity and epitope-binding regions of peptide-reactive T-cell lines and clones. By restricting analysis to sorted IFN-γ-producing T cells we could assure epitope-specificity, functionality, and TH1 polarization. Results A peptide-specific T-cell response against three of the four vaccine peptides could be detected sequentially. Molecular TCR analysis revealed a broad vaccine-reactive TCR repertoire with clones of discernible specificity. Four identical or convergent TCR sequences could be identified at more than one time-point, indicating timely persistence of vaccine-reactive T cells. One dominant TCR expressing a dual TCRVα chain could be found in three T-cell clones. The observed T-cell responses possibly contributed to clinical outcome: The patient is alive 6 years after initial diagnosis and in complete remission for 4 years now. Conclusions Therapeutic vaccination with a neoantigen-derived four-peptide vaccine resulted in a diverse and long-lasting immune response against these targets which was associated with prolonged clinical remission. These data warrant confirmation in a larger proof-of concept clinical trial

Untangling the dinosaur family tree

For over a century, the standard classification scheme has split dinosaurs into two fundamental groups: ‘lizard-hipped’ saurischians (including meat-eating theropods and long-necked sauropodomorphs) and ‘bird-hipped’ ornithischians (including a variety of herbivorous species).In a recent paper, Baron et al. challenged this paradigm with a new phylogenetic analysis that places theropods and ornithischians together in a group called Ornithoscelida, to the exclusion of sauropodomorphs, and used their phylogeny to argue that dinosaurs may have originated in northern Pangaea, not in the southern part of the supercontinent, as has more commonly been considered. Here we evaluate and reanalyse the morphological dataset underpinning the proposal by Baron et al. and provide quantitative biogeographic analyses, which challenge the key results of their study by recovering a classical monophyletic Saurischia and a Gondwanan origin for dinosaurs. This shows that the Ornithoscelida hypothesis is not the final word, and that there is still great uncertainty around the basic structure of the dinosaur family tree.Fil: Langer, Max C.. Universidade de Sao Paulo; BrasilFil: Ezcurra, Martin Daniel. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Parque Centenario. Museo Argentino de Ciencias Naturales “Bernardino Rivadavia”; ArgentinaFil: Rauhut, Oliver Walter Mischa. Ludwig Maximilians Universitat; AlemaniaFil: Benton, Michael J.. University of Bristol; Reino UnidoFil: Knoll, Fabien. University of Manchester; Reino UnidoFil: McPhee, Blair W.. Universidade de Sao Paulo; BrasilFil: Novas, Fernando Emilio. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Parque Centenario. Museo Argentino de Ciencias Naturales “Bernardino Rivadavia”; ArgentinaFil: Pol, Diego. Museo Paleontológico Egidio Feruglio; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Brusatte, Stephen L.. University of Edinburgh; Reino Unid

The development of a resource-efficient photovoltaic system

This paper presents the measures taken in the demonstration of the photovoltaic case study developed within the European project ‘Towards zero waste in industrial networks’ (Zerowin), integrating the D4R (Design for recycling, repair, refurbishment and reuse) criteria at both system and industrial network level. The demonstration is divided into three phases. The first phase concerns the development of a D4R photovoltaic concept, the second phase focused on the development of a specific component of photovoltaic systems and the third phase was the demonstration of the D4R design in two complete photovoltaic systems (grid-connected and stand-alone). This paper includes a description of the installed photovoltaic systems, including a brief summary at component level of the lithium ion battery system and the D4R power conditioning system developed for the pilot installations. Additionally, industrial symbioses within the network associated with the photovoltaic systems and the production model for the network are described

ENIGMA and global neuroscience: A decade of large-scale studies of the brain in health and disease across more than 40 countries

This review summarizes the last decade of work by the ENIGMA (Enhancing NeuroImaging Genetics through Meta Analysis) Consortium, a global alliance of over 1400 scientists across 43 countries, studying the human brain in health and disease. Building on large-scale genetic studies that discovered the first robustly replicated genetic loci associated with brain metrics, ENIGMA has diversified into over 50 working groups (WGs), pooling worldwide data and expertise to answer fundamental questions in neuroscience, psychiatry, neurology, and genetics. Most ENIGMA WGs focus on specific psychiatric and neurological conditions, other WGs study normal variation due to sex and gender differences, or development and aging; still other WGs develop methodological pipelines and tools to facilitate harmonized analyses of "big data" (i.e., genetic and epigenetic data, multimodal MRI, and electroencephalography data). These international efforts have yielded the largest neuroimaging studies to date in schizophrenia, bipolar disorder, major depressive disorder, post-traumatic stress disorder, substance use disorders, obsessive-compulsive disorder, attention-deficit/hyperactivity disorder, autism spectrum disorders, epilepsy, and 22q11.2 deletion syndrome. More recent ENIGMA WGs have formed to study anxiety disorders, suicidal thoughts and behavior, sleep and insomnia, eating disorders, irritability, brain injury, antisocial personality and conduct disorder, and dissociative identity disorder. Here, we summarize the first decade of ENIGMA's activities and ongoing projects, and describe the successes and challenges encountered along the way. We highlight the advantages of collaborative large-scale coordinated data analyses for testing reproducibility and robustness of findings, offering the opportunity to identify brain systems involved in clinical syndromes across diverse samples and associated genetic, environmental, demographic, cognitive, and psychosocial factors