Beyond the status quo in deep reinforcement learning

L’apprentissage par renforcement profond (RL) a connu d’énormes progrès ces dernières années, mais il est encore difficile d’appliquer le RL aux problèmes de prise de décision du monde réel. Cette thèse identifie trois défis clés avec la façon dont nous faisons la recherche RL elle-même qui entravent les progrès de la recherche RL. — Évaluation et comparaison peu fiables des algorithmes RL ; les méthodes d’évaluation actuelles conduisent souvent à des résultats peu fiables. — Manque d’informations préalables dans la recherche RL ; Les algorithmes RL sont souvent formés à partir de zéro, ce qui peut nécessiter de grandes quantités de données ou de ressources informatiques. — Manque de compréhension de la façon dont les réseaux de neurones profonds interagissent avec RL, ce qui rend difficile le développement de méthodes évolutives de RL. Pour relever ces défis susmentionnés, cette thèse apporte les contributions suivantes : — Une méthodologie plus rigoureuse pour évaluer les algorithmes RL. — Un flux de travail de recherche alternatif qui se concentre sur la réutilisation des progrès existants sur une tâche. — Identification d’un phénomène de perte de capacité implicite avec un entraînement RL hors ligne prolongé. Dans l’ensemble, cette thèse remet en question le statu quo dans le RL profond et montre comment cela peut conduire à des algorithmes de RL plus efficaces, fiables et mieux applicables dans le monde réel.Deep reinforcement learning (RL) has seen tremendous progress in recent years, but it is still difficult to apply RL to real-world decision-making problems. This thesis identifies three key challenges with how we do RL research itself that hinder the progress of RL research. — Unreliable evaluation and comparison of RL algorithms; current evaluation methods often lead to unreliable results. — Lack of prior information in RL research; RL algorithms are often trained from scratch, which can require large amounts of data or computational resources. — Lack of understanding of how deep neural networks interact with RL, making it hard to develop scalable RL methods. To tackle these aforementioned challenges, this thesis makes the following contributions: — A more rigorous methodology for evaluating RL algorithms. — An alternative research workflow that focuses on reusing existing progress on a task. — Identifying an implicit capacity loss phenomenon with prolonged offline RL training. Overall, this thesis challenges the status quo in deep reinforcement learning and shows that doing so can make RL more efficient, reliable and improve its real-world applicabilit

Endothelial Cell Junctional Adhesion Molecules: Role and Regulation of Expression in Inflammation

The authors are funded by the Wellcome Trust (Investigator Award to S. Nourshargh Ref: 098291/Z/12/Z). N. Reglero-Real is additionally supported by funding from the People Programme (Marie Curie Actions) of the European Union’s Seventh Framework Programme (FP7/2007–2013) under REA grant agreement no [608765]

Timing of magmatic-hydrothermal activity in the Variscan Orogenic Belt: LA-ICP-MS U–Pb geochronology of skarn-related garnet from the Schwarzenberg District, Erzgebirge

[EN] Here, we present in situ U-Pb laser ablation-inductively coupled plasma-mass spectrometry (LA-ICP-MS) ages of andradite-grossular garnet from four magmatic-hydrothermal polymetallic skarn prospects in the Schwarzenberg District, Erzgebirge (Germany), located in the internal zone of the Variscan Orogenic Belt. Within the geochronological framework of igneous rocks and hydrothermal mineralization in the Erzgebirge, the obtained garnet ages define three distinct episodes of Variscan skarn formation: (I) early late-collisional mineralization (338-331 Ma) recording the onset of magmatic-hydrothermal fluid flow shortly after the peak metamorphic event, (II) late-collisional mineralization (similar to 327-310 Ma) related to the emplacement of large peraluminous granites following large-scale extension caused by orogenic collapse and (III) post-collisional mineralization (similar to 310-295 Ma) contemporaneous with widespread volcanism associated with Permian crustal reorganization. Our results demonstrate that the formation of skarns in the Schwarzenberg District occurred episodically in all sub-stages of the Variscan orogenic cycle over a time range of at least 40 Ma. This observation is consistent with the age range of available geochronological data related to magmatic-hydrothermal ore deposits from other internal zones of the Variscan Orogenic Belt in central and western Europe. In analogy to the time-space relationship of major porphyry-Cu belts in South America, the congruent magmatic-hydrothermal evolution in the internal zones and the distinctly later (by similar to 30 Ma) occurrence of magmatic-hydrothermal ore deposits in the external zones of the Variscan Orogenic Belt may be interpreted as a function of their tectonic position relative to the Variscan collisional front.Open Access funding enabled and organized by Projekt DEAL. This study was funded by the Federal State of Saxony and the European Social Fund (Grant no. 100339454 received by M. Burisch

Causal effects of monetary shocks: semiparametric conditional independence tests with a multinomial propensity score

Macroeconomists have long been concerned with the causal effects of monetary policy. When the identification of causal effects is based on a selection-on-observables assumption, non-causality amounts to the conditional independence of outcomes and policy changes. This paper develops a semiparametric test for conditional independence in time series models linking a multinomial policy variable with unobserved potential outcomes. Our approach to conditional independence testing is motivated by earlier parametric tests, as in Romer and Romer (1989, 1994, 2004). The procedure developed here is semiparametric in the sense that we model the process determining the distribution of treatment - the policy propensity score - but leave the model for outcomes unspecified. A conceptual innovation is that we adapt the cross-sectional potential outcomes framework to a time series setting. This leads to a generalized definition of Sims (1980) causality. A technical contribution is the development of root-T consistent distribution-free inference methods for full conditional independence testing, appropriate for dependent data and allowing for first-step estimation of the propensity score

Monoclonal antibodies specific for the hemagglutinin-neuraminidase protein define neutralizing epitopes specific for Newcastle disease virus genotype 2.VII from Egypt

Background Newcastle disease is a devastating disease in poultry caused by virulent Newcastle disease virus (NDV), a paramyxovirus endemic in many regions of the world despite intensive vaccination. Phylogenetic analyses reveal ongoing evolution of the predominant circulating genotype 2.VII, and the relevance of potential antigenic drift is under discussion. To investigate variation within neutralization-sensitive epitopes within the protein responsible for receptor binding, i.e. the Hemagglutinin-Neuraminidase (HN) spike protein, we were interested in establishing genotype-specific monoclonal antibodies (MAbs). Methods An HN-enriched fraction of a gradient-purified NDV genotype 2.VII was prepared and successfully employed to induce antibodies in BalbC mice that recognize conformationally intact sites reactive by haemagglutination inhibition (HI). For subsequent screening of mouse hybridoma cultures, an NDV-ELISA was established that utilizes Concanavalin A (ConA-ELISA) coupled glycoproteins proven to present conformation-dependent epitopes. Results Six out of nine selected MAbs were able to block receptor binding as demonstrated by HI activity. One MAb recognized an epitope only present in the homologue virus, while four other MAbs showed weak reactivity to selected other genotypes. On the other hand, one broadly cross-reacting MAb reacted with all genotypes tested and resembled the reactivity profile of genotype-specific polyclonal antibody preparations that point to minor antigenic differences between tested NDV genotpyes. Conclusions These results point to the concurrent presence of variable and conserved epitopes within the HN molecule of NDV. The described protocol should help to generate MAbs against a variety of NDV strains and to enable in depth analysis of the antigenic profiles of different genotypes

Two Modes of Cluster Dynamics Govern the Viscoelasticity of Colloidal Gels

Colloidal gels formed by strongly attractive particles at low particle volume fractions are composed of space-spanning networks of uniformly sized clusters. We study the thermal fluctuations of the clusters using differential dynamic microscopy by decomposing them into two modes of dynamics, and link them to the macroscopic viscoelasticity via rheometry. The first mode, dominant at early times, represents the localized, elastic fluctuations of individual clusters. The second mode, pronounced at late times, reflects the collective, viscoelastic dynamics facilitated by the connectivity of the clusters. By mixing two types of particles of distinct attraction strengths in different proportions, we control the transition time at which the collective mode starts to dominate, and hence tune the frequency dependence of the linear viscoelastic moduli of the binary gels

Dating minerals by ID-TIMS geochronology at times of in situ analysis: selected case studies from the CPGeo-IGc-USP laboratory

Since 1964, the Center for Geochronological Research - CPGeo, one of the interdepartmental centers of the Instituto de Geociências (IG) of São Paulo University, has developed studies related to several geological processes associated with different rock types. Thermal Ionization Mass Spectrometry Isotopic Dilution (ID-TIMS) has been the technique widely used in the CPGeo U-Pb Laboratory. It provides reliable and accurate results in age determination of superposed events. However, the open-system behavior such as Pb-loss, the inheritance problem and metamictization processes allow and impel us to a much richer understanding of the power and limitations of U-Pb geochronology and thermochronology. In this article, we present the current methodology used at the CPGeo-IGc-USP U-Pb laboratory, the improvements on ID-TIMS method, and report high-precision U-Pb data from zircon, monazite, epidote, titanite, baddeleyite and rutile from different rock types of several domains of the Brazilian south-southeast area, Argentina and Uruguay.O Centro de Pesquisas Geocronológicas (CPGeo), um dos centros interdepartamentais do Instituto de Geociências (IG) da Universidade de São Paulo (USP), desde 1964 desenvolve estudos relacionados a diversos processos geológicos que se associam a diferentes tipos de rochas. A técnica amplamente utilizada no Laboratório U-Pb é a diluição isotópica por espectrometria de massa termo ionizada (ID-TIMS). Esta sistemática proporciona resultados bastante confiáveis e precisos na determinação das idades de eventos geológicos superpostos. Entretanto, o comportamento de sistema aberto como perda de Pb, problemas de herança isotópica e processos de metamictização, nos permite o entendimento do poder e limitação da geocronologia e termocronologia U-Pb. Neste artigo apresentamos a metodologia atualmente utilizada no Laboratório U-Pb do CPGeo-IGc-USP, as melhorias atingidas na técnica ID-TIMS e alguns dados obtidos em zircão, epídoto, titanita, baddeleyita e rutilo de diferentes tipos de rochas de alguns domínios da região sul-sudeste brasileira e da Argentina e Uruguai.Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP

AMPK:regulating energy balance at the cellular and whole body levels

AMP-activated protein kinase appears to have evolved in single-celled eukaryotes as an adenine nucleotide sensor that maintains energy homeostasis at the cellular level. However, during evolution of more complex multicellular organisms, the system has adapted to interact with hormones so that it also plays a key role in balancing energy intake and expenditure at the whole body level