Estimating the reliability of MDP policies: A confidence interval approach

Past approaches for using reinforcement learning to derive dialog control policies have assumed that there was enough collected data to derive a reliable policy. In this paper we present a methodology for numerically constructing confidence intervals for the expected cumulative reward for a learned policy. These intervals are used to (1) better assess the reliability of the expected cumulative reward, and (2) perform a refined comparison between policies derived from different Markov Decision Processes (MDP) models. We applied this methodology to a prior experiment where the goal was to select the best features to include in the MDP statespace. Our results show that while some of the policies developed in the prior work exhibited very large confidence intervals, the policy developed from the best feature set had a much smaller confidence interval and thus showed very high reliability. © 2007 Association for Computational Linguistics

Plasma Dynamics

Contains reports on ten research projects split into two sections.National Science Foundation (Grant ENG77-00340)U.S. Department of Energy (Contract EY-76-S-02-2766)U.S. Air Force - Office of Scientific Research (Grant AFOSR-77-3143)U.S. Department of Energy (Contract ET-78-C-01-3019)U.S. Department of Energy (Contract ET-78-S-02-4681)U.S. Department of Energy (Contract ET-78-S-02-4682)U.S. Department of Energy (Grant EG-77-G-01-4107)U.S. Department of Energy (Contract ET-78-S-02-4714)U.S. Department of Energy (Contract ET-78-S-02-4886)U.S. Department of Energy (Contract ET-78-S-02-4690

Plasma Dynamics

Contains reports on ten research projects divided into two sections.National Science Foundation (Grant ENG79-07047)U.S. Air Force - Office of Scientific Research (Grant AFOSR-77-3143)U.S. Department of Energy (Contract DE-ACO2-78ET51013)U.S. Department of Energy (Contract DE-ASO2-78ET53073.AO02)U.S. Department of Energy (Contract ET-78-S-02-4682)U.S. Department of Energy (Contract DE-AS02-78ET53074)U.S. Department of Energy (Contract DE-ASO2-78ET53050)U.S. Department of Energy (Contract DE-AS02-78ET51002)U.S. Department of Energy (Contract DE-ASO2-78ET53076

Plasma Dynamics

Contains reports on seventeen research projects split into two sections.National Science Foundation (Grant ENG77-00340)U. S. Energy Research and Development Administration (Contract E(11-1)-2766)U. S. Energy Research and Development Administration (Contract EY-76-S-02-2766)U. S. Air Force - Office of Scientific Research (Grant AFOSR-77-3143)U. S. Department of Energy (Grant EG-77-G-01-4107

Plasma Dynamics

Contains research objectives and summary of research on nineteen research projects split into five sections.National Science Foundation (Grant ENG75-06242-A01)U.S. Energy Research and Development Administration (Contract E(11-1)-2766)U.S. Air Force - Office of Scientific Research (Grant AFOSR-77-3143)U.S. Energy Research and Development Administration (Contract EY-76-C2-02-3070.*000

A New Strategy to Identify and Annotate Human RPE-Specific Gene Expression

Background: To identify and functionally annotate cell type-specific gene expression in the human retinal pigment epithelium (RPE), a key tissue involved in age-related macular degeneration and retinitis pigmentosa. Methodology: RPE, photoreceptor and choroidal cells were isolated from selected freshly frozen healthy human donor eyes using laser microdissection. RNA isolation, amplification and hybridization to 44 k microarrays was carried out according to Agilent specifications. Bioinformatics was carried out using Rosetta Resolver, David and Ingenuity software. Principal Findings: Our previous 22 k analysis of the RPE transcriptome showed that the RPE has high levels of protein synthesis, strong energy demands, is exposed to high levels of oxidative stress and a variable degree of inflammation. We currently use a complementary new strategy aimed at the identification and functional annotation of RPE-specific expressed transcripts. This strategy takes advantage of the multilayered cellular structure of the retina and overcomes a number of limitations of previous studies. In triplicate, we compared the transcriptomes of RPE, photoreceptor and choroidal cells and we deduced RPE specific expression. We identified at least 114 entries with RPE-specific gene expression. Thirty-nine of these 114 genes also show high expression in the RPE, comparison with the literature showed that 85% of these 39 were previously identified to be expressed in the RPE. In the group of 114 RPE specific genes there was an overrepresentation of genes involved in (membrane) transport, vision and ophthalmic disease. More fundamentally, we found RPE-specific involvement in the RAR-activation, retinol metabolism and GABA receptor signaling pathways. Conclusions: In this study we provide a further specification and understanding of the RPE transcriptome by identifying and analyzing genes that are specifically expressed in the RPE

Spearfishing Regulation Benefits Artisanal Fisheries: The ReGS Indicator and Its Application to a Multiple-Use Mediterranean Marine Protected Area

The development of fishing efficiency coupled with an increase of fishing effort led to the overexploitation of numerous natural marine resources. In addition to this commercial pressure, the impact of recreational activities on fish assemblages remains barely known. Here we examined the impact of spearfishing limitation on resources in a marine protected area (MPA) and the benefit it provides for the local artisanal fishery through the use of a novel indicator. We analysed trends in the fish assemblage composition using artisanal fisheries data collected in the Bonifacio Strait Natural Reserve (BSNR), a Mediterranean MPA where the spearfishing activity has been forbidden over 15% of its area. Fish species were pooled into three response groups according to their target level by spearfishing. We developed the new flexible ReGS indicator reflecting shifts in species assemblages according to the relative abundance of each response group facing external pressure. The catch per unit effort (CPUE) increased by ca. 60% in the BSNR between 2000 and 2007, while the MPA was established in 1999. The gain of CPUE strongly depended on the considered response group: for the highly targeted group, the CPUE doubled while the CPUE of the untargeted group increased by only 15.5%. The ReGS value significantly increased from 0.31 to 0.45 (on a scale between 0 and 1) in the general perimeter of this MPA while it has reached a threshold of 0.43, considered as a reference point, in the area protected from spearfishing since 1982. Our results demonstrated that limiting recreational fishing by appropriate zoning in multiple-use MPAs represents a real benefit for artisanal fisheries. More generally we showed how our new indicator may reveal a wide range of impacts on coastal ecosystems such as global change or habitat degradation

Structure of human RNA polymerase III

In eukaryotes, RNA Polymerase (Pol) III is specialized for the transcription of tRNAs and other short, untranslated RNAs. Pol III is a determinant of cellular growth and lifespan across eukaryotes. Upregulation of Pol III transcription is observed in cancer and causative Pol III mutations have been described in neurodevelopmental disorders and hypersensitivity to viral infection. Here, we report a cryo-EM reconstruction at 4.0 Å of human Pol III, allowing mapping and rationalization of reported genetic mutations. Mutations causing neurodevelopmental defects cluster in hotspots affecting Pol III stability and/or biogenesis, whereas mutations affecting viral sensing are located in proximity to DNA binding regions, suggesting an impairment of Pol III cytosolic viral DNA-sensing. Integrating x-ray crystallography and SAXS, we also describe the structure of the higher eukaryote specific RPC5 C-terminal extension. Surprisingly, experiments in living cells highlight a role for this module in the assembly and stability of human Pol III

Completeness of reporting and risks of overstating impact in cluster randomised trials : a systematic review

Acknowledgments We received no funding specifically for this systematic review. ELT is funded in part by awards R01-AI141444 from the National Institute of Allergy and Infectious Diseases and R01-MH120649 from the US National Institute of Mental Health; both Institutes are part of the National Institutes of Health (NIH). JAG and ACP's support of this project was made possible (in part) by grant number UL1TR002553 from the National Center for Advancing Translational Sciences of the NIH, and the NIH Roadmap for Medical Research. JEM is supported by an Australian National Health and Medical Research Council Career Development Fellowship (APP1143429). SN was supported by an award that is jointly funded by the UK Medical Research Council (MRC) and the UK Department for International Development (DFID) under the MRC/DFID Concordat agreement, and part of the EDCTP2 programme supported by the European Union (grant reference MR/R010161/1). ABF acknowledges funding support from the National Health and Medical Research Council of Australia (grant ID 1183303). KH is funded by a National Institute for Health Research Senior Research Fellowship SRF-2017-10-002. The contents of the research included in this manuscript are solely the responsibility of the authors and do not necessarily represent the official views of any of the funders. The research contributed by all authors of this manuscript are independent of their funders. Specifically, the funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. We wish to thank the three reviewers for their insightful comments and constructive feedback.Peer reviewedPublisher PD