Human-level Atari 200x faster

The task of building general agents that perform well over a wide range of tasks has been an important goal in reinforcement learning since its inception. The problem has been subject of research of a large body of work, with performance frequently measured by observing scores over the wide range of environments contained in the Atari 57 benchmark. Agent57 was the first agent to surpass the human benchmark on all 57 games, but this came at the cost of poor data-efficiency, requiring nearly 80 billion frames of experience to achieve. Taking Agent57 as a starting point, we employ a diverse set of strategies to achieve a 200-fold reduction of experience needed to out perform the human baseline. We investigate a range of instabilities and bottlenecks we encountered while reducing the data regime, and propose effective solutions to build a more robust and efficient agent. We also demonstrate competitive performance with high-performing methods such as Muesli and MuZero. The four key components to our approach are (1) an approximate trust region method which enables stable bootstrapping from the online network, (2) a normalisation scheme for the loss and priorities which improves robustness when learning a set of value functions with a wide range of scales, (3) an improved architecture employing techniques from NFNets in order to leverage deeper networks without the need for normalization layers, and (4) a policy distillation method which serves to smooth out the instantaneous greedy policy overtime

Robust Binding of Disulfide-Substituted Rhenium Bipyridyl Complexes for CO2 Reduction on Gold Electrodes

Heterogenization of homogenous catalysts on electrode surfaces provides a valuable approach for characterization of catalytic processes in operando conditions using surface selective spectroelectrochemistry methods. Ligand design plays a central role in the attachment mode and the resulting functionality of the heterogenized catalyst as determined by the orientation of the catalyst relative to the surface and the nature of specific interactions that modulate the redox properties under the heterogeneous electrode conditions. Here, we introduce new [Re(L)(CO)3Cl] catalysts for CO2 reduction with sulfur-based anchoring groups on a bipyridyl ligand, where L = 3,3′-disulfide-2,2′-bipyridine (SSbpy) and 3,3′-thio-2,2′-bipyridine (Sbpy). Spectroscopic and electrochemical analysis complemented by computational modeling at the density functional theory level identify the complex [Re(SSbpy)(CO)3Cl] as a multi-electron acceptor that combines the redox properties of both the rhenium tricarbonyl core and the disulfide functional group on the bipyridyl ligand. The first reduction at −0.85 V (vs. SCE) involves a two-electron process that breaks the disulfide bond, activating it for surface attachment. The heterogenized complex exhibits robust anchoring on gold surfaces, as probed by vibrational sum-frequency generation (SFG) spectroscopy. The binding configuration is normal to the surface, exposing the active site to the CO2 substrate in solution. The attachment mode is thus particularly suitable for electrocatalytic CO2 reduction.Fil: Cattaneo, Mauricio. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Tucumán. Instituto de Química del Noroeste. Universidad Nacional de Tucumán. Facultad de Bioquímica, Química y Farmacia. Instituto de Química del Noroeste; ArgentinaFil: Guo, Facheng. University of Yale; Estados UnidosFil: Kelly, H. Ray. University of Yale; Estados UnidosFil: Videla, Pablo E.. University of Yale; Estados UnidosFil: Kiefer, Laura. Emory University; Estados UnidosFil: Gebre, Sara. Emory University; Estados UnidosFil: Ge, Aimin. Emory University; Estados UnidosFil: Liu, Qiliang. Emory University; Estados UnidosFil: Wu, Shaoxiong. Emory University; Estados UnidosFil: Lian, Tianquan. Emory University; Estados UnidosFil: Batista, Víctor S.. University of Yale; Estados Unido

Analysis of four scales for global severity evaluation in Parkinson’s disease

Global evaluations of Parkinson?s disease (PD) severity are available, but their concordance and accuracy have not been previously tested. The present international, cross-sectional study was aimed at determining the agreement level among four global scales for PD (Hoehn and Yahr, HY; Clinical Global Impression of Severity, CGIS; Clinical Impression of Severity Index, CISI-PD; and Patient Global Impression of Severity, PGIS) and identifying which of them better correlates with itemized PD assessments. Assessments included additional scales for evaluation of the movement impairment, disability, affective disorders, and quality of life. Spearman correlation coefficients, weighted and generalized kappa, and Kendall?s concordance coefficient were used. Four hundred thirty three PD patients, 66% in HY stages 2 or 3, mean disease duration 8.8 years, were analyzed. Correlation between the global scales ranged from 0.60 (HY with PGIS) to 0.91 (CGIS with CISI-PD). Kendall?s coefficient of concordance resulted 0.76 (P<0.0001). HY and CISI-PD showed the highest association with age, disease duration, and levodopa-equivalent daily dose, and CISI-PD with measures of PD manifestations, disability, and quality of life. PGIS and CISI-PD correlated similarly with anxiety and depression scores. The lowest agreement in classifying patients as mild, moderate, or severe was observed between PGIS and HY or CISI-PD (58%) and the highest between CGIS and CISI-PD (84.3%). The four PD global severity scales agree moderately to strongly among them; clinician-based ratings estimate PD severity, as established by other measures, better than PGIS; and the CISI-PD showed the highest association with measures of impairment, disability, and quality of life.Fil: Martinez Martin, Pablo. Universidad Carlos III de Madrid. Instituto de Salud; EspañaFil: Rojo Abuin, José Manuel. Consejo Superior de Investigaciones Cientificas. Centro de Ciencias Humanas y Sociales. Instituto de Historia.; EspañaFil: Rodríguez Violante, Mayela. Instituto Nacional de Neurología y Neurocirugía; MéxicoFil: Serrano Dueñas, Marcos. Pontificia Universidad Católica del Ecuador; EcuadorFil: Garreto, Nélida Susana. Universidad de Buenos Aires. Facultad de Medicina. Centro Universitario de Neurologia "dr. Jose Maria Ramos Mejia".; ArgentinaFil: Martínez Castrillo, Juan Carlos. Instituto Ramón y Cajal de Investigación Sanitaria; EspañaFil: Campos Arillo, Víctor. Hospital Xanit International; EspañaFil: Fernández, William. Universidad Nacional de Colombia; ColombiaFil: Chaná Cuevas, Pedro. Universidad de Santiago de Chile. Facultad de Humanidades. Instituto de Ciencias Biomédicas.; ChileFil: Arakaki, Tomoko. Universidad de Buenos Aires. Facultad de Medicina. Centro Universitario de Neurologia "dr. Jose Maria Ramos Mejia".; Argentina. Fundación para la Lucha contra las Enfermedades Neurológicas de la Infancia; ArgentinaFil: Alvarez, Mario Gustavo. Centro Internacional de Restauración Neurológica ; CubaFil: Pedroso Ibañez, Ivonne. Centro Internacional de Restauración Neurológica ; CubaFil: Rodríguez Blázquez , Carmen. Universidad Carlos III de Madrid. Instituto de Salud; EspañaFil: Ray Chaudhuri , Kallol. National Parkinson Foundation International Centre of Excellence; Reino UnidoFil: Merello, Marcelo Jorge. Fundación para la Lucha contra las Enfermedades Neurológicas de la Infancia; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentin

Measuring membrane permeation rates through the optical visualization of a single pore

Membranes are a critical technology for energy-efficient separation processes. The routine method of evaluating membrane performance is a permeation measurement. However, such measurements can be limited in terms of their utility: membrane microstructure is often poorly characterized; membranes or sealants leak; and conditions in the gas phase are poorly controlled and frequently far-removed from the conditions employed in the majority of real processes. Here, we demonstrate a new integrated approach to determine permeation rates, using two novel supported molten-salt membrane geometries. In both cases, the membranes comprise a solid support with laser-drilled pores, which are infiltrated with a highly CO2-selective molten carbonate salt. First, we fabricate an optically transparent single-crystal, single-pore model membrane by local laser drilling. By infiltrating the single pore with molten carbonate, monitoring the gas-liquid interface optically, and using image analysis on gas bubbles within the molten carbonate (because they change volume upon controlled changes in gas composition), we extract CO2 permeation rates with exceptional speed and precision. Additionally, in this arrangement, microstructural characterization is more straightforward and a sealant is not required, eliminating a major source of leakage. Furthermore, we demonstrate that the technique can be used to probe a previously unexplored driving force region, too low to access with conventional methods. Subsequently, we fabricate a leak-free tubular-supported molten-salt membrane with 1000 laser-drilled pores (infiltrated with molten carbonate) and employ a CO2-containing sweep gas to obtain permeation rates in a system that can be described with unprecedented precision. Together, the two approaches provide new ways to measure permeation rates with increased speed and at previously inaccesible conditions

Anatomy of the ankle ligaments: a pictorial essay

Understanding the anatomy of the ankle ligaments is important for correct diagnosis and treatment. Ankle ligament injury is the most frequent cause of acute ankle pain. Chronic ankle pain often finds its cause in laxity of one of the ankle ligaments. In this pictorial essay, the ligaments around the ankle are grouped, depending on their anatomic orientation, and each of the ankle ligaments is discussed in detail

The 16th Data Release of the Sloan Digital Sky Surveys: First Release from the APOGEE-2 Southern Survey and Full Release of eBOSS Spectra

This paper documents the 16th data release (DR16) from the Sloan Digital Sky Surveys (SDSS), the fourth and penultimate from the fourth phase (SDSS-IV). This is the first release of data from the Southern Hemisphere survey of the Apache Point Observatory Galactic Evolution Experiment 2 (APOGEE-2); new data from APOGEE-2 North are also included. DR16 is also notable as the final data release for the main cosmological program of the Extended Baryon Oscillation Spectroscopic Survey (eBOSS), and all raw and reduced spectra from that project are released here. DR16 also includes all the data from the Time Domain Spectroscopic Survey and new data from the SPectroscopic IDentification of ERosita Survey programs, both of which were co-observed on eBOSS plates. DR16 has no new data from the Mapping Nearby Galaxies at Apache Point Observatory (MaNGA) survey (or the MaNGA Stellar Library "MaStar"). We also preview future SDSS-V operations (due to start in 2020), and summarize plans for the final SDSS-IV data release (DR17)

The Seventeenth Data Release of the Sloan Digital Sky Surveys: Complete Release of MaNGA, MaStar and APOGEE-2 Data

This paper documents the seventeenth data release (DR17) from the Sloan Digital Sky Surveys; the fifth and final release from the fourth phase (SDSS-IV). DR17 contains the complete release of the Mapping Nearby Galaxies at Apache Point Observatory (MaNGA) survey, which reached its goal of surveying over 10,000 nearby galaxies. The complete release of the MaNGA Stellar Library (MaStar) accompanies this data, providing observations of almost 30,000 stars through the MaNGA instrument during bright time. DR17 also contains the complete release of the Apache Point Observatory Galactic Evolution Experiment 2 (APOGEE-2) survey which publicly releases infra-red spectra of over 650,000 stars. The main sample from the Extended Baryon Oscillation Spectroscopic Survey (eBOSS), as well as the sub-survey Time Domain Spectroscopic Survey (TDSS) data were fully released in DR16. New single-fiber optical spectroscopy released in DR17 is from the SPectroscipic IDentification of ERosita Survey (SPIDERS) sub-survey and the eBOSS-RM program. Along with the primary data sets, DR17 includes 25 new or updated Value Added Catalogs (VACs). This paper concludes the release of SDSS-IV survey data. SDSS continues into its fifth phase with observations already underway for the Milky Way Mapper (MWM), Local Volume Mapper (LVM) and Black Hole Mapper (BHM) surveys

The IDENTIFY study: the investigation and detection of urological neoplasia in patients referred with suspected urinary tract cancer - a multicentre observational study

Objective To evaluate the contemporary prevalence of urinary tract cancer (bladder cancer, upper tract urothelial cancer [UTUC] and renal cancer) in patients referred to secondary care with haematuria, adjusted for established patient risk markers and geographical variation. Patients and Methods This was an international multicentre prospective observational study. We included patients aged ≥16 years, referred to secondary care with suspected urinary tract cancer. Patients with a known or previous urological malignancy were excluded. We estimated the prevalence of bladder cancer, UTUC, renal cancer and prostate cancer; stratified by age, type of haematuria, sex, and smoking. We used a multivariable mixed-effects logistic regression to adjust cancer prevalence for age, type of haematuria, sex, smoking, hospitals, and countries. Results Of the 11 059 patients assessed for eligibility, 10 896 were included from 110 hospitals across 26 countries. The overall adjusted cancer prevalence (n = 2257) was 28.2% (95% confidence interval [CI] 22.3–34.1), bladder cancer (n = 1951) 24.7% (95% CI 19.1–30.2), UTUC (n = 128) 1.14% (95% CI 0.77–1.52), renal cancer (n = 107) 1.05% (95% CI 0.80–1.29), and prostate cancer (n = 124) 1.75% (95% CI 1.32–2.18). The odds ratios for patient risk markers in the model for all cancers were: age 1.04 (95% CI 1.03–1.05; P < 0.001), visible haematuria 3.47 (95% CI 2.90–4.15; P < 0.001), male sex 1.30 (95% CI 1.14–1.50; P < 0.001), and smoking 2.70 (95% CI 2.30–3.18; P < 0.001). Conclusions A better understanding of cancer prevalence across an international population is required to inform clinical guidelines. We are the first to report urinary tract cancer prevalence across an international population in patients referred to secondary care, adjusted for patient risk markers and geographical variation. Bladder cancer was the most prevalent disease. Visible haematuria was the strongest predictor for urinary tract cancer

Modelling human choices: MADeM and decision‑making

Research supported by FAPESP 2015/50122-0 and DFG-GRTK 1740/2. RP and AR are also part of the Research, Innovation and Dissemination Center for Neuromathematics FAPESP grant (2013/07699-0). RP is supported by a FAPESP scholarship (2013/25667-8). ACR is partially supported by a CNPq fellowship (grant 306251/2014-0)