Reward-Predictive Clustering

Recent advances in reinforcement-learning research have demonstrated impressive results in building algorithms that can out-perform humans in complex tasks. Nevertheless, creating reinforcement-learning systems that can build abstractions of their experience to accelerate learning in new contexts still remains an active area of research. Previous work showed that reward-predictive state abstractions fulfill this goal, but have only be applied to tabular settings. Here, we provide a clustering algorithm that enables the application of such state abstractions to deep learning settings, providing compressed representations of an agent's inputs that preserve the ability to predict sequences of reward. A convergence theorem and simulations show that the resulting reward-predictive deep network maximally compresses the agent's inputs, significantly speeding up learning in high dimensional visual control tasks. Furthermore, we present different generalization experiments and analyze under which conditions a pre-trained reward-predictive representation network can be re-used without re-training to accelerate learning -- a form of systematic out-of-distribution transfer

Novel single-source precursors for the fabrication of PbTiO3, PbZrO3 and Pb(Zr1-x Tix)O3 thin-films by chemical vapor deposition

Lead titanate, lead zirconate, and lead zirconate titanate (PZT) films in the sub-μm-range were produced at temperatures around 400 °C using novel single-source precursors in a classical thermal CVD process. The design of two bimetallic alkoxide compounds, a lead titanate and a lead zirconate source with almost identical physical properties and complement miscibility, resulted in a new quasi-single-source PZT precursor, an azeotropic mixture that evaporates at 30 °C and at a pressure of 4 × 10−1 mbar. After thermal treatment at 650 °C, transparent (100)-oriented PZT films with remnant polarization of 20 μC cm−2 and a coercive field strength of 20 V μm−1 were achieved. An additional lead source is not required

Influence of Increasing Nutrient Availability on Fern and Lycophyte Diversity

Increased nutrient supply can have drastic effects on natural ecosystems, especially in naturally nutrient-poor ones such as most tropical rainforests. Many studies have focused on the reaction of trees to fertilization, but little is known about herbaceous plants. Ferns are a particularly common group in tropical forests, spanning all vegetation types and zones. Here, we assess how seven years of moderate addition of nitrogen (N), phosphorus (P), and N+P along an elevational gradient (1000–3000 m) have impacted richness and composition of fern and lycophyte assemblages in tropical montane rain forests growing on naturally nutrient deficient soils in the Ecuadorian Andes. We found that fertilization does not affect overall species richness, but that there were strong differences in species abundances (∼60% of species), both negative and positive, that were apparently related to the systematic affiliations and ecological properties of the affected species. These diverse responses of ferns to fertilization provide insight into the sensitivity and complexity of the relationships of nutrient availability and community composition in tropical forests

Proxying economic activity with daytime satellite imagery: Filling data gaps across time and space

This paper develops a novel procedure for proxying economic activity with daytime satellite imagery across time periods and spatial units, for which reliable data on economic activity are otherwise not available. In developing this unique proxy, we apply machine-learning techniques to a historical time series of daytime satellite imagery dating back to 1984. Compared to satellite data on night light intensity, another common economic proxy, our proxy more precisely predicts economic activity at smaller regional levels and over longer time horizons. We demonstrate our measure’s usefulness for the example of Germany, where East German data on economic activity are unavailable for detailed regional levels and historical time series. Our procedure is generalizable to any region in the world, and it has great potential for analyzing historical economic developments, evaluating local policy reforms, and controlling for economic activity at highly disaggregated regional levels in econometric applications

Changes in the prevalence, treatment and control of hypertension in Germany? : a clinical-epidemiological study of 50.000 primary care patients

INTRODUCTION: Medical societies have developed guidelines for the detection, treatment and control of hypertension (HTN). Our analysis assessed the extent to which such guidelines were implemented in Germany in 2003 and 2001. METHODS: Using standardized clinical diagnostic and treatment appraisal forms, blood pressure levels and patient questionnaires for 55,518 participants from the cross-sectional Targets and Essential Data for Commitment of Treatment (DETECT) study (2003) were analyzed. Physician's diagnosis of hypertension (HTN(doc)) was defined as coding hypertension in the clinical appraisal questionnaire. Alternative definitions used were physician's diagnosis or the patient's self-reported diagnosis of hypertension (HTN(doc,pat)), physician's or patient's self-reported diagnosis or a BP measurement with a systolic BP≥140 mmHg and/or a diastolic BP≥90 (HTN(doc,pat,bp)) and diagnosis according to the National Health and Nutrition Examination Survey (HTN(NHANES)). The results were compared with the similar German HYDRA study to examine whether changes had occurred in diagnosis, treatment and adequate blood pressure control (BP below 140/90 mmHg) since 2001. Factors associated with pharmacotherapy and control were determined. RESULTS: The overall prevalence rate for hypertension was 35.5% according to HTN(doc) and 56.0% according to NHANES criteria. Among those defined by NHANES criteria, treatment and control rates were 56.0% and 20.3% in 2003, and these rates had improved from 55.3% and 18.0% in 2001. Significant predictors of receiving antihypertensive medication were: increasing age, female sex, obesity, previous myocardial infarction and the prevalence of comorbid conditions such as coronary heart disease (CHD), hyperlipidemia and diabetes mellitus (DM). Significant positive predictors of adequate blood pressure control were CHD and antihypertensive medication. Inadequate control was associated with increasing age, male sex and obesity. CONCLUSIONS: Rates of treated and controlled hypertension according to NHANES criteria in DETECT remained low between 2001 and 2003, although there was some minor improvement

Visual selective attention is equally functional for individuals with low and high working memory capacity: Evidence from accuracy and eye movements

Selective attention and working memory capacity (WMC) are related constructs, but debate about the manner in which they are related remains active. One elegant explanation of variance in WMC is that the efficiency of filtering irrelevant information is the crucial determining factor, rather than differences in capacity per se. We examined this hypothesis by relating WMC (as measured by complex span tasks) to accuracy and eye movements during visual change detection tasks with different degrees of attentional filtering and allocation requirements. Our results did not indicate strong filtering differences between high- and low-WMC groups, and where differences were observed, they were counter to those predicted by the strongest attentional filtering hypothesis. Bayes factors indicated evidence favoring positive or null relationships between WMC and correct responses to unemphasized information, as well as between WMC and the time spent looking at unemphasized information. These findings are consistent with the hypothesis that individual differences in storage capacity, not only filtering efficiency, underlie individual differences in working memory

Particle-Accelerator Constraints on Isotropic Modifications of the Speed of Light

The absence of vacuum Cherenkov radiation for 104.5 GeV electrons and positrons at LEP combined with the observed stability of 300 GeV photons at the Tevatron constrains deviations of the speed of light relative to the maximal attainable speed of electrons. Within the Standard-Model Extension (SME), the limit $-5.8\times10^{-12} \leq \tilde{\kappa}_{\rm tr}-{4/3}c_e^{00} \leq 1.2\times10^{-11}$ is extracted, which sharpens previous bounds by more than 3 orders of magnitude. The potential for further refinements of this limit with terrestrial experiments and astrophysical observations is discussed.Physic

The Planet, 1991, Winter

https://cedar.wwu.edu/planet/1006/thumbnail.jp