Meta Reinforcement Learning for Sim-to-real Domain Adaptation

Modern reinforcement learning methods suffer from low sample efficiency and unsafe exploration, making it infeasible to train robotic policies entirely on real hardware. In this work, we propose to address the problem of sim-to-real domain transfer by using meta learning to train a policy that can adapt to a variety of dynamic conditions, and using a task-specific trajectory generation model to provide an action space that facilitates quick exploration. We evaluate the method by performing domain adaptation in simulation and analyzing the structure of the latent space during adaptation. We then deploy this policy on a KUKA LBR 4+ robot and evaluate its performance on a task of hitting a hockey puck to a target. Our method shows more consistent and stable domain adaptation than the baseline, resulting in better overall performance.Comment: Submitted to ICRA 202

Training and Evaluation of Deep Policies using Reinforcement Learning and Generative Models

We present a data-efficient framework for solving sequential decision-making problems which exploits the combination of reinforcement learning (RL) and latent variable generative models. The framework, called GenRL, trains deep policies by introducing an action latent variable such that the feed-forward policy search can be divided into two parts: (i) training a sub-policy that outputs a distribution over the action latent variable given a state of the system, and (ii) unsupervised training of a generative model that outputs a sequence of motor actions conditioned on the latent action variable. GenRL enables safe exploration and alleviates the data-inefficiency problem as it exploits prior knowledge about valid sequences of motor actions. Moreover, we provide a set of measures for evaluation of generative models such that we are able to predict the performance of the RL policy training prior to the actual training on a physical robot. We experimentally determine the characteristics of generative models that have most influence on the performance of the final policy training on two robotics tasks: shooting a hockey puck and throwing a basketball. Furthermore, we empirically demonstrate that GenRL is the only method which can safely and efficiently solve the robotics tasks compared to two state-of-the-art RL methods.Comment: arXiv admin note: substantial text overlap with arXiv:2007.1313

Global maps of soil temperature

Research in global change ecology relies heavily on global climatic grids derived from estimates of air temperature in open areas at around 2 m above the ground. These climatic grids do not reflect conditions below vegetation canopies and near the ground surface, where critical ecosystem functions occur and most terrestrial species reside. Here, we provide global maps of soil temperature and bioclimatic variables at a 1-km2 resolution for 0–5 and 5–15 cm soil depth. These maps were created by calculating the difference (i.e. offset) between in situ soil temperature measurements, based on time series from over 1200 1-km2 pixels (summarized from 8519 unique temperature sensors) across all the world\u27s major terrestrial biomes, and coarse-grained air temperature estimates from ERA5-Land (an atmospheric reanalysis by the European Centre for Medium-Range Weather Forecasts). We show that mean annual soil temperature differs markedly from the corresponding gridded air temperature, by up to 10°C (mean = 3.0 ± 2.1°C), with substantial variation across biomes and seasons. Over the year, soils in cold and/or dry biomes are substantially warmer (+3.6 ± 2.3°C) than gridded air temperature, whereas soils in warm and humid environments are on average slightly cooler (−0.7 ± 2.3°C). The observed substantial and biome-specific offsets emphasize that the projected impacts of climate and climate change on near-surface biodiversity and ecosystem functioning are inaccurately assessed when air rather than soil temperature is used, especially in cold environments. The global soil-related bioclimatic variables provided here are an important step forward for any application in ecology and related disciplines. Nevertheless, we highlight the need to fill remaining geographic gaps by collecting more in situ measurements of microclimate conditions to further enhance the spatiotemporal resolution of global soil temperature products for ecological applications

Global maps of soil temperature

Research in global change ecology relies heavily on global climatic grids derived from estimates of air temperature in open areas at around 2 m above the ground. These climatic grids do not reflect conditions below vegetation canopies and near the ground surface, where critical ecosystem functions occur and most terrestrial species reside. Here, we provide global maps of soil temperature and bioclimatic variables at a 1-km² resolution for 0–5 and 5–15 cm soil depth. These maps were created by calculating the difference (i.e., offset) between in-situ soil temperature measurements, based on time series from over 1200 1-km² pixels (summarized from 8500 unique temperature sensors) across all the world’s major terrestrial biomes, and coarse-grained air temperature estimates from ERA5-Land (an atmospheric reanalysis by the European Centre for Medium-Range Weather Forecasts). We show that mean annual soil temperature differs markedly from the corresponding gridded air temperature, by up to 10°C (mean = 3.0 ± 2.1°C), with substantial variation across biomes and seasons. Over the year, soils in cold and/or dry biomes are substantially warmer (+3.6 ± 2.3°C) than gridded air temperature, whereas soils in warm and humid environments are on average slightly cooler (-0.7 ± 2.3°C). The observed substantial and biome-specific offsets emphasize that the projected impacts of climate and climate change on near-surface biodiversity and ecosystem functioning are inaccurately assessed when air rather than soil temperature is used, especially in cold environments. The global soil-related bioclimatic variables provided here are an important step forward for any application in ecology and related disciplines. Nevertheless, we highlight the need to fill remaining geographic gaps by collecting more in-situ measurements of microclimate conditions to further enhance the spatiotemporal resolution of global soil temperature products for ecological applications

Multiple novel prostate cancer susceptibility signals identified by fine-mapping of known risk loci among Europeans

Genome-wide association studies (GWAS) have identified numerous common prostate cancer (PrCa) susceptibility loci. We have fine-mapped 64 GWAS regions known at the conclusion of the iCOGS study using large-scale genotyping and imputation in 25 723 PrCa cases and 26 274 controls of European ancestry. We detected evidence for multiple independent signals at 16 regions, 12 of which contained additional newly identified significant associations. A single signal comprising a spectrum of correlated variation was observed at 39 regions; 35 of which are now described by a novel more significantly associated lead SNP, while the originally reported variant remained as the lead SNP only in 4 regions. We also confirmed two association signals in Europeans that had been previously reported only in East-Asian GWAS. Based on statistical evidence and linkage disequilibrium (LD) structure, we have curated and narrowed down the list of the most likely candidate causal variants for each region. Functional annotation using data from ENCODE filtered for PrCa cell lines and eQTL analysis demonstrated significant enrichment for overlap with bio-features within this set. By incorporating the novel risk variants identified here alongside the refined data for existing association signals, we estimate that these loci now explain ∼38.9% of the familial relative risk of PrCa, an 8.9% improvement over the previously reported GWAS tag SNPs. This suggests that a significant fraction of the heritability of PrCa may have been hidden during the discovery phase of GWAS, in particular due to the presence of multiple independent signals within the same regio

Measurement of t(t)over-bar normalised multi-differential cross sections in pp collisions at root s=13 TeV, and simultaneous determination of the strong coupling strength, top quark pole mass, and parton distribution functions

Peer reviewe

An embedding technique to determine ττ backgrounds in proton-proton collision data

An embedding technique is presented to estimate standard model tau tau backgrounds from data with minimal simulation input. In the data, the muons are removed from reconstructed mu mu events and replaced with simulated tau leptons with the same kinematic properties. In this way, a set of hybrid events is obtained that does not rely on simulation except for the decay of the tau leptons. The challenges in describing the underlying event or the production of associated jets in the simulation are avoided. The technique described in this paper was developed for CMS. Its validation and the inherent uncertainties are also discussed. The demonstration of the performance of the technique is based on a sample of proton-proton collisions collected by CMS in 2017 at root s = 13 TeV corresponding to an integrated luminosity of 41.5 fb(-1).Peer reviewe

Measurement of the Splitting Function in &ITpp &ITand Pb-Pb Collisions at root&ITsNN&IT=5.02 TeV

Data from heavy ion collisions suggest that the evolution of a parton shower is modified by interactions with the color charges in the dense partonic medium created in these collisions, but it is not known where in the shower evolution the modifications occur. The momentum ratio of the two leading partons, resolved as subjets, provides information about the parton shower evolution. This substructure observable, known as the splitting function, reflects the process of a parton splitting into two other partons and has been measured for jets with transverse momentum between 140 and 500 GeV, in pp and PbPb collisions at a center-of-mass energy of 5.02 TeV per nucleon pair. In central PbPb collisions, the splitting function indicates a more unbalanced momentum ratio, compared to peripheral PbPb and pp collisions.. The measurements are compared to various predictions from event generators and analytical calculations.Peer reviewe

Electroweak production of two jets in association with a Z boson in proton-proton collisions root s =13 TeV

A measurement of the electroweak (EW) production of two jets in association with a Z boson in proton-proton collisions at root s = 13 TeV is presented, based on data recorded in 2016 by the CMS experiment at the LHC corresponding to an integrated luminosity of 35.9 fb(-1). The measurement is performed in the lljj final state with l including electrons and muons, and the jets j corresponding to the quarks produced in the hard interaction. The measured cross section in a kinematic region defined by invariant masses m(ll) > 50 GeV, m(jj) > 120 GeV, and transverse momenta P-Tj > 25 GeV is sigma(EW) (lljj) = 534 +/- 20 (stat) fb (syst) fb, in agreement with leading-order standard model predictions. The final state is also used to perform a search for anomalous trilinear gauge couplings. No evidence is found and limits on anomalous trilinear gauge couplings associated with dimension-six operators are given in the framework of an effective field theory. The corresponding 95% confidence level intervals are -2.6 <cwww/Lambda(2) <2.6 TeV-2 and -8.4 <cw/Lambda(2) <10.1 TeV-2. The additional jet activity of events in a signal-enriched region is also studied, and the measurements are in agreement with predictions.Peer reviewe

Measurement of nuclear modification factors of gamma(1S)), gamma(2S), and gamma(3S) mesons in PbPb collisions at root s(NN)=5.02 TeV

The cross sections for ϒ(1S), ϒ(2S), and ϒ(3S) production in lead-lead (PbPb) and proton-proton (pp) collisions at √sNN = 5.02 TeV have been measured using the CMS detector at the LHC. The nuclear modification factors, RAA, derived from the PbPb-to-pp ratio of yields for each state, are studied as functions of meson rapidity and transverse momentum, as well as PbPb collision centrality. The yields of all three states are found to be significantly suppressed, and compatible with a sequential ordering of the suppression, RAA(ϒ(1S)) > RAA(ϒ(2S)) > RAA(ϒ(3S)). The suppression of ϒ(1S) is larger than that seen at √sNN = 2.76 TeV, although the two are compatible within uncertainties. The upper limit on the RAA of ϒ(3S) integrated over pT, rapidity and centrality is 0.096 at 95% confidence level, which is the strongest suppression observed for a quarkonium state in heavy ion collisions to date. © 2019 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). Funded by SCOAP3.Peer reviewe