Sample-Efficient Model-Free Reinforcement Learning with Off-Policy Critics

Value-based reinforcement-learning algorithms provide state-of-the-art results in model-free discrete-action settings, and tend to outperform actor-critic algorithms. We argue that actor-critic algorithms are limited by their need for an on-policy critic. We propose Bootstrapped Dual Policy Iteration (BDPI), a novel model-free reinforcement-learning algorithm for continuous states and discrete actions, with an actor and several off-policy critics. Off-policy critics are compatible with experience replay, ensuring high sample-efficiency, without the need for off-policy corrections. The actor, by slowly imitating the average greedy policy of the critics, leads to high-quality and state-specific exploration, which we compare to Thompson sampling. Because the actor and critics are fully decoupled, BDPI is remarkably stable, and unusually robust to its hyper-parameters. BDPI is significantly more sample-efficient than Bootstrapped DQN, PPO, and ACKTR, on discrete, continuous and pixel-based tasks. Source code: https://github.com/vub-ai-lab/bdpi.Comment: Accepted at the European Conference on Machine Learning 2019 (ECML

The current use, and opinions of elite athletes and support staff in relation to genetic testing in elite sport within the UK

The purpose of the study was to investigate the current use of genetic testing in UK elite sport and assess how genetic testing might be received by those employed in elite sport. Seventy-two elite athletes and 95 support staff at UK sports clubs and governing bodies completed an online survey of 11 questions concerning their experience of genetic testing and beliefs regarding the use of genetic testing in sport. Genetic testing related to sports performance and injury susceptibility is conducted in UK elite sport, albeit by a relatively small proportion of athletes (≤17%) and support staff (≤8%). Athletes and their support staff agree that genetics are important in determining elite status (≥79%) and appear willing to engage in genetic testing for individualising training to improve sport performance and reduce injury risk. Opinion was divided on whether genetic information should be used to identify talented athletes and influence selection, eligibility or employment status. Genetic testing for sports performance and injury susceptibility occurs in UK elite sport, however it is not commonly conducted. There is a belief that genetics is an important factor in determining an athlete and there is a willingness to engage in genetic testing for sports performance and injury susceptibility

On the use of radon for quantifying the effects of atmospheric stability on urban emissions

Radon is increasingly being used as a tool for quantifying stability influences on urban pollutant concentrations. Bulk radon gradients are ideal for this purpose, since the vertical differencing substantially removes contributions from processes on timescales greater than diurnal and (assuming a constant radon source) gradients are directly related to the intensity of nocturnal mixing. More commonly, however, radon measurements are available only at a single height. In this study we argue that single-height radon observations should not be used quantitatively as an indicator of atmospheric stability without prior conditioning of the time series to remove contributions from larger-scale "non-local" processes. We outline a simple technique to obtain an approximation of the diurnal radon gradient signal from a single-height measurement time series, and use it to derive a four category classification scheme for atmospheric stability on a "whole night" basis. A selection of climatological and pollution observations in the Sydney region are then subdivided according to the radon-based scheme on an annual and seasonal basis. We compare the radon-based scheme against a commonly used Pasquill–Gifford (P–G) type stability classification and reveal that the most stable category in the P–G scheme is less selective of the strongly stable nights than the radon-based scheme; this lead to significant underestimation of pollutant concentrations on the most stable nights by the P–G scheme. Lastly, we applied the radon-based classification scheme to mixing height estimates calculated from the diurnal radon accumulation time series, which provided insight to the range of nocturnal mixing depths expected at the site for each of the stability classes. © 2015, Author(s)

Do PTK2 gene polymorphisms contribute to the interindividual variability in muscle strength and the response to resistance training? A preliminary report.

The protein tyrosine kinase-2 (PTK2) gene encodes focal adhesion kinase, a structural protein involved in lateral transmission of muscle fiber force. We investigated whether single-nucleotide polymorphisms (SNPs) of the PTK2 gene were associated with various indexes of human skeletal muscle strength and the interindividual variability in the strength responses to resistance training. We determined unilateral knee extension single repetition maximum (1-RM), maximum isometric voluntary contraction (MVC) knee joint torque, and quadriceps femoris muscle specific force (maximum force per unit physiological cross-sectional area) before and after 9 wk of knee extension resistance training in 51 untrained young men. All participants were genotyped for the PTK2 intronic rs7843014 A/C and 3'-untranslated region (UTR) rs7460 A/T SNPs. There were no genotype associations with baseline measures or posttraining changes in 1-RM or MVC. Although the training-induced increase in specific force was similar for all PTK2 genotypes, baseline specific force was higher in PTK2 rs7843014 AA and rs7460 TT homozygotes than in the respective rs7843014 C- (P = 0.016) and rs7460 A-allele (P = 0.009) carriers. These associations between muscle specific force and PTK2 SNPs suggest that interindividual differences exist in the way force is transmitted from the muscle fibers to the tendon. Therefore, our results demonstrate for the first time the impact of genetic variation on the intrinsic strength of human skeletal muscle

Next-to-minimal SOFTSUSY

We describe an extension to the SOFTSUSY program that provides for the calculation of the sparticle spectrum in the Next-to-Minimal Supersymmetric Standard Model (NMSSM), where a chiral superfield that is a singlet of the Standard Model gauge group is added to the Minimal Supersymmetric Standard Model (MSSM) fields. Often, a $\mathbb{Z}_{3}$ symmetry is imposed upon the model. SOFTSUSY can calculate the spectrum in this case as well as the case where general $\mathbb{Z}_{3}$ violating (denoted as $\,\mathbf{\backslash}\mkern-11.0mu{\mathbb{Z}}_{3}$) terms are added to the soft supersymmetry breaking terms and the superpotential. The user provides a theoretical boundary condition for the couplings and mass terms of the singlet. Radiative electroweak symmetry breaking data along with electroweak and CKM matrix data are used as weak-scale boundary conditions. The renormalisation group equations are solved numerically between the weak scale and a high energy scale using a nested iterative algorithm. This paper serves as a manual to the NMSSM mode of the program, detailing the approximations and conventions used

Improved mixing height monitoring through a combination of lidar and radon measurements

Surface-based radon (222Rn) measurements can be combined with lidar backscatter to obtain a higher quality time series of mixing height within the planetary boundary layer (PBL) than is possible from lidar alone, and a more quantitative measure of mixing height than is possible from only radon. The reason why lidar measurements are improved is that there are times when lidar signals are ambiguous, and reliably attributing the mixing height to the correct aerosol layer presents a challenge. By combining lidar with a mixing length scale derived from a time series of radon concentration, automated and robust attribution is possible during the morning transition. Radon measurements provide mixing information during the night, but concentrations also depend on the strength of surface emissions. After processing radon in combination with lidar, we obtain nightly measurements of radon emissions and are able to normalise the mixing length scale for changing emissions. After calibration with lidar, the radonderived equivalent mixing height agrees with other measures of mixing on daily and hourly timescales and is a potential method for studying intermittent mixing in nocturnal boundary layers.© 2013, Copernicus Publications

Radon-based assessment of stability effects on potential radiological releases

It is a requirement of nuclear energy and research facilities to conduct continuous and comprehensive atmospheric monitoring in order to better forecast public or environmental exposure to routine or accidental releases of radioactive substances to the atmosphere. A key aspect of such monitoring programs is the assessment of the atmospheric mixing state (or “stability”). Whether these facilities are in dense urban areas, or surrounded by heavily vegetated exclusion zones, local roughness heterogeneity can hamper attempts to accurately categorise stability by conventional meteorological techniques. Based on an analysis of 8 months of hourly climatology and atmospheric radon observations from a 60 m tower at the IFIN-HH nuclear research facility (Bucharest, Romania), we develop and apply a continuous (i.e. not categorical) radon-based scheme for the classification of the nocturnal atmospheric stability state. We demonstrate the superior performance of the radon-based technique to Pasquill-Gifford or bulk Richardson number stability typing at this site where heterogeneous roughness elements reach to 15 m a.g.l. Under stable nocturnal conditions the Pasquill-Gifford scheme overestimates the atmosphere’s capacity to dilute pollutants with near-surface sources by 20% compared to the radon-based scheme. Under these conditions, near-surface wind speeds drop well below 1 m s-1 and nocturnal mixing depths vary from ~25 m to less than 10 m a.g.l. Climatological parameters are characterised by season and 4 arbitrarily-defined nocturnal stability categories. Benchmarks (based on 10/50/90th percentile distributions) of 30-60 m wind and temperature gradients are devised for each stability category for evaluation of model performance. Lastly, nocturnal radon-derived effective mixing depth estimates constrained by tower observations are used to better-constrain the seasonal variability in the Bucharest regional radon flux: 13 mBq m-2 s-1 (winter), 18 mBq m-2 s-1 (summer)

Characterising terrestrial influences on Antarctic air masses using Radon-222 measurements at King George Island

We report on one year of high-precision direct hourly radon observations at King Sejong Station (King George Island) beginning in February 2013. Findings are compared with historic and ongoing radon measurements from other Antarctic sites. Monthly median concentrations reduced from 72 mBq m−3 in late-summer to 44 mBq m−3 in late winter and early spring. Monthly 10th percentiles, ranging from 29 to 49 mBq m−3, were typical of oceanic baseline values. Diurnal cycles were rarely evident and local influences were minor, consistent with regional radon flux estimates one tenth of the global average for ice-free land. The predominant fetch region for terrestrially influenced air masses was South America (47–53° S), with minor influences also attributed to aged Australian air masses and local sources. Plume dilution factors of 2.8–4.0 were estimated for the most terrestrially influenced (South American) air masses, and a seasonal cycle in terrestrial influence on tropospheric air descending at the pole was identified and characterised. © Author(s) 201

Field-enhanced direct tunneling in ultrathin atomic-layer-deposition-grown Au-Al2O3-Cr metal-insulator-metal structures

Metal-insulator-metal structures based on ultrathin high-k dielectric films are underpinning a rapidly increasing number of devices and applications. Here, we report detailed electrical characterizations of asymmetric metal-insulator-metal devices featuring atomic layer deposited 2-nm-thick Al2O3 films. We find a high consistency in the current density as a function of applied electric field between devices with very different surface areas and significant asymmetries in the IV characteristics. We show by TEM that the thickness of the dielectric film and the quality of the metal-insulator interfaces are highly uniform and of high quality, respectively. In addition, we develop a model which accounts for the field enhancement due to the small sharp features on the electrode surface and show that this can very accurately describe the observed asymmetry in the current-voltage characteristic, which cannot be explained by the difference in work function alone

QCD Approach to B->D \pi Decays and CP Violation

The branching ratios and CP violations of the $B\to D\pi$ decays, including both the color-allowed and the color-suppressed modes, are investigated in detail within QCD framework by considering all diagrams which lead to three effective currents of two quarks. An intrinsic mass scale as a dynamical gluon mass is introduced to treat the infrared divergence caused by the soft collinear approximation in the endpoint regions, and the Cutkosky rule is adopted to deal with a physical-region singularity of the on mass-shell quark propagators. When the dynamical gluon mass $\mu_g$ is regarded as a universal scale, it is extracted to be around $\mu_g = 440$ MeV from one of the well-measured $B\to D\pi$ decay modes. The resulting predictions for all branching ratios are in agreement with the current experimental measurements. As these decays have no penguin contributions, there are no direct $CP$ asymmetries. Due to interference between the Cabibbo-suppressed and the Cabibbo-favored amplitudes, mixing-induced CP violations are predicted in the $B\to D^{\pm}\pi^{\mp}$ decays to be consistent with the experimental data at 1-$\sigma$ level. More precise measurements will be helpful to extract weak angle $2\beta+\gamma$.Comment: 21pages,5 figures,3 tables, typos corrected and numerical result for one of decay channels is improve