Human-level Atari 200x faster

The task of building general agents that perform well over a wide range of tasks has been an important goal in reinforcement learning since its inception. The problem has been subject of research of a large body of work, with performance frequently measured by observing scores over the wide range of environments contained in the Atari 57 benchmark. Agent57 was the first agent to surpass the human benchmark on all 57 games, but this came at the cost of poor data-efficiency, requiring nearly 80 billion frames of experience to achieve. Taking Agent57 as a starting point, we employ a diverse set of strategies to achieve a 200-fold reduction of experience needed to out perform the human baseline. We investigate a range of instabilities and bottlenecks we encountered while reducing the data regime, and propose effective solutions to build a more robust and efficient agent. We also demonstrate competitive performance with high-performing methods such as Muesli and MuZero. The four key components to our approach are (1) an approximate trust region method which enables stable bootstrapping from the online network, (2) a normalisation scheme for the loss and priorities which improves robustness when learning a set of value functions with a wide range of scales, (3) an improved architecture employing techniques from NFNets in order to leverage deeper networks without the need for normalization layers, and (4) a policy distillation method which serves to smooth out the instantaneous greedy policy overtime

Masses of Neutron Stars in High-Mass X-ray Binaries with Optical Astrometry

Determining the type of matter that is inside a neutron star (NS) has been a long-standing goal of astrophysics. Despite this, most of the NS equations of state (EOS) that predict maximum masses in the range 1.4-2.8 solar masses are still viable. Most of the precise NS mass measurements that have been made to date show values close to 1.4 solar masses, but a reliable measurement of an over-massive NS would constrain the EOS possibilities. Here, we investigate how optical astrometry at the microarcsecond level can be used to map out the orbits of High-Mass X-ray Binaries (HMXBs), leading to tight constraints on NS masses. While previous studies by Unwin and co-workers and Tomsick and co-workers discuss the fact that the future Space Interferometry Mission should be capable of making such measurements, the current work describes detailed simulations for 6 HMXB systems, including predicted constraints on all orbital parameters. We find that the direct NS masses can be measured to an accuracy of 2.5% (1-sigma) in the best case (X Per), to 6.5% for Vela X-1, and to 10% for two other HMXBs.Comment: 8 pages, Accepted by Ap

A comparison of A-level performance in economics and business studies: how much more difficult is economics?

This paper uses ALIS data to compare academic performance in two subjects often viewed as relatively close substitutes for one another at A-level. The important role of GCSE achievement is confirmed for both subjects. There is evidence of strong gender effects and variation in outcomes across Examination Boards. A counterfactual exercise suggests that if the sample of Business Studies candidates had studied Economics nearly 40% of those who obtained a grade C or better in the former subject would not have done so in the latter. The opposite exercise uggests that 12% more Economics candidates would have achieved a grade C or better if they had taken Business Studies. In order to render a Business Studies A-level grade comparable to an Economics one in terms of relative difficulty, we estimate that a downward adjustment of 1.5 UCAS points should be applied to the former subject. This adjustment is lower than that suggested by correction factors based on conventional subject pair analysis for these two subjects

The High-Density Ionized Gas in the Central Parsecs of the Galaxy

We report the results from observations of H30$\alpha$ line emission in Sgr A West with the Submillimeter Array at a resolution of 2\arcsec and a field of view of about 40\arcsec. The H30$\alpha$ line is sensitive to the high-density ionized gas in the minispiral structure. We compare the velocity field obtained from H30$\alpha$ line emission to a Keplerian model, and our results suggest that the supermassive black hole at Sgr A* dominates the dynamics of the ionized gas. However, we also detect significant deviations from the Keplerian motion, which show that the impact of strong stellar winds from the massive stars along the ionized flows and the interaction between Northern and Eastern arms play significant roles in the local gas dynamics.Comment: 4 pages, 2 figure

Clustering between high-mass X-ray binaries and OB associations in the Milky Way

We present the first direct measurement of the spatial cross-correlation function of high-mass X-ray binaries (HMXBs) and active OB star-forming complexes in the Milky Way. This result relied on a sample containing 79 hard X-ray selected HMXBs and 458 OB associations. Clustering between the two populations is detected with a significance above 7-sigmas for distances < 1 kpc. Thus, HMXBs closely trace the underlying distribution of the massive star-forming regions that are expected to produce the progenitor stars of HMXBs. The average offset of 0.4+-0.2 kpc between HMXBs and OB associations is consistent with being due to natal kicks at velocities of the order of 100+-50 km/s. The characteristic scale of the correlation function suggests an average kinematical age (since the supernova phase) of ~4 Myr for the HMXB population. Despite being derived from a global view of our Galaxy, these signatures of HMXB evolution are consistent with theoretical expectations as well as observations of individual objects.Comment: 18 pages, 10 figures, 3 tables, accepted for publication in Ap

Parity Violating Measurements of Neutron Densities

Parity violating electron nucleus scattering is a clean and powerful tool for measuring the spatial distributions of neutrons in nuclei with unprecedented accuracy. Parity violation arises from the interference of electromagnetic and weak neutral amplitudes, and the $Z^0$ of the Standard Model couples primarily to neutrons at low $Q^2$. The data can be interpreted with as much confidence as electromagnetic scattering. After briefly reviewing the present theoretical and experimental knowledge of neutron densities, we discuss possible parity violation measurements, their theoretical interpretation, and applications. The experiments are feasible at existing facilities. We show that theoretical corrections are either small or well understood, which makes the interpretation clean. The quantitative relationship to atomic parity nonconservation observables is examined, and we show that the electron scattering asymmetries can be directly applied to atomic PNC because the observables have approximately the same dependence on nuclear shape.Comment: 38 pages, 7 ps figures, very minor changes, submitted to Phys. Rev.

1.3 mm Wavelength VLBI of Sagittarius A*: Detection of Time-Variable Emission on Event Horizon Scales

Sagittarius A*, the ~4 x 10^6 solar mass black hole candidate at the Galactic Center, can be studied on Schwarzschild radius scales with (sub)millimeter wavelength Very Long Baseline Interferometry (VLBI). We report on 1.3 mm wavelength observations of Sgr A* using a VLBI array consisting of the JCMT on Mauna Kea, the ARO/SMT on Mt. Graham in Arizona, and two telescopes of the CARMA array at Cedar Flat in California. Both Sgr A* and the quasar calibrator 1924-292 were observed over three consecutive nights, and both sources were clearly detected on all baselines. For the first time, we are able to extract 1.3 mm VLBI interferometer phase information on Sgr A* through measurement of closure phase on the triangle of baselines. On the third night of observing, the correlated flux density of Sgr A* on all VLBI baselines increased relative to the first two nights, providing strong evidence for time-variable change on scales of a few Schwarzschild radii. These results suggest that future VLBI observations with greater sensitivity and additional baselines will play a valuable role in determining the structure of emission near the event horizon of Sgr A*.Comment: 8 pages, submitted to ApJ

Atomic Parity Nonconservation: Electroweak Parameters and Nuclear Structure

There have been suggestions to measure atomic parity nonconservation (PNC) along an isotopic chain, by taking ratios of observables in order to cancel complicated atomic structure effects. Precise atomic PNC measurements could make a significant contribution to tests of the Standard Model at the level of one loop radiative corrections. However, the results also depend upon certain features of nuclear structure, such as the spatial distribution of neutrons in the nucleus. To examine the sensitivity to nuclear structure, we consider the case of Pb isotopes using various recent relativistic and non-relativistic nuclear model calculations. Contributions from nucleon internal weak structure are included, but found to be fairly negligible. The spread among present models in predicted sizes of nuclear structure effects may preclude using Pb isotope ratios to test the Standard Model at better than a one percent level, unless there are adequate independent tests of the nuclear models by various alternative strong and electroweak nuclear probes. On the other hand, sufficiently accurate atomic PNC experiments would provide a unique method to measure neutron distributions in heavy nuclei.Comment: 44 pages, INT Preprint DOE/ER/40561-050-INT92-00-1