Increasing the Action Gap: New Operators for Reinforcement Learning

This paper introduces new optimality-preserving operators on Q-functions. We first describe an operator for tabular representations, the consistent Bellman operator, which incorporates a notion of local policy consistency. We show that this local consistency leads to an increase in the action gap at each state; increasing this gap, we argue, mitigates the undesirable effects of approximation and estimation errors on the induced greedy policies. This operator can also be applied to discretized continuous space and time problems, and we provide empirical results evidencing superior performance in this context. Extending the idea of a locally consistent operator, we then derive sufficient conditions for an operator to preserve optimality, leading to a family of operators which includes our consistent Bellman operator. As corollaries we provide a proof of optimality for Baird's advantage learning algorithm and derive other gap-increasing operators with interesting properties. We conclude with an empirical study on 60 Atari 2600 games illustrating the strong potential of these new operators

Reduction of quartz to silicon monoxide by methane-hydrogen mixtures

The reduction of quartz was studied isothermally in a fluidized bed reactor using continuously flowing methane-hydrogen gas mixture in the temperature range from 1623 K to 1773 K (1350 °C to 1500 °C). The CO content in the off-gas was measured online using an infrared gas analyzer. The main phases of the reduced samples identified by XRD analysis were quartz and cristobalite. Significant weight loss in the reduction process indicated that the reduction products were SiO and CO. Reduction of SiO2 to SiO by methane starts with adsorption and dissociation of CH4 on the silica surface. The high carbon activity in the CH4-H2 gas mixture provided a strongly reducing condition. At 1623 K (1350 °C), the reduction was very slow. The rate and extent of reduction increased with the increasing temperature to 1723 K (1450 °C). A further increase in temperature to 1773 K (1500 °C) resulted in a decrease in the rate and extent of reduction. An increase in the gas flow rate from 0.4 to 0.8 NL/min and an increase in the methane content in the CH4-H2 gas mixture from 0 to 5 vol pct facilitated the reduction. Methane content in the gas mixture should be maintained at less than 5 vol pct in order to suppress methane cracking

Discovery of two gravitationally lensed quasars in the Dark Energy Survey

We present spectroscopic confirmation of two new gravitationally lensed quasars, discovered in the Dark Energy Survey (DES) and Wide-field Infrared Survey Explorer (WISE) based on their multiband photometry and extended morphology in DES images. Images of DES J0115−5244 show a red galaxy with two blue point sources at either side, which are images of the same quasar at zs = 1.64 as obtained by our long-slit spectroscopic data. The Einstein radius estimated from the DES images is 0.51 arcsec. DES J2146−0047 is in the area of overlap between DES and the Sloan Digital Sky Survey (SDSS). Two blue components are visible in the DES and SDSS images. The SDSS fibre spectrum shows a quasar component at zs = 2.38 and absorption by Mg ii and Fe ii at zl = 0.799, which we tentatively associate with the foreground lens galaxy. Our long-slit spectra show that the blue components are resolved images of the same quasar. The Einstein radius is 0.68 arcsec, corresponding to an enclosed mass of 1.6 × 1011 Mȯ. Three other candidates were observed and rejected, two being low-redshift pairs of starburst galaxies, and one being a quasar behind a blue star. These first confirmation results provide an important empirical validation of the data mining and model-based selection that is being applied to the entire DES data set

Toward an internally consistent astronomical distance scale

Accurate astronomical distance determination is crucial for all fields in astrophysics, from Galactic to cosmological scales. Despite, or perhaps because of, significant efforts to determine accurate distances, using a wide range of methods, tracers, and techniques, an internally consistent astronomical distance framework has not yet been established. We review current efforts to homogenize the Local Group's distance framework, with particular emphasis on the potential of RR Lyrae stars as distance indicators, and attempt to extend this in an internally consistent manner to cosmological distances. Calibration based on Type Ia supernovae and distance determinations based on gravitational lensing represent particularly promising approaches. We provide a positive outlook to improvements to the status quo expected from future surveys, missions, and facilities. Astronomical distance determination has clearly reached maturity and near-consistency.Comment: Review article, 59 pages (4 figures); Space Science Reviews, in press (chapter 8 of a special collection resulting from the May 2016 ISSI-BJ workshop on Astronomical Distance Determination in the Space Age

DES J0454-4448: discovery of the first luminous z >= 6 quasar from the Dark Energy Survey

We present the first results of a survey for high-redshift, z >= 6, quasars using izY multicolour photometric observations from the Dark Energy Survey (DES). Here we report the discovery and spectroscopic confirmation of the zAB, YAB = 20.2, 20.2 (M1450 = -26.5) quasar DES J0454-4448 with a redshift of z = 6.09±0.02 based on the onset of the Ly alpha forest and an H I near zone size of 4.1_{-1.2}^{+1.1} proper Mpc. The quasar was selected as an i-band drop out with i-z = 2.46 and zAB 6 including 3-10 with z > 7 dramatically increasing the numbers of quasars currently known that are suitable for detailed studies

VDES J2325-5229 a z=2.7 gravitationally lensed quasar discovered using morphology independent supervised machine learning

We present the discovery and preliminary characterization of a gravitationally lensed quasar with a source redshift $\textit{zs}$ = 2.74 and image separation of 2.9 arcsec lensed by a foreground $\textit{zl}$ = 0.40 elliptical galaxy. Since optical observations of gravitationally lensed quasars show the lens system as a superposition of multiple point sources and a foreground lensing galaxy, we have developed a morphology-independent multi-wavelength approach to the photometric selection of lensed quasar candidates based on Gaussian Mixture Models (GMM) supervised machine learning. Using this technique and $\textit{gi}$ multicolour photometric observations from the Dark Energy Survey (DES), near-IR $\textit{JK}$ photometry from the VISTA Hemisphere Survey (VHS) and WISE mid-IR photometry, we have identified a candidate system with two catalogue components with $\textit{iAB}$ = 18.61 and $\textit{iAB}$ = 20.44 comprising an elliptical galaxy and two blue point sources. Spectroscopic follow-up with NTT and the use of an archival AAT spectrum show that the point sources can be identified as a lensed quasar with an emission line redshift of $\textit{z}$ = 2.739 ± 0.003 and a foreground early-type galaxy with $\textit{z}$ = 0.400 ± 0.002. We model the system as a single isothermal ellipsoid and find the Einstein radius θE ∼ 1.47 arcsec, enclosed mass $\textit{M}$enc ∼ 4 × 10$^{11}$$\textit{M}$⊙ and a time delay of ∼52 d. The relatively wide separation, month scale time delay duration and high redshift make this an ideal system for constraining the expansion rate beyond a redshift of 1.FO is supported jointly by CAPES (the Science without Borders programme) and the Cambridge Commonwealth Trust. RGM, CAL, MWA, MB, SLR acknowledge the support of UK Science and Technology Research Council (STFC). AJC acknowledges the support of a Raymond and Beverly Sackler visiting fellowship at the Institute of Astronomy. For further information regarding funding please visit the publisher's website

Models of the strongly lensed quasar DES J0408−5354

We present detailed modelling of the recently discovered, quadruply lensed quasar J0408−5354, with the aim of interpreting its remarkable configuration: besides three quasar images (A,B,D) around the main deflector (G1), a fourth image (C) is significantly reddened and dimmed by a perturber (G2) which is not detected in the Dark Energy Survey imaging data. From lens models incorporating (dust-corrected) flux ratios, we find a perturber Einstein radius 0.04 arcsec ≲ RE, G2 ≲ 0.2 arcsec and enclosed mass Mp(RE, G2) ≲ 1.0 × 1010 M⊙. The main deflector has stellar mass log10(M⋆/M⊙)=11.49+0.46−0.32, a projected mass Mp(RE, G1) ≈ 6 × 1011M within its Einstein radius RE, G1 = (1.85 ± 0.15) arcsec and predicted velocity dispersion 267–280 km s−1. Follow-up images from a companion monitoring campaign show additional components, including a candidate second source at a redshift between the quasar and G1. Models with free perturbers, and dust-corrected and delay-corrected flux ratios, are also explored. The predicted time-delays (ΔtAB = (135.0 ± 12.6) d, ΔtBD = (21.0 ± 3.5) d) roughly agree with those measured, but better imaging is required for proper modelling and comparison. We also discuss some lessons learnt from J0408−5354  on lensed quasar finding strategies, due to its chromaticity and morphology

Erratum: Is every strong lens model unhappy in its own way? Uniform modelling of a sample of 13 quadruply+ imaged quasars

The paper ‘Is every strong lens model unhappy in its own way? Uniform modelling of a sample of 13 quadruply+ imaged quasars’ was published in MNRAS, 483, 4, 5649–5671 (2019). The coordinate values of the image positions in table 4 were wrongly printed due to a clerical error. At a later stage of writing the manuscript, we have changed the zero-point definition of the lens coordinate systems, but the relative image positions were not accounted for this change of definition while printing out table 4. We provide the updated Table 4 below. This error does not impact any other results of the paper in any way, except for the table itself. We thank Collin Werner and Paul Schechter for helping us identify this error