Automated Detection of Antenna Malfunctions in Large-N Interferometers: A case study With the Hydrogen Epoch of Reionization Array

We present a framework for identifying and flagging malfunctioning antennas in large radio interferometers. We outline two distinct categories of metrics designed to detect outliers along known failure modes of large arrays: cross-correlation metrics, based on all antenna pairs, and auto-correlation metrics, based solely on individual antennas. We define and motivate the statistical framework for all metrics used, and present tailored visualizations that aid us in clearly identifying new and existing systematics. We implement these techniques using data from 105 antennas in the Hydrogen Epoch of Reionization Array (HERA) as a case study. Finally, we provide a detailed algorithm for implementing these metrics as flagging tools on real data sets

Foreground modelling via Gaussian process regression: An application to HERA data

The key challenge in the observation of the redshifted 21-cm signal from cosmic reionization is its separation from the much brighter foreground emission. Such separation relies on the different spectral properties of the two components, although, in real life, the foreground intrinsic spectrum is often corrupted by the instrumental response, inducing systematic effects that can further jeopardize the measurement of the 21-cm signal. In this paper, we use Gaussian Process Regression to model both foreground emission and instrumental systematics in $\sim 2$ hours of data from the Hydrogen Epoch of Reionization Array. We find that a simple co-variance model with three components matches the data well, giving a residual power spectrum with white noise properties. These consist of an "intrinsic" and instrumentally corrupted component with a coherence-scale of 20 MHz and 2.4 MHz respectively (dominating the line of sight power spectrum over scales $k_{\parallel} \le 0.2$ h cMpc$^{-1}$) and a baseline dependent periodic signal with a period of $\sim 1$ MHz (dominating over $k_{\parallel} \sim 0.4 - 0.8$h cMpc$^{-1}$) which should be distinguishable from the 21-cm EoR signal whose typical coherence-scales is $\sim 0.8$ MHz

Foreground modelling via Gaussian process regression: An application to HERA data

The key challenge in the observation of the redshifted 21-cm signal from cosmic reionization is its separation from the much brighter foreground emission. Such separation relies on the different spectral properties of the two components, although, in real life, the foreground intrinsic spectrum is often corrupted by the instrumental response, inducing systematic effects that can further jeopardize the measurement of the 21-cm signal. In this paper, we use Gaussian Process Regression to model both foreground emission and instrumental systematics in $\sim 2$ hours of data from the Hydrogen Epoch of Reionization Array. We find that a simple co-variance model with three components matches the data well, giving a residual power spectrum with white noise properties. These consist of an "intrinsic" and instrumentally corrupted component with a coherence-scale of 20 MHz and 2.4 MHz respectively (dominating the line of sight power spectrum over scales $k_{\parallel} \le 0.2$ h cMpc$^{-1}$) and a baseline dependent periodic signal with a period of $\sim 1$ MHz (dominating over $k_{\parallel} \sim 0.4 - 0.8$h cMpc$^{-1}$) which should be distinguishable from the 21-cm EoR signal whose typical coherence-scales is $\sim 0.8$ MHz

Measuring HERA's Primary Beam in Situ: Methodology and First Results

The central challenge in 21 cm cosmology is isolating the cosmological signal from bright foregrounds. Many separation techniques rely on the accurate knowledge of the sky and the instrumental response, including the antenna primary beam. For drift-scan telescopes, such as the Hydrogen Epoch of Reionization Array (HERA), that do not move, primary beam characterization is particularly challenging because standard beam-calibration routines do not apply (Cornwell et al.) and current techniques require accurate source catalogs at the telescope resolution. We present an extension of the method from Pober et al. where they use beam symmetries to create a network of overlapping source tracks that break the degeneracy between source flux density and beam response and allow their simultaneous estimation. We fit the beam response of our instrument using early HERA observations and find that our results agree well with electromagnetic simulations down to a -20 dB level in power relative to peak gain for sources with high signal-to-noise ratio. In addition, we construct a source catalog with 90 sources down to a flux density of 1.4 Jy at 151 MHz

First Results from HERA Phase I: Upper Limits on the Epoch of Reionization 21 cm Power Spectrum

We report upper-limits on the Epoch of Reionization (EoR) 21 cm power spectrum at redshifts 7.9 and 10.4 with 18 nights of data ($\sim36$ hours of integration) from Phase I of the Hydrogen Epoch of Reionization Array (HERA). The Phase I data show evidence for systematics that can be largely suppressed with systematic models down to a dynamic range of $\sim10^9$ with respect to the peak foreground power. This yields a 95% confidence upper limit on the 21 cm power spectrum of $\Delta^2_{21} \le (30.76)^2\ {\rm mK}^2$ at $k=0.192\ h\ {\rm Mpc}^{-1}$at$z=7.9$, and also$\Delta^2_{21} \le (95.74)^2\ {\rm mK}^2$at$k=0.256\ h\ {\rm Mpc}^{-1}$at$z=10.4$. At$z=7.9$, these limits are the most sensitive to-date by over an order of magnitude. While we find evidence for residual systematics at low line-of-sight Fourier$k_\parallel$modes, at high$k_\parallel$ modes we find our data to be largely consistent with thermal noise, an indicator that the system could benefit from deeper integrations. The observed systematics could be due to radio frequency interference, cable sub-reflections, or residual instrumental cross-coupling, and warrant further study. This analysis emphasizes algorithms that have minimal inherent signal loss, although we do perform a careful accounting in a companion paper of the small forms of loss or bias associated with the pipeline. Overall, these results are a promising first step in the development of a tuned, instrument-specific analysis pipeline for HERA, particularly as Phase II construction is completed en route to reaching the full sensitivity of the experiment

Polarized redundant-baseline calibration for 21 cm cosmology without adding spectral structure

21 cm cosmology is a promising new probe of the evolution of visible matter in our universe, especially during the poorly constrained Cosmic Dawn and Epoch of Reionization. However, in order to separate the 21 cm signal from bright astrophysical foregrounds, we need an exquisite understanding of our telescopes so as to avoid adding spectral structure to spectrally smooth foregrounds. One powerful calibration method relies on repeated simultaneous measurements of the same interferometric baseline to solve for the sky signal and for instrumental parameters simultaneously. However, certain degrees of freedom are not constrained by asserting internal consistency between redundant measurements. In this paper, we review the origin of these degeneracies of redundant-baseline calibration and demonstrate how they can source unwanted spectral structure in our measurement and show how to eliminate that additional, artificial structure. We also generalize redundant calibration to dual-polarization instruments, derive the degeneracy structure, and explore the unique challenges to calibration and preserving spectral smoothness presented by a polarized measurement

Redundant-baseline calibration of the hydrogen epoch of reionization array

In 21-cm cosmology, precision calibration is key to the separation of the neutral hydrogen signal from very bright but spectrally smooth astrophysical foregrounds. The Hydrogen Epoch of Reionization Array (HERA), an interferometer specialized for 21-cm cosmology and now under construction in South Africa, was designed to be largely calibrated using the self-consistency of repeated measurements of the same interferometric modes. This technique, known as redundant-baseline calibration resolves most of the internal degrees of freedom in the calibration problem. It assumes, however, on antenna elements with identical primary beams placed precisely on a redundant grid. In this work, we review the detailed implementation of the algorithms enabling redundant-baseline calibration and report results with HERA data.We quantify the effects of real-world non-redundancy and how they compare to the idealized scenario in which redundant measurements differ only in their noise realizations. Finally, we study how non-redundancy can produce spurious temporal structure in our calibration solutions-both in data and in simulations-and present strategies for mitigating that structure

Absolute Calibration Strategies for the Hydrogen Epoch of Reionization Array and Their Impact on the 21 cm Power Spectrum

We discuss absolute calibration strategies for Phase I of the Hydrogen Epoch of Reionization Array (HERA), which aims to measure the cosmological 21 cm signal from the Epoch of Reionization (EoR). HERA is a drift-scan array with a 10 degree wide field of view, meaning bright, well-characterized point source transits are scarce. This, combined with HERA's redundant sampling of the uv plane and the modest angular resolution of the Phase I instrument, make traditional sky-based and self-calibration techniques difficult to implement with high dynamic range. Nonetheless, in this work we demonstrate calibration for HERA using point source catalogues and electromagnetic simulations of its primary beam. We show that unmodeled diffuse flux and instrumental contaminants can corrupt the gain solutions, and present a gain smoothing approach for mitigating their impact on the 21 cm power spectrum. We also demonstrate a hybrid sky and redundant calibration scheme and compare it to pure sky-based calibration, showing only a marginal improvement to the gain solutions at intermediate delay scales. Our work suggests that the HERA Phase I system can be well-calibrated for a foreground-avoidance power spectrum estimator by applying direction-independent gains with a small set of degrees of freedom across the frequency and time axes

The HERA-19 Commissioning Array: Direction-dependent Effects

Foreground power dominates the measurements of interferometers that seek a statistical detection of highly-redshifted HI emission from the Epoch of Reionization (EoR). The chromaticity of the instrument creates a boundary in the Fourier transform of frequency (proportional to $k_\parallel$) between spectrally smooth emission, characteristic of the strong synchrotron foreground (the "wedge"), and the spectrally structured emission from HI in the EoR (the "EoR window"). Faraday rotation can inject spectral structure into otherwise smooth polarized foreground emission, which through instrument effects or miscalibration could possibly pollute the EoR window. Using data from the HERA 19-element commissioning array, we investigate the polarization response of this new instrument in the power spectrum domain. We perform a simple image-based calibration based on the unpolarized diffuse emission of the Global Sky Model, and show that it achieves qualitative redundancy between the nominally-redundant baselines of the array and reasonable amplitude accuracy. We construct power spectra of all fully polarized coherencies in all pseudo-Stokes parameters. We compare to simulations based on an unpolarized diffuse sky model and detailed electromagnetic simulations of the dish and feed, confirming that in Stokes I, the calibration does not add significant spectral structure beyond the expected level. Further, this calibration is stable over the 8 days of observations considered. Excess power is seen in the power spectra of the linear polarization Stokes parameters which is not easily attributable to leakage via the primary beam, and results from some combination of residual calibration errors and actual polarized emission. Stokes V is found to be highly discrepant from the expectation of zero power, strongly pointing to the need for more accurate polarized calibration

Foreground modelling via Gaussian process regression: An application to HERA data

The key challenge in the observation of the redshifted 21-cm signal from cosmic reionization is its separation from the much brighter foreground emission. Such separation relies on the different spectral properties of the two components, although, in real life, the foreground intrinsic spectrum is often corrupted by the instrumental response, inducing systematic effects that can further jeopardize the measurement of the 21-cm signal. In this paper, we use Gaussian Process Regression to model both foreground emission and instrumental systematics in ∼2 h of data from the Hydrogen Epoch of Reionization Array. We find that a simple co-variance model with three components matches the data well, giving a residual power spectrum with white noise properties. These consist of an 'intrinsic' and instrumentally corrupted component with a coherence scale of 20 and 2.4 MHz, respectively (dominating the line-of-sight power spectrum over scales kâ ≤ 0.2 h cMpc-1) and a baseline-dependent periodic signal with a period of ∼1 MHz (dominating over kâ ∼0.4-0.8 h cMpc-1), which should be distinguishable from the 21-cm Epoch of Reionization signal whose typical coherence scale is ∼0.8 MHz