A Scalable Correlator Architecture Based on Modular FPGA Hardware, Reuseable Gateware, and Data Packetization

A new generation of radio telescopes is achieving unprecedented levels of sensitivity and resolution, as well as increased agility and field-of-view, by employing high-performance digital signal processing hardware to phase and correlate large numbers of antennas. The computational demands of these imaging systems scale in proportion to BMN^2, where B is the signal bandwidth, M is the number of independent beams, and N is the number of antennas. The specifications of many new arrays lead to demands in excess of tens of PetaOps per second. To meet this challenge, we have developed a general purpose correlator architecture using standard 10-Gbit Ethernet switches to pass data between flexible hardware modules containing Field Programmable Gate Array (FPGA) chips. These chips are programmed using open-source signal processing libraries we have developed to be flexible, scalable, and chip-independent. This work reduces the time and cost of implementing a wide range of signal processing systems, with correlators foremost among them,and facilitates upgrading to new generations of processing technology. We present several correlator deployments, including a 16-antenna, 200-MHz bandwidth, 4-bit, full Stokes parameter application deployed on the Precision Array for Probing the Epoch of Reionization.Comment: Accepted to Publications of the Astronomy Society of the Pacific. 31 pages. v2: corrected typo, v3: corrected Fig. 1

Data Provenance and Management in Radio Astronomy: A Stream Computing Approach

New approaches for data provenance and data management (DPDM) are required for mega science projects like the Square Kilometer Array, characterized by extremely large data volume and intense data rates, therefore demanding innovative and highly efficient computational paradigms. In this context, we explore a stream-computing approach with the emphasis on the use of accelerators. In particular, we make use of a new generation of high performance stream-based parallelization middleware known as InfoSphere Streams. Its viability for managing and ensuring interoperability and integrity of signal processing data pipelines is demonstrated in radio astronomy. IBM InfoSphere Streams embraces the stream-computing paradigm. It is a shift from conventional data mining techniques (involving analysis of existing data from databases) towards real-time analytic processing. We discuss using InfoSphere Streams for effective DPDM in radio astronomy and propose a way in which InfoSphere Streams can be utilized for large antennae arrays. We present a case-study: the InfoSphere Streams implementation of an autocorrelating spectrometer, and using this example we discuss the advantages of the stream-computing approach and the utilization of hardware accelerators

DiFX2: A more flexible, efficient, robust and powerful software correlator

Software correlation, where a correlation algorithm written in a high-level language such as C++ is run on commodity computer hardware, has become increasingly attractive for small to medium sized and/or bandwidth constrained radio interferometers. In particular, many long baseline arrays (which typically have fewer than 20 elements and are restricted in observing bandwidth by costly recording hardware and media) have utilized software correlators for rapid, cost-effective correlator upgrades to allow compatibility with new, wider bandwidth recording systems and improve correlator flexibility. The DiFX correlator, made publicly available in 2007, has been a popular choice in such upgrades and is now used for production correlation by a number of observatories and research groups worldwide. Here we describe the evolution in the capabilities of the DiFX correlator over the past three years, including a number of new capabilities, substantial performance improvements, and a large amount of supporting infrastructure to ease use of the code. New capabilities include the ability to correlate a large number of phase centers in a single correlation pass, the extraction of phase calibration tones, correlation of disparate but overlapping sub-bands, the production of rapidly sampled filterbank and kurtosis data at minimal cost, and many more. The latest version of the code is at least 15% faster than the original, and in certain situations many times this value. Finally, we also present detailed test results validating the correctness of the new code.Comment: 28 pages, 9 figures, accepted for publication in PAS

Real-Time Dedispersion for Fast Radio Transient Surveys, using Auto Tuning on Many-Core Accelerators

Dedispersion, the removal of deleterious smearing of impulsive signals by the interstellar matter, is one of the most intensive processing steps in any radio survey for pulsars and fast transients. We here present a study of the parallelization of this algorithm on many-core accelerators, including GPUs from AMD and NVIDIA, and the Intel Xeon Phi. We find that dedispersion is inherently memory-bound. Even in a perfect scenario, hardware limitations keep the arithmetic intensity low, thus limiting performance. We next exploit auto-tuning to adapt dedispersion to different accelerators, observations, and even telescopes. We demonstrate that the optimal settings differ between observational setups, and that auto-tuning significantly improves performance. This impacts time-domain surveys from Apertif to SKA.Comment: 8 pages, accepted for publication in Astronomy and Computin

Calibration Challenges for Future Radio Telescopes

Instruments for radio astronomical observations have come a long way. While the first telescopes were based on very large dishes and 2-antenna interferometers, current instruments consist of dozens of steerable dishes, whereas future instruments will be even larger distributed sensor arrays with a hierarchy of phased array elements. For such arrays to provide meaningful output (images), accurate calibration is of critical importance. Calibration must solve for the unknown antenna gains and phases, as well as the unknown atmospheric and ionospheric disturbances. Future telescopes will have a large number of elements and a large field of view. In this case the parameters are strongly direction dependent, resulting in a large number of unknown parameters even if appropriately constrained physical or phenomenological descriptions are used. This makes calibration a daunting parameter estimation task, that is reviewed from a signal processing perspective in this article.Comment: 12 pages, 7 figures, 20 subfigures The title quoted in the meta-data is the title after release / final editing

The Murchison Widefield Array: Design Overview

The Murchison Widefield Array (MWA) is a dipole-based aperture array synthesis telescope designed to operate in the 80-300 MHz frequency range. It is capable of a wide range of science investigations, but is initially focused on three key science projects. These are detection and characterization of 3-dimensional brightness temperature fluctuations in the 21cm line of neutral hydrogen during the Epoch of Reionization (EoR) at redshifts from 6 to 10, solar imaging and remote sensing of the inner heliosphere via propagation effects on signals from distant background sources,and high-sensitivity exploration of the variable radio sky. The array design features 8192 dual-polarization broad-band active dipoles, arranged into 512 tiles comprising 16 dipoles each. The tiles are quasi-randomly distributed over an aperture 1.5km in diameter, with a small number of outliers extending to 3km. All tile-tile baselines are correlated in custom FPGA-based hardware, yielding a Nyquist-sampled instantaneous monochromatic uv coverage and unprecedented point spread function (PSF) quality. The correlated data are calibrated in real time using novel position-dependent self-calibration algorithms. The array is located in the Murchison region of outback Western Australia. This region is characterized by extremely low population density and a superbly radio-quiet environment,allowing full exploitation of the instrumental capabilities.Comment: 9 pages, 5 figures, 1 table. Accepted for publication in Proceedings of the IEE