DiFX2: A more flexible, efficient, robust and powerful software correlator

Software correlation, where a correlation algorithm written in a high-level language such as C++ is run on commodity computer hardware, has become increasingly attractive for small to medium sized and/or bandwidth constrained radio interferometers. In particular, many long baseline arrays (which typically have fewer than 20 elements and are restricted in observing bandwidth by costly recording hardware and media) have utilized software correlators for rapid, cost-effective correlator upgrades to allow compatibility with new, wider bandwidth recording systems and improve correlator flexibility. The DiFX correlator, made publicly available in 2007, has been a popular choice in such upgrades and is now used for production correlation by a number of observatories and research groups worldwide. Here we describe the evolution in the capabilities of the DiFX correlator over the past three years, including a number of new capabilities, substantial performance improvements, and a large amount of supporting infrastructure to ease use of the code. New capabilities include the ability to correlate a large number of phase centers in a single correlation pass, the extraction of phase calibration tones, correlation of disparate but overlapping sub-bands, the production of rapidly sampled filterbank and kurtosis data at minimal cost, and many more. The latest version of the code is at least 15% faster than the original, and in certain situations many times this value. Finally, we also present detailed test results validating the correctness of the new code.Comment: 28 pages, 9 figures, accepted for publication in PAS

A Scalable Correlator Architecture Based on Modular FPGA Hardware, Reuseable Gateware, and Data Packetization

A new generation of radio telescopes is achieving unprecedented levels of sensitivity and resolution, as well as increased agility and field-of-view, by employing high-performance digital signal processing hardware to phase and correlate large numbers of antennas. The computational demands of these imaging systems scale in proportion to BMN^2, where B is the signal bandwidth, M is the number of independent beams, and N is the number of antennas. The specifications of many new arrays lead to demands in excess of tens of PetaOps per second. To meet this challenge, we have developed a general purpose correlator architecture using standard 10-Gbit Ethernet switches to pass data between flexible hardware modules containing Field Programmable Gate Array (FPGA) chips. These chips are programmed using open-source signal processing libraries we have developed to be flexible, scalable, and chip-independent. This work reduces the time and cost of implementing a wide range of signal processing systems, with correlators foremost among them,and facilitates upgrading to new generations of processing technology. We present several correlator deployments, including a 16-antenna, 200-MHz bandwidth, 4-bit, full Stokes parameter application deployed on the Precision Array for Probing the Epoch of Reionization.Comment: Accepted to Publications of the Astronomy Society of the Pacific. 31 pages. v2: corrected typo, v3: corrected Fig. 1

Technology for Submillimeter Astronomy

Despite about three decades of progress, the field of submillimeter astronomy remains quite challenging, because the detection technology is still under development and the transmission of the atmosphere is poor. The latter problem has been overcome by constructing submillimeter telescopes at excellent sites, first on Mauna Kea and later in Chile and Antarctica, and also by using airborne and space telescopes. Meanwhile, the improvements in technology over the past several decades have been remarkable. While considerable opportunities for improvement remain, existing detector and receiver technologies now often approach fundamental limits. This technological revolution has brought submillimeter astronomy from the fringes to the forefront of modern astrophysics and has stimulated major investments such as the 50-element ALMA interferometer and the ESA/NASA Herschel Space Observatory

Hydrogen Epoch of Reionization Array (HERA)

The Hydrogen Epoch of Reionization Array (HERA) is a staged experiment to measure 21 cm emission from the primordial intergalactic medium (IGM) throughout cosmic reionization ($z=6-12$), and to explore earlier epochs of our Cosmic Dawn ($z\sim30$). During these epochs, early stars and black holes heated and ionized the IGM, introducing fluctuations in 21 cm emission. HERA is designed to characterize the evolution of the 21 cm power spectrum to constrain the timing and morphology of reionization, the properties of the first galaxies, the evolution of large-scale structure, and the early sources of heating. The full HERA instrument will be a 350-element interferometer in South Africa consisting of 14-m parabolic dishes observing from 50 to 250 MHz. Currently, 19 dishes have been deployed on site and the next 18 are under construction. HERA has been designated as an SKA Precursor instrument. In this paper, we summarize HERA's scientific context and provide forecasts for its key science results. After reviewing the current state of the art in foreground mitigation, we use the delay-spectrum technique to motivate high-level performance requirements for the HERA instrument. Next, we present the HERA instrument design, along with the subsystem specifications that ensure that HERA meets its performance requirements. Finally, we summarize the schedule and status of the project. We conclude by suggesting that, given the realities of foreground contamination, current-generation 21 cm instruments are approaching their sensitivity limits. HERA is designed to bring both the sensitivity and the precision to deliver its primary science on the basis of proven foreground filtering techniques, while developing new subtraction techniques to unlock new capabilities. The result will be a major step toward realizing the widely recognized scientific potential of 21 cm cosmology.Comment: 26 pages, 24 figures, 2 table

The S2 VLBI Correlator: A Correlator for Space VLBI and Geodetic Signal Processing

We describe the design of a correlator system for ground and space-based VLBI. The correlator contains unique signal processing functions: flexible LO frequency switching for bandwidth synthesis; 1 ms dump intervals, multi-rate digital signal-processing techniques to allow correlation of signals at different sample rates; and a digital filter for very high resolution cross-power spectra. It also includes autocorrelation, tone extraction, pulsar gating, signal-statistics accumulation.Comment: 44 pages, 13 figure

Multi-Reference Pseudo-Random Phase-Encoded Joint Transfrom Correlation

We propose and demonstrate the superiority of using a phase SLM only in a multi-reference phase encoded joint transform correlator(JTC) compared to an ordinary JTC. Maximal length sequences are shifted to form two dimensional orthogonal arrays referred as m-arrays. The phase mask is used in one step to encode multiple reference images and at the same time eliminate false correlation peaks through power spectrum dispersion. A theoretical model of the implemented JTC is mathematically expressed and explained in this thesis. Basic performance criteria, PSR (peak to side lobe ratio) and PCE (peak to correlation energy), are used for comparative analysis, and their relationship to joint input image size and SLM size are investigated and the results simulated

Investigation of charge coupled device correlation techniques

Analog Charge Transfer Devices (CTD's) offer unique advantages to signal processing systems, which often have large development costs, making it desirable to define those devices which can be developed for general system's use. Such devices are best identified and developed early to give system's designers some interchangeable subsystem blocks, not requiring additional individual development for each new signal processing system. The objective of this work is to describe a discrete analog signal processing device with a reasonably broad system use and to implement its design, fabrication, and testing

Bandpass filters for unconstrained target recognition and their implementation in coherent optical correlators

An up-dateable correlator is simulated which is based on the non-degenerate four wave mixing (NDFWM) interaction in the photorefractive material bismuth silicon oxide (Bi12SiO20). Specifically, it is shown that variable bandpass filters can be implemented directly in the correlator by adjusting the relative strengths of the signal and reference beams used to write the Fourier transform hologram into the photorefractive. The synthetic discriminant function (SDF) method of grey-level multiplexing is reviewed. A bandpass modification of this technique is used in the design of a multiplexed filter for the recognition of an industrial test component from a limited number of known stable state orientations when viewed from an overhead camera position. Its performance in this task when implemented in the up-dateable correlator is assessed through simulation. The conclusion of this work is that filter multiplexing must be used judiciously for orientation invariant recognition. Only a limited number of images, typically under ten, may be multiplexed into each filter since correlation peak heights and peak-to-sidelobe ratios inevitably progressively deteriorate as images are added to the filter. The effect of severe amplitude disruptions in the frequency plane on correlation peak localisation is examined. In two or higher dimensions simulations show the localisation is very robust to this disruption; an analysis is developed to indicate the reason for this. The effect is exploited by the implementation of an algorithm that locally removes the spatial frequencies that exhibit close phase matching between intra- and inter-class images. The inter-class response can be forced to zero while simultaneously improving the intra-class tolerance to orientation changes. The technique is assessed through simulation with images of two types of motor vehicle, in a variety of orientations, and shown to be effective in improving discrimination and intra-class tolerance for examples in which these were initially very poor. Bandpass filters are experimentally implemented in a joint transform correlator (JTC) based on a NDFWM interaction in Bi12SiO20. The JTC is described and its full bandwidth performance initially assessed. As anticipated from the previous considerations, inter-class discrimination was high but the intra-class tolerance very poor due to the high sensitivity of the filter. The difference of Gaussian approximation to a Laplacian of a Gaussian filter is described and its experimental implementation in the JTC detailed. Experimental results are presented for the orientation independent recognition of a car while maintaining discrimination against another car. An intra-class to inter-class correlation ratio of 7.5 dB was obtained as a best case and 3.6 dB as a worst case, the intra-class variation being at 11 ° increments in orientation at zero elevation angle. The results are extrapolated to estimate that approximately 80 filters would be required for a full 2 steradian orientation coverage. The implementation of the frequency removal technique and the Wiener filter in the JTC is briefly considered in conclusion to this work