SERPent: Automated reduction and RFI-mitigation software for e-MERLIN

The Scripted E-merlin Rfi-mitigation PipelinE for iNTerferometry (SERPent) is an automated reduction and RFI-mitigation procedure utilising the SumThreshold methodology (Offringa et al., 2010a), originally developed for the LOFAR pipeline. SERPent is written in the Parseltongue language enabling interaction with the Astronomical Image Processing Software (AIPS) program. Moreover, SERPent is a simple ‘out of the box’ Python script, which is easy to set up and is free of compilers. In addition to the flagging of RFI affected visibilities, the script also flags antenna zero-amplitude dropouts and Lovell telescope phase calibrator stationary scans inherent to the e-MERLIN system. Both the flagging and computational performances of SERPent are presented here, for e-MERLIN commissioning datasets for both L-band (1.3–1.8 GHz) and C-band (4–8 GHz) observations. RFI typically amounts to <20%–25% for the more problematic L-band observations and <5% for the generally RFI quieter C-band. The level of RFI detection and flagging is more accurate and delicate than visual manual flagging, with the output immediately ready for AIPS calibration. SERPent is fully parallelised and has been tested on a range of computing systems. The current flagging rate is at 110 GB day−1 on a ‘high-end’ computer (16 CPUs, 100 GB memory) which amounts to ∼6.9 GB CPU−1 day−1, with an expected increase in performance when e-MERLIN has completed its commissioning. The refining of automated reduction and calibration procedures is essential for the e-MERLIN legacy projects and future interferometers such as the SKA and the associated pathfinders (MeerKAT and ASKAP), where the vast data sizes (>TB) make traditional astronomer interactions unfeasible

The Effelsberg-Bonn HI Survey: Data reduction

Starting in winter 2008/2009 an L-band 7-Feed-Array receiver is used for a 21-cm line survey performed with the 100-m telescope, the Effelsberg-Bonn HI survey (EBHIS). The EBHIS will cover the whole northern hemisphere for decl.>-5 deg comprising both the galactic and extragalactic sky out to a distance of about 230 Mpc. Using state-of-the-art FPGA-based digital fast Fourier transform spectrometers, superior in dynamic range and temporal resolution to conventional correlators, allows us to apply sophisticated radio frequency interference (RFI) mitigation schemes. In this paper, the EBHIS data reduction package and first results are presented. The reduction software consists of RFI detection schemes, flux and gain-curve calibration, stray-radiation removal, baseline fitting, and finally the gridding to produce data cubes. The whole software chain is successfully tested using multi-feed data toward many smaller test fields (1--100 square degrees) and recently applied for the first time to data of two large sky areas, each covering about 2000 square degrees. The first large area is toward the northern galactic pole and the second one toward the northern tip of the Magellanic Leading Arm. Here, we demonstrate the data quality of EBHIS Milky Way data and give a first impression on the first data release in 2011.Comment: 17 pages, 14 figures; to be published in ApJ

Radio Frequency Interference Mitigation

Radio astronomy observational facilities are under constant upgradation and development to achieve better capabilities including increasing the time and frequency resolutions of the recorded data, and increasing the receiving and recording bandwidth. As only a limited spectrum resource has been allocated to radio astronomy by the International Telecommunication Union, this results in the radio observational instrumentation being inevitably exposed to undesirable radio frequency interference (RFI) signals which originate mainly from terrestrial human activity and are becoming stronger with time. RFIs degrade the quality of astronomical data and even lead to data loss. The impact of RFIs on scientific outcome is becoming progressively difficult to manage. In this article, we motivate the requirement for RFI mitigation, and review the RFI characteristics, mitigation techniques and strategies. Mitigation strategies adopted at some representative observatories, telescopes and arrays are also introduced. We also discuss and present advantages and shortcomings of the four classes of RFI mitigation strategies, applicable at the connected causal stages: preventive, pre-detection, pre-correlation and post-correlation. The proper identification and flagging of RFI is key to the reduction of data loss and improvement in data quality, and is also the ultimate goal of developing RFI mitigation techniques. This can be achieved through a strategy involving a combination of the discussed techniques in stages. Recent advances in high speed digital signal processing and high performance computing allow for performing RFI excision of large data volumes generated from large telescopes or arrays in both real time and offline modes, aiding the proposed strategy.Comment: 26 pages, 10 figures, Chinese version accepted for publication in Acta Astronomica Sinica; English version to appear in Chinese Astronomy and Astrophysic

A novel astronomical application for formation flying small satellites

OLFAR, Orbiting Low Frequency Antennas for Radio Astronomy, will be a space mission to observe the universe frequencies below 30 MHz, as it was never done before with an orbiting telescope. Because of the ionospheric scintillations below 30 MHz and the opaqueness of the ionosphere below 15 MHz, a space mission is the only opportunity for this as yet unexplored frequency range in radio astronomy. The frequency band is scientifically very interesting for exploring the early cosmos at high hydrogen redshifts, the so-called dark-ages and the epoch of reionization, the discovery of planetary and solar bursts in other solar systems, for obtaining a tomographic view of space weather, ultra-high energy cosmic rays and for many other astronomical areas of interest. Because of the low observing frequency the aperture size of the instrument must be in the order of 100 km. This requires a distributed space mission which is proposed to be implemented using formation flying of small satellites. The individual satellites are broken down in five major subsystems: the spacecraft bus, the antenna design, the frontend, backend and data transport. One of the largest challenges is the inter-satellite communication. In this paper the concept and design considerations of OLFAR are presented

Multi-Level Pre-Correlation RFI Flagging for Real-Time Implementation on UniBoard

Because of the denser active use of the spectrum, and because of radio telescopes higher sensitivity, radio frequency interference (RFI) mitigation has become a sensitive topic for current and future radio telescope designs. Even if quite sophisticated approaches have been proposed in the recent years, the majority of RFI mitigation operational procedures are based on post-correlation corrupted data flagging. Moreover, given the huge amount of data delivered by current and next generation radio telescopes, all these RFI detection procedures have to be at least automatic and, if possible, real-time. In this paper, the implementation of a real-time pre-correlation RFI detection and flagging procedure into generic high-performance computing platforms based on Field Programmable Gate Arrays (FPGA) is described, simulated and tested. One of these boards, UniBoard, developed under a Joint Research Activity in the RadioNet FP7 European programme is based on eight FPGAs interconnected by a high speed transceiver mesh. It provides up to ~4 TMACs with Altera Stratix IV FPGA and 160 Gbps data rate for the input data stream. Considering the high in-out data rate in the pre-correlation stages, only real-time and go-through detectors (i.e. no iterative processing) can be implemented. In this paper, a real-time and adaptive detection scheme is described. An ongoing case study has been set up with the Electronic Multi-Beam Radio Astronomy Concept (EMBRACE) radio telescope facility at Nan\c{c}ay Observatory. The objective is to evaluate the performances of this concept in term of hardware complexity, detection efficiency and additional RFI metadata rate cost. The UniBoard implementation scheme is described.Comment: 16 pages, 13 figure

Pulsar data analysis with PSRCHIVE

PSRCHIVE is an open-source, object-oriented, scientific data analysis software library and application suite for pulsar astronomy. It implements an extensive range of general-purpose algorithms for use in data calibration and integration, statistical analysis and modeling, and visualisation. These are utilised by a variety of applications specialised for tasks such as pulsar timing, polarimetry, radio frequency interference mitigation, and pulse variability studies. This paper presents a general overview of PSRCHIVE functionality with some focus on the integrated interfaces developed for the core applications.Comment: 21 pages, 5 figures; tutorial presented at IPTA 2010 meeting in Leiden merged with talk presented at 2011 pulsar conference in Beijing; includes further research and development on algorithms for RFI mitigation and TOA bias correctio