Multi-Level Pre-Correlation RFI Flagging for Real-Time Implementation on UniBoard

Because of the denser active use of the spectrum, and because of radio telescopes higher sensitivity, radio frequency interference (RFI) mitigation has become a sensitive topic for current and future radio telescope designs. Even if quite sophisticated approaches have been proposed in the recent years, the majority of RFI mitigation operational procedures are based on post-correlation corrupted data flagging. Moreover, given the huge amount of data delivered by current and next generation radio telescopes, all these RFI detection procedures have to be at least automatic and, if possible, real-time. In this paper, the implementation of a real-time pre-correlation RFI detection and flagging procedure into generic high-performance computing platforms based on Field Programmable Gate Arrays (FPGA) is described, simulated and tested. One of these boards, UniBoard, developed under a Joint Research Activity in the RadioNet FP7 European programme is based on eight FPGAs interconnected by a high speed transceiver mesh. It provides up to ~4 TMACs with Altera Stratix IV FPGA and 160 Gbps data rate for the input data stream. Considering the high in-out data rate in the pre-correlation stages, only real-time and go-through detectors (i.e. no iterative processing) can be implemented. In this paper, a real-time and adaptive detection scheme is described. An ongoing case study has been set up with the Electronic Multi-Beam Radio Astronomy Concept (EMBRACE) radio telescope facility at Nan\c{c}ay Observatory. The objective is to evaluate the performances of this concept in term of hardware complexity, detection efficiency and additional RFI metadata rate cost. The UniBoard implementation scheme is described.Comment: 16 pages, 13 figure

Radio Frequency Interference Mitigation

Radio astronomy observational facilities are under constant upgradation and development to achieve better capabilities including increasing the time and frequency resolutions of the recorded data, and increasing the receiving and recording bandwidth. As only a limited spectrum resource has been allocated to radio astronomy by the International Telecommunication Union, this results in the radio observational instrumentation being inevitably exposed to undesirable radio frequency interference (RFI) signals which originate mainly from terrestrial human activity and are becoming stronger with time. RFIs degrade the quality of astronomical data and even lead to data loss. The impact of RFIs on scientific outcome is becoming progressively difficult to manage. In this article, we motivate the requirement for RFI mitigation, and review the RFI characteristics, mitigation techniques and strategies. Mitigation strategies adopted at some representative observatories, telescopes and arrays are also introduced. We also discuss and present advantages and shortcomings of the four classes of RFI mitigation strategies, applicable at the connected causal stages: preventive, pre-detection, pre-correlation and post-correlation. The proper identification and flagging of RFI is key to the reduction of data loss and improvement in data quality, and is also the ultimate goal of developing RFI mitigation techniques. This can be achieved through a strategy involving a combination of the discussed techniques in stages. Recent advances in high speed digital signal processing and high performance computing allow for performing RFI excision of large data volumes generated from large telescopes or arrays in both real time and offline modes, aiding the proposed strategy.Comment: 26 pages, 10 figures, Chinese version accepted for publication in Acta Astronomica Sinica; English version to appear in Chinese Astronomy and Astrophysic

RFI mitigation: cyclostationary criterion

Radio astronomical observations are increasingly corrupted by radio frequency interferences. Thus, real- time ltering algorithms are becoming essential. One approach is to use a speci c time property of the Telecoms signals : the cyclostationarity. This property can be exploited for detection purpose or ltering purpose. In par- ticular, new generations of radio telescopes will be based on antenna arrays providing the possibility of applying spatial ltering techniques. In this paper, we compare the performance between classical approaches based on power statistics and cyclic approaches. This comparison is done through simulations on synthetic data and through simulations on real data acquired with the new generation low frequency array radio telescope, LOFAR

Calibration Challenges for Future Radio Telescopes

Instruments for radio astronomical observations have come a long way. While the first telescopes were based on very large dishes and 2-antenna interferometers, current instruments consist of dozens of steerable dishes, whereas future instruments will be even larger distributed sensor arrays with a hierarchy of phased array elements. For such arrays to provide meaningful output (images), accurate calibration is of critical importance. Calibration must solve for the unknown antenna gains and phases, as well as the unknown atmospheric and ionospheric disturbances. Future telescopes will have a large number of elements and a large field of view. In this case the parameters are strongly direction dependent, resulting in a large number of unknown parameters even if appropriately constrained physical or phenomenological descriptions are used. This makes calibration a daunting parameter estimation task, that is reviewed from a signal processing perspective in this article.Comment: 12 pages, 7 figures, 20 subfigures The title quoted in the meta-data is the title after release / final editing

Localization and litigation of radio frequency interference for interferometric arrays

Thesis (PhD)--Stellenbosch University, 2018.ENGLISH ABSTRACT: Radio telescopes have increased exponentially in sensitivity ever since the first single dish radio telescopes were built in the 1930's. This trend continues with the development of next generation telescopes such as the Square Kilometre Array (SKA). Parallel to the development of radio telescopes, has been the rapid expansion of telecommunication technologies. Consequently, radio telescopes are becoming more sensitive in an environment with ever increasing radio frequency interference (RFI). The ideal solution to RFI that is detected by a radio telescope is to locate its source and then have it removed. Removal of the source is usually only possible if it is occurring in a protected band or the radio telescope is in a radio quiet zone. Unfortunately, most of the radio spectrum has been allocated to active communication services and not all radio telescopes are in radio quiet zones. The alternative is to mitigate its effect using methods such as spatial RFI mitigation. The contributions of this PhD dissertation are twofold: firstly, a source localization algorithm that takes into account the constraints and advantages of the arrays used for radio astronomy has been developed; and secondly, existing spatial RFI mitigation techniques have been adapted to take into account the bandwidth of the RFI signals. The computationally efficient localization algorithm that was developed is best suited for interferometric arrays with low array beam sidelobes. Two variants of the algorithm were developed, one that works for sources in the near-field and the other for far-field sources. In the near-field, the computational complexity of the algorithm is linear with search grid size compared to cubic scaling of the state-of-the-art 3-D MUSIC method. The trade-off is that the proposed algorithm requires a once-off a priori calculation and storing of weighting matrices. In an experiment using a station of the Low Frequency Array (LOFAR) a hexacopter was flown around the array, at a mean radial distance of 190 m, broadcasting a signal. The mean error in distance between the estimated position of the hexacopter and the GPS position of the hexacopter was 2 m for a wavelength of 6.7 m. The non-narrowband RFI mitigation method developed consists of a second order filter that is used to mitigate powerful RFI with bandwidth sufficient to cause aberrations that are below the noise, but with power that competes with the astronomical sources. The second order filter consists of a first order subspace subtraction filter combined with a flat frequency response model for the RFI source. Taking into account mutual coupling as well as a calibration step to account for unknown complex gains, the algorithm was found to process approximately 1.6 times more bandwidth than using just a first order subspace subtraction filter.AFRIKAANSE OPSOMMING: Sedert die eerste enkelskottel radioteleskope in die 1930's gebou is, het die sensitiwiteit van radioteleskope eksponensieël toegeneem. Hierdie tendens gaan voort met die ontwikkeling van volgende generasie teleskope, soos byvoorbeeld die Square Kilometer Array (SKA). In parallel met die ontwikkeling van radioteleskope, het telekommunikasietegnologieë ook vinnig uitgebrei. Gevolglik word radioteleskope meer sensitief in 'n omgewing met toenemende radiofrekwensie-inmenging (RFI). Die optimale oplossing vir RFI is om die bron daarvan op te spoor en te verwyder. Verwydering van die bron is gewoonlik net moontlik as dit teenwoording is in 'n beskermde band of as die radio teleskoop in 'n radio-stil gebied is. Ongelukkig is meeste van die radio spectrum toegeken aan kommunikasiedienste en nie alle radio teleskope is in radio-stil gebiede nie. Die alternatief om die effek daarvan te mitigeer deur middel van metodes soos ruimtelike RFI-mitigasie. Die bydraes van hierdie doktorale proefskrif is tweeledig: eerstens, die ontwikkeling van 'n bronlokaliseringsalgoritme wat die beperkings en voordele van die skikkings wat gebruik word vir radio astronomie in ag neem en tweedens, die aanpassing van bestaande ruimtelike RFI mitigeringstegnieke om die bandwydte van die RFI seine in ag te neem. Die berekeningsdoeltreffende lokaliseringsalgoritme wat ontwikkel is, is die beste geskik vir interferometriese skikkings met lae samestelling-bundel sylobbe. Twee weergawes van die algoritme is ontwikkel, die eerste hanteer bronne in die nabyveld en die ander hanteer vêrveld bronne. In die nabyveld is die berekeningskompleksiteit van die algoritme lineêr met soektogroostergrootte in vergelyking met die kubieke skalering van die 3-D MUSIC-metode. Die nadeel is dat die voorgestelde algoritme 'n eenmalige a priori berekening en stoor van gewigsmatrikse vereis. In 'n eksperiment by 'n stasie van die Low Frequency Array (LOFAR), het 'n heksakopter oor die skikking gevlieg met 'n gemiddelde radiale afstand van 190 m en 'n sein uitgesaai. Die gemiddelde fout in die afstand tussen die beraamde posisie van die heksakopter en die GPS-posisie van die heksakopter was 2 m vir 'n golflengte van 6.7 m. Die nie-smalband RFI mitigasie metode wat ontwikkel is, fasiliteer die de-finieering van 'n tweede-orde filter wat gebruik word om kragtige RFI met bandwydte verwante krag onder die geruis, maar met krag wat met die astronomiese bronne kompeteer, te mitigeer. Die tweede order filter bestaan uit 'n eerste orde subruimte verminderingsfilter gekombineer met 'n plat frekwensie responsmodel vir die RFI bron. Met inagneming van wedersydse koppeling asook 'n kalibrasie stap om vir onbekende komplekse antenna aanwinste voorsiening te maak, is gevind dat die algoritme ongeveer 1.6 meer bandwydte kan verwerk as 'n eerste orde subruimte verminderingsfiter

Phased Array Feed Calibration, Beamforming and Imaging

Phased array feeds (PAFs) for reflector antennas offer the potential for increased reflector field of view and faster survey speeds. To address some of the development challenges that remain for scientifically useful PAFs, including calibration and beamforming algorithms, sensitivity optimization, and demonstration of wide field of view imaging, we report experimental results from a 19 element room temperature L-band PAF mounted on the Green Bank 20-Meter Telescope. Formed beams achieved an aperture efficiency of 69% and system noise temperature of 66 K. Radio camera images of several sky regions are presented. We investigate the noise performance and sensitivity of the system as a function of elevation angle with statistically optimal beamforming and demonstrate cancelation of radio frequency interference sources with adaptive spatial filtering.Comment: 19 pages, 13 figure

The Data Big Bang and the Expanding Digital Universe: High-Dimensional, Complex and Massive Data Sets in an Inflationary Epoch

Recent and forthcoming advances in instrumentation, and giant new surveys, are creating astronomical data sets that are not amenable to the methods of analysis familiar to astronomers. Traditional methods are often inadequate not merely because of the size in bytes of the data sets, but also because of the complexity of modern data sets. Mathematical limitations of familiar algorithms and techniques in dealing with such data sets create a critical need for new paradigms for the representation, analysis and scientific visualization (as opposed to illustrative visualization) of heterogeneous, multiresolution data across application domains. Some of the problems presented by the new data sets have been addressed by other disciplines such as applied mathematics, statistics and machine learning and have been utilized by other sciences such as space-based geosciences. Unfortunately, valuable results pertaining to these problems are mostly to be found only in publications outside of astronomy. Here we offer brief overviews of a number of concepts, techniques and developments, some "old" and some new. These are generally unknown to most of the astronomical community, but are vital to the analysis and visualization of complex datasets and images. In order for astronomers to take advantage of the richness and complexity of the new era of data, and to be able to identify, adopt, and apply new solutions, the astronomical community needs a certain degree of awareness and understanding of the new concepts. One of the goals of this paper is to help bridge the gap between applied mathematics, artificial intelligence and computer science on the one side and astronomy on the other.Comment: 24 pages, 8 Figures, 1 Table. Accepted for publication: "Advances in Astronomy, special issue "Robotic Astronomy