Software correlators as testbeds for RFI algorithms

In-correlator techniques offer the possibility of identifying and/or excising radio frequency interference (RFI) from interferometric observations at much higher time and/or frequency resolution than is generally possible with the final visibility dataset. Due to the considerable computational requirements of the correlation procedure, cross-correlators have most commonly been implemented using high-speed digital signal processing boards, which typically require long development times and are difficult to alter once complete. "Software" correlators, on the other hand, make use of commodity server machines and a correlation algorithm coded in a high-level language. They are inherently much more flexible and can be developed - and modified - much more rapidly than purpose-built "hardware" correlators. Software correlators are thus a natural choice for testing new RFI detection and mitigation techniques for interferometers. The ease with which software correlators can be adapted to test RFI detection algorithms is demonstrated by the addition of kurtosis detection and plotting to the widely used DiFX software correlator, which highlights previously unknown short -duration RFI at the Hancock VLBA station.Comment: 6 pages, 1 figure, accepted for publication in Proceedings of Science [PoS(RFI2010)035]. Presented at RFI2010, the Third Workshop on RFI Mitigation in Radio Astronomy, 29-31 March 2010, Groningen, The Netherland

Soil Moisture Active Passive (SMAP) Project Algorithm Theoretical Basis Document SMAP L1B Radiometer Data Product: L1B_TB

The purpose of the Soil Moisture Active Passive (SMAP) radiometer calibration algorithm is to convert Level 0 (L0) radiometer digital counts data into calibrated estimates of brightness temperatures referenced to the Earth's surface within the main beam. The algorithm theory in most respects is similar to what has been developed and implemented for decades for other satellite radiometers; however, SMAP includes two key features heretofore absent from most satellite borne radiometers: radio frequency interference (RFI) detection and mitigation, and measurement of the third and fourth Stokes parameters using digital correlation. The purpose of this document is to describe the SMAP radiometer and forward model, explain the SMAP calibration algorithm, including approximations, errors, and biases, provide all necessary equations for implementing the calibration algorithm and detail the RFI detection and mitigation process. Section 2 provides a summary of algorithm objectives and driving requirements. Section 3 is a description of the instrument and Section 4 covers the forward models, upon which the algorithm is based. Section 5 gives the retrieval algorithm and theory. Section 6 describes the orbit simulator, which implements the forward model and is the key for deriving antenna pattern correction coefficients and testing the overall algorithm

DiFX2: A more flexible, efficient, robust and powerful software correlator

Software correlation, where a correlation algorithm written in a high-level language such as C++ is run on commodity computer hardware, has become increasingly attractive for small to medium sized and/or bandwidth constrained radio interferometers. In particular, many long baseline arrays (which typically have fewer than 20 elements and are restricted in observing bandwidth by costly recording hardware and media) have utilized software correlators for rapid, cost-effective correlator upgrades to allow compatibility with new, wider bandwidth recording systems and improve correlator flexibility. The DiFX correlator, made publicly available in 2007, has been a popular choice in such upgrades and is now used for production correlation by a number of observatories and research groups worldwide. Here we describe the evolution in the capabilities of the DiFX correlator over the past three years, including a number of new capabilities, substantial performance improvements, and a large amount of supporting infrastructure to ease use of the code. New capabilities include the ability to correlate a large number of phase centers in a single correlation pass, the extraction of phase calibration tones, correlation of disparate but overlapping sub-bands, the production of rapidly sampled filterbank and kurtosis data at minimal cost, and many more. The latest version of the code is at least 15% faster than the original, and in certain situations many times this value. Finally, we also present detailed test results validating the correctness of the new code.Comment: 28 pages, 9 figures, accepted for publication in PAS

Characteristics of the Global Radio Frequency Interference in the Protected Portion of L-Band

The National Aeronautics and Space Administration’s (NASA’s) Soil Moisture Active–Passive (SMAP) radiometer has been providing geolocated power moments measured within a 24 MHz band in the protected portion of L-band, i.e., 1400–1424 MHz, with 1.2 ms and 1.5 MHz time and frequency resolutions, as its Level 1A data. This paper presents important spectral and temporal properties of the radio frequency interference (RFI) in the protected portion of L-band using SMAP Level 1A data. Maximum and average bandwidth and duration of RFI signals, average RFI-free spectrum availability, and variations in such properties between ascending and descending satellite orbits have been reported across the world. The average bandwidth and duration of individual RFI sources have been found to be usually less than 4.5 MHz and 4.8 ms; and the average RFI-free spectrum is larger than 20 MHz in most regions with exceptions over the Middle East and Central and Eastern Asia. It has also been shown that, the bandwidth and duration of RFI signals can vary as much as 10 MHz and 10 ms, respectively, between ascending and descending orbits over certain locations. Furthermore, to identify frequencies susceptible to RFI contamination in the protected portion of L-band, observed RFI signals have been assigned to individual 1.5 MHz SMAP channels according to their frequencies. It has been demonstrated that, contrary to common perception, the center of the protected portion can be as RFI contaminated as its edges. Finally, there have been no significant correlations noted among different RFI properties such as amplitude, bandwidth, and duration within the 1400–1424 MHz ban

Radio frequency interference detection and mitigation techniques for navigation and Earth observation

Radio-Frequency Interference (RFI) signals are undesired signals that degrade or disrupt the performance of a wireless receiver. RFI signals can be troublesome for any receiver, but they are especially threatening for applications that use very low power signals. This is the case of applications that rely on the Global Navigation Satellite Systems (GNSS), or passive microwave remote sensing applications such as Microwave Radiometry (MWR) and GNSS-Reflectometry (GNSS-R). In order to solve the problem of RFI, RFI-countermeasures are under development. This PhD thesis is devoted to the design, implementation and test of innovative RFI-countermeasures in the fields of MWR and GNSS. In the part devoted to RFI-countermeasures for MWR applications, first, this PhD thesis completes the development of the MERITXELL instrument. The MERITXELL is a multi-frequency total-power radiometer conceived to be an outstanding platform to perform detection, characterization, and localization of RFI signals at the most common MWR imaging bands up to 92 GHz. Moreover, a novel RFI mitigation technique is proposed for MWR: the Multiresolution Fourier Transform (MFT). An assessment of the performance of the MFT has been carried out by comparison with other time-frequency mitigation techniques. According to the results, the MFT technique is a good trade-off solution among all other techniques since it can mitigate efficiently all kinds of RFI signals under evaluation. In the part devoted to RFI-countermeasures for GNSS and GNSS-R applications, first, a system for RFI detection and localization at GNSS bands is proposed. This system is able to detect RFI signals at the L1 band with a sensitivity of -108 dBm at full-band, and of -135 dBm for continuous wave and chirp-like signals when using the averaged spectrum technique. Besides, the Generalized Spectral Separation Coefficient (GSSC) is proposed as a figure of merit to evaluate the Signal-to-Noise Ratio (SNR) degradation in the Delay-Doppler Maps (DDMs) due to the external RFI effect. Furthermore, the FENIX system has been conceived as an innovative system for RFI detection and mitigation and anti-jamming for GNSS and GNSS-R applications. FENIX uses the MFT blanking as a pre-correlation excision tool to perform the mitigation. In addition, FENIX has been designed to be cross-GNSS compatible and RFI-independent. The principles of operation of the MFT blanking algorithm are assessed and compared with other techniques for GNSS signals. Its performance as a mitigation tool is proven using GNSS-R data samples from a real airborne campaign. After that, the main building blocks of the patented architecture of FENIX have been described. The FENIX architecture has been implemented in three real-time prototypes. Moreover, a simulator named FENIX-Sim allows for testing its performance under different jamming scenarios. The real-time performance of FENIX prototype has been tested using different setups. First, a customized VNA has been built in order to measure the transfer function of FENIX in the presence of several representative RFI/jamming signals. The results show how the power transfer function adapts itself to mitigate the RFI/jamming signal. Moreover, several real-time tests with GNSS receivers have been performed using GPS L1 C/A, GPS L2C, and Galileo E1OS. The results show that FENIX provides an extra resilience against RFI and jamming signals up to 30 dB. Furthermore, FENIX is tested using a real GNSS timing setup. Under nominal conditions, when no RFI/jamming signal is present, a small additional jitter on the order of 2-4 ns is introduced in the system. Besides, a maximum bias of 45 ns has been measured under strong jamming conditions (-30 dBm), which is acceptable for current timing systems requiring accuracy levels of 100 ns. Finally, the design of a backup system for GNSS in tracking applications that require high reliability against RFI and jamming attacks is proposed.Les interferències de radiofreqüència (RFI) són senyals no desitjades que degraden o interrompen el funcionament dels receptors sense fils. Les RFI poden suposar un problema per qualsevol receptor, però són especialment amenaçadores per les a aplicacions que fan servir senyals de molt baixa potència. Aquest és el cas de les aplicacions que depenen dels sistemes mundials de navegació per satèl·lit (GNSS) o de les aplicacions de teledetecció passiva de microones, com la radiometria de microones (MWR) i la reflectometria GNSS (GNSS-R). Per combatre aquest problema, sistemes anti-RFI s'estan desenvolupament actualment. Aquesta tesi doctoral està dedicada al disseny, la implementació i el test de sistemes anti-RFI innovadors en els camps de MWR i GNSS. A la part dedicada als sistemes anti-RFI en MWR, aquesta tesi doctoral completa el desenvolupament de l'instrument MERITXELL. El MERITXELL és un radiòmetre multifreqüència concebut com una plataforma excepcional per la detecció, caracterització i localització de RFI a les bandes de MWR més utilitzades per sota dels 92 GHz. A més a més, es proposa una nova tècnica de mitigació de RFI per MWR: la Transformada de Fourier amb Multiresolució (MFT). El funcionament de la MFT s'ha comparat amb el d'altres tècniques de mitigació en els dominis del temps i la freqüència. D'acord amb els resultats obtinguts, la MFT és una bona solució de compromís entre les altres tècniques, ja que pot mitigar de manera eficient tots els tipus de senyals RFI considerats. A la part dedicada als sistemes anti-RFI en GNSS i GNSS-R, primer es proposa un sistema per a la detecció i localització de RFI a les bandes GNSS. Aquest sistema és capaç de detectar senyals RFI a la banda L1 amb una sensibilitat de -108 dBm a tota la banda, i de -135 dBm per a senyals d'ona contínua i chirp fen un mitjana de l'espectre. A més a més, el Coeficient de Separació Espectral Generalitzada (GSSC) es proposa com una mesura per avaluar la degradació de la relació senyal a soroll (SNR) en els Mapes de Delay-Doppler (DDM) a causa del impacte de les RFI. La major contribució d'aquesta tesi doctoral és el sistema FENIX. FENIX és un sistema innovador de detecció i mitigació de RFI i inhibidors de freqüència per aplicacions GNSS i GNSS-R. FENIX utilitza la MFT per eliminar la interferència abans del procés de correlació amb el codi GNSS independentment del tipus de RFI. L'algoritme de mitigació de FENIX s'ha avaluat i comparat amb altres tècniques i els principals components de la seva arquitectura patentada es descriuen. Finalment, un simulador anomenat FENIX-Sim permet avaluar el seu rendiment en diferents escenaris d'interferència. El funcionament en temps real del prototip FENIX ha estat provat utilitzant diferents mètodes. En primer lloc, s'ha creat un analitzador de xarxes per a mesurar la funció de transferència del FENIX en presència de diverses RFI representatives. Els resultats mostren com la funció de transferència s'adapta per mitigar el senyal interferent. A més a més, s'han realitzat diferents proves en temps real amb receptors GNSS compatibles amb els senyals GPS L1 C/A, GPS L2C i Galileo E1OS. Els resultats mostren que FENIX proporciona una resistència addicional contra les RFI i els senyals dels inhibidors de freqüència de fins a 30 dB. A més a més, FENIX s'ha provat amb un sistema comercial de temporització basat en GNSS. En condicions nominals, sense RFI, FENIX introdueix un petit error addicional de tan sols 2-4 ns. Per contra, el biaix màxim mesurat en condicions d'alta interferència (-30 dBm) és de 45 ns, el qual és acceptable per als sistemes de temporització actuals que requereixen nivells de precisió d'uns 100 ns. Finalment, es proposa el disseny d'un sistema robust de seguiment, complementari als GNSS, per a aplicacions que requereixen alta fiabilitat contra RFI.Postprint (published version

RFI mitigation in radio astronomy

Thesis (MEng)--Stellenbosch University, 2014.ENGLISH ABSTRACT: Technical advances in electromagnetics, signal processing and processing power have led to a significant increase in sensitivity and accuracy in radio telescopes. With this increase in sensitivity, radio frequency interference (RFI) has become a much larger problem. The notable growth in wireless communication as well as self generated RFI has further escalated this problem. In order to utilise the full capabilities of modern radio telescopes, RFI mitigation is required on the captured signals. With the enormous data rates of modern radio telescopes, managing RFI has become increasingly difficult, and in order to utilise the full captured radio spectrum, more accurate RFI mitigation strategies will be necessary. The use of different RFI mitigation strategies is studied in the form of online and offline techniques. This includes Spectral Kurtosis, Spectral Flatness and the Var/SumThreshold method. The special case for RFI mitigation in timing pulsars will also be studied. These techniques are well known in the radio astronomy community; here, spectral kurtosis and spectral flatness will be implemented on the raw data as well as the post correlated data. System speed and accuracy will be the deciding factors when testing these methods as possible solutions to this problem.AFRIKAANSE OPSOMMING: ‘n Toename in die sensitiwiteit van hedendagse radioteleskope kan toegedra word aan die tegniese bevordering in elktromagnetika en seinverwerking. Die toename in sensitiwiteit het egter tot die gevolg dat radiofrekwensiesteuring ‘n groter rol speel in hedendaagse radioteleskope. Die groei in die gebruik van radioverbindings asook die gevolge van self gei¨nduseerde radiofrekwensiesteuring dra ook verder by tot hierdie probleem. Radiofrekwensiesteuring matiging word toegepas op die opgevangde seine, om sodoende gebruik te maak van die volle kapasiteit van moderne radioteleskope. Die bestuur van radiofrekwensiesteuring word bemoeilik deur die groot hoeveelheid intydse data van die radioteleskope. Meer akurate radiofrekwensiesteuring matigingstegnieke word vereis om die bandwydte ten volle te hanteer. Daar word op ‘n aantal verskillende matingstegnieke gefokus. Hierdie tegenieke kan in twee kategorieë verdeel word, naamlik aanlyn- en aflyntegenieke. Onderafdelings van hierdie kategorieë sluit in: spektrale kurtose, spektrale matheid en “Var/SumThreshold”. Daar word ook na ‘n spesiale geval van radiofrekwensiesteuring matiging gekyk, in die opmeeting van tydsberekening-pulsars. Alhoewel hierdie tegnieke bekend is in die radioastronomie gemeenskap, word spektrale kurtose en spektrale matheid egter toegepas op die rou data sowel as postgekorreleerde data. Daar sal op stelsel spoed en akuratheid gefokus word, om vas te stel of hierdie metodes wel moontlike oplossings bied tot die probleem bespreek

Timing Jitter from Broadband Radio Observations of Millisecond Pulsars

Timing the arrival of pulsar light is a critical tool for scrutinizing the characteristics of the interstellar medium (ISM) and the pulsar itself. Pulsar timing is affected by factors such as the solar system\u27s kinematics, turbulence in the ISM, and intrinsic timing noise inherent to each pulsar. The North American Nanohertz Observatory for Gravitational Waves (NANOGrav) maintains a catalog of high-precision timing for millisecond pulsars. Using NANOGrav pulsars, and accounting for relative time-of-arrival (TOA) differences within observation epochs, the remaining noise in the TOA can be parsed out to just the jitter innate to the pulsar. This research utilizes wide band radio observations by the Very Large Array of NANOGrav pulsars to detail the characteristics of pulsar jitter, explore frequency dependence of apparent jitter measurements, and characterize other signal-dependent contributions to the timing white noise

Applying hybrid clustering in pulsar candidate sifting with multi-modality for FAST survey

Pulsar search is always the basis of pulsar navigation, gravitational wave detection and other research topics. Currently, the volume of pulsar candidates collected by Five-hundred-meter Aperture Spherical radio Telescope (FAST) shows an explosive growth rate that has brought challenges for its pulsar candidate filtering System. Particularly, the multi-view heterogeneous data and class imbalance between true pulsars and non-pulsar candidates have negative effects on traditional single-modal supervised classification methods. In this study, a multi-modal and semi-supervised learning based pulsar candidate sifting algorithm is presented, which adopts a hybrid ensemble clustering scheme of density-based and partition-based methods combined with a feature-level fusion strategy for input data and a data partition strategy for parallelization. Experiments on both HTRU (The High Time Resolution Universe Survey) 2 and FAST actual observation data demonstrate that the proposed algorithm could excellently identify the pulsars: On HTRU2, the precision and recall rates of its parallel mode reach 0.981 and 0.988. On FAST data, those of its parallel mode reach 0.891 and 0.961, meanwhile, the running time also significantly decrease with the increment of parallel nodes within limits. So, we can get the conclusion that our algorithm could be a feasible idea for large scale pulsar candidate sifting of FAST drift scan observation

Development of Radio Frequency Interference Detection Algorithm for Passive Microwave Remote Sensing

Radio Frequency Interference (RFI) signals are man-made sources that are increasingly plaguing passive microwave remote sensing measurements. RFI is of insidious nature, with some signals low power enough to go undetected but large enough to impact science measurements and their results. With the launch of the European Space Agency (ESA) Soil Moisture and Ocean Salinity (SMOS) satellite in November 2009 and the upcoming launches of the new NASA sea-surface salinity measuring Aquarius mission in June 2011 and soil-moisture measuring Soil Moisture Active Passive (SMAP) mission around 2015, active steps are being taken to detect and mitigate RFI at L-band. An RFI detection algorithm was designed for the Aquarius mission. The algorithm performance was analyzed using kurtosis based RFI ground-truth. The algorithm has been developed with several adjustable location dependant parameters to control the detection statistics (false-alarm rate and probability of detection). The kurtosis statistical detection algorithm has been compared with the Aquarius pulse detection method. The comparative study determines the feasibility of the kurtosis detector for the SMAP radiometer, as a primary RFI detection algorithm in terms of detectability and data bandwidth. The kurtosis algorithm has superior detection capabilities for low duty-cycle radar like pulses, which are more prevalent according to analysis of field campaign data. Most RFI algorithms developed have generally been optimized for performance with individual pulsed-sinusoidal RFI sources. A new RFI detection model is developed that takes into account multiple RFI sources within an antenna footprint. The performance of the kurtosis detection algorithm under such central-limit conditions is evaluated. The SMOS mission has a unique hardware system, and conventional RFI detection techniques cannot be applied. Instead, an RFI detection algorithm for SMOS is developed and applied in the angular domain. This algorithm compares brightness temperature values at various incidence angles for a particular grid location. This algorithm is compared and contrasted with other algorithms present in the visibility domain of SMOS, as well as the spatial domain. Initial results indicate that the SMOS RFI detection algorithm in the angular domain has a higher sensitivity and lower false-alarm rate than algorithms developed in the other two domains.Ph.D.Atmospheric and Space SciencesUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/86308/1/samisra_1.pd