Towards Dynamic Catalogues

The International LOFAR Telescope is designed to carry out unique science in the spatial, spectral, polarisation and temporal domains. The Transients Key Science Project aims to study all transient and variable sources detected by LOFAR. One of its products will be an up-to-date catalogue of all sources detected by LOFAR, i.e. a spectral light-curve database, with real-time capabilities, and expected to grow gradually with 50−100 TB/yr. The response time to transient and variable events depends strongly on the query execution plans of the algorithms that search the LOFAR light-curve database for previous (non-)detections in the spatial, spectral, polarisation and temporal domains. Here we show how the Transients Key Science Project of LOFAR approaches these challenges by using column-stores, sharded databases and implementing the new array query language SciQL (pronounced as ’cycle’)

Observing pulsars and fast transients with LOFAR

Low frequency radio waves, while challenging to observe, are a rich source of information about pulsars. The LOw Frequency ARray (LOFAR) is a new radio interferometer operating in the lowest 4 octaves of the ionospheric “radio window”: 10–240 MHz, that will greatly facilitate observing pulsars at low radio frequencies. Through the huge collecting area, long baselines, and flexible digital hardware, it is expected that LOFAR will revolutionize radio astronomy at the lowest frequencies visible from Earth. LOFAR is a next-generation radio telescope and a pathfinder to the Square Kilometre Array (SKA), in that it incorporates advanced multi-beaming techniques between thousands of individual elements. We discuss the motivation for low-frequency pulsar observations in general and the potential of LOFAR in addressing these science goals.We present LOFAR as it is designed to perform high-time-resolution observations of pulsars and other fast transients, and outline the various relevant observing modes and data reduction pipelines that are already or will soon be implemented to facilitate these observations. A number of results obtained from commissioning observations are presented to demonstrate the exciting potential of the telescope. This paper outlines the case for low frequency pulsar observations and is also intended to serve as a reference for upcoming pulsar/fast transient science papers with LOFAR

Astronomical Data Processing Using SciQL, an SQL Based Query Language for Array Data

SciQL (pronounced as ‘cycle’) is a novel SQL-based array query language for scientific applications with both tables and arrays as first class citizens. SciQL lowers the entrance fee of adopting relational DBMS (RDBMS) in scienti

Fast in-database cross-matching of high-cadence, high-density source lists with an up-to-date sky model

Coming high-cadence wide-field optical telescopes will image hundreds of thousands of sources per minute. Besides inspecting the near real-time data streams for transient and variability events, the accumulated data archive is a wealthy laboratory for making complementary scientific discoveries. The goal of this work is to optimise column-oriented database techniques to enable the construction of a full-source and light-curve database for large-scale surveys, that is accessible by the astronomical community. We adopted LOFAR's Transients Pipeline as the baseline and modified it to enable the processing of optical images that have much higher source densities. The pipeline adds new source lists to the archive database, while cross-matching them with the known cataloguedsources in order to build a full light-curve archive. We investigated several techniques of indexing and partitioning the largest tables, allowing for faster positional source look-ups in the cross matching algorithms. We monitored all query run times in long-term pipeline runs where we processed a subset of IPHAS data that have image source density peaks over 170,000 per field of view (500,000 deg−2). Our analysis demonstrates that horizontal table partitions of declination widths of one-degree control the query run times. Usage of an index strategy where the partitions are densely sorted according to source declination yields ano

An automated archival VLA transients survey

In this paper we present the results of a survey for radio transients using data obtained from the Very Large Array archive. We have reduced, using a pipeline procedure, 5037 observations of the most common pointings - i.e. the calibrator fields. These fields typically contain a relatively bright point source and are used to calibrate 'target' observations: they are therefore rarely imaged themselves. The observations used span a time range ˜1984-2008 and consist of eight different pointings, three different frequencies (8.4, 4.8 and 1.4 GHz) and have a total observing time of 435 h. We have searched for transient and variable radio sources within these observations using components from the prototype LOFAR transient detection system. In this paper we present the methodology for reducing large volumes of Very Large Array data; and we also present a brief overview of the prototype LOFAR transient detection algorithms. No radio transients were detected in this survey, therefore we place an upper limit on the snapshot rate of GHz frequency transients >8.0 mJy to ρ≤ 0.032 deg-2 that have typical time-scales 4.3 to 45.3 d. We compare and contrast our upper limit with the snapshot rates - derived from either detections or non-detections of transient and variable radio sources - reported in the literature. When compared with the current Log N-Log S distribution formed from previous surveys, we show that our upper limit is consistent with the observed population. Current and future radio transient surveys will hopefully further constrain these statistics, and potentially discover dominant transient source populations. In this paper we also briefly explore the current transient commissioning observations with LOFAR, and the impact they will make on the field

Pulsar polarisation below 200 MHz: Average profiles and propagation effects

Aims: We present the highest-quality polarisation profiles to date of 16 non-recycled pulsars and four millisecond pulsars, observed below 200 MHz with the LOFAR high-band antennas. Based on the observed profiles, we perform an initial investigation of expected observational effects resulting from the propagation of polarised emission in the pulsar magnetosphere and the interstellar medium. Methods: The polarisation data presented in this paper have been calibrated for the geometric-projection and beam-shape effects that distort the polarised information as detected with the LOFAR antennas. We have used RM Synthesis to determine the amount of Faraday rotation in the data at the time of the observations. The ionospheric contribution to the measured Faraday rotation was estimated using a model of the ionosphere. To study the propagation effects, we have compared our low-frequency polarisation observations with archival data at 240, 400, 600, and 1400 MHz. Results: The predictions of magnetospheric birefringence in pulsars have been tested using spectra of the pulse width and fractional polarisation from multifrequency data. The derived spectra offer only partial support for the expected effects of birefringence on the polarisation properties, with only about half of our sample being consistent with the model's predictions. It is noted that for some pulsars these measurements are contaminated by the effects of interstellar scattering. For a number of pulsars in our sample, we have observed significant variations in the amount of Faraday rotation as a function of pulse phase, which is possibly an artefact of scattering. These variations are typically two orders of magnitude smaller than that observed at 1400 MHz by Noutsos et al. (2009), for a different sample of southern pulsars. In this paper we present a possible explanation for the difference in magnitude of this effect between the two frequencies, based on scattering. Finally, we have estimated the magnetospheric emission heights of low-frequency radiation from four pulsars, based on the phase lags between the flux-density and the PA profiles, and the theoretical framework of Blaskiewicz et al. (1991, ApJ, 370, 643). These estimates yielded heights of a few hundred km; at least for PSR B1133+16, this is consistent with emission heights derived based on radius-to-frequency mapping, but is up to a few times larger than the recent upper limit based on pulsar timing. Conclusions: Our work has shown that models, like magnetospheric birefringence, cannot be the sole explanation for the complex polarisation behaviour of pulsars. On the other hand, we have reinforced the claim that interstellar scattering can introduce a rotation of the PA with frequency that is indistinguishable from Faraday rotation and also varies as a function of pulse phase. In one case, the derived emission heights appear to be consistent with the predictions of radius-to-frequency mapping at 150 MHz, although this interpretation is subject to a number of systematic uncertainties