1 research outputs found
A Processing Pipeline for High Volume Pulsar Data Streams
Pulsar data analysis pipelines have historically been comprised of bespoke
software systems, supporting the off-line analysis of data. However modern data
acquisition systems are making off-line analyses impractical. They often output
multiple simultaneous high volume data streams, significantly increasing data
capture rates. This leads to the accumulation of large data volumes, which are
prohibitively expensive to retain. To maintain processing capabilities when
off-line analysis becomes infeasible due to cost, requires a shift to on-line
data processing. This paper makes four contributions facilitating this shift
with respect to the search for radio pulsars: i) it characterises for the
modern era, the key components of a pulsar search science (not signal
processing) pipeline, ii) it examines the feasibility of implementing on-line
pulsar search via existing tools, iii) problems preventing an easy transition
to on-line search are identified and explained, and finally iv) it provides the
design for a new prototype pipeline capable of overcoming such problems.
Realised using Commercial off-the-shelf (COTS) software components, the
deployable system is open source, simple, scalable, and cheap to produce. It
has the potential to achieve pulsar search design requirements for the Square
Kilometre Array (SKA), illustrated via testing under simulated SKA loads.Comment: pre-print submitted to Astronomy & Computin