Scalable Solutions for Automated Single Pulse Identification and Classification in Radio Astronomy

Data collection for scientific applications is increasing exponentially and is forecasted to soon reach peta- and exabyte scales. Applications which process and analyze scientific data must be scalable and focus on execution performance to keep pace. In the field of radio astronomy, in addition to increasingly large datasets, tasks such as the identification of transient radio signals from extrasolar sources are computationally expensive. We present a scalable approach to radio pulsar detection written in Scala that parallelizes candidate identification to take advantage of in-memory task processing using Apache Spark on a YARN distributed system. Furthermore, we introduce a novel automated multiclass supervised machine learning technique that we combine with feature selection to reduce the time required for candidate classification. Experimental testing on a Beowulf cluster with 15 data nodes shows that the parallel implementation of the identification algorithm offers a speedup of up to 5X that of a similar multithreaded implementation. Further, we show that the combination of automated multiclass classification and feature selection speeds up the execution performance of the RandomForest machine learning algorithm by an average of 54% with less than a 2% average reduction in the algorithm's ability to correctly classify pulsars. The generalizability of these results is demonstrated by using two real-world radio astronomy data sets.Comment: In Proceedings of the 47th International Conference on Parallel Processing (ICPP 2018). ACM, New York, NY, USA, Article 11, 11 page

Separating Gravitational Wave Signals from Instrument Artifacts

Central to the gravitational wave detection problem is the challenge of separating features in the data produced by astrophysical sources from features produced by the detector. Matched filtering provides an optimal solution for Gaussian noise, but in practice, transient noise excursions or ``glitches'' complicate the analysis. Detector diagnostics and coincidence tests can be used to veto many glitches which may otherwise be misinterpreted as gravitational wave signals. The glitches that remain can lead to long tails in the matched filter search statistics and drive up the detection threshold. Here we describe a Bayesian approach that incorporates a more realistic model for the instrument noise allowing for fluctuating noise levels that vary independently across frequency bands, and deterministic ``glitch fitting'' using wavelets as ``glitch templates'', the number of which is determined by a trans-dimensional Markov chain Monte Carlo algorithm. We demonstrate the method's effectiveness on simulated data containing low amplitude gravitational wave signals from inspiraling binary black hole systems, and simulated non-stationary and non-Gaussian noise comprised of a Gaussian component with the standard LIGO/Virgo spectrum, and injected glitches of various amplitude, prevalence, and variety. Glitch fitting allows us to detect significantly weaker signals than standard techniques.Comment: 21 pages, 18 figure

An improved algorithm for narrow-band searches of continuous gravitational waves

Continuous gravitational waves signals, emitted by asymmetric spinning neutron stars, are among the main targets of current detectors like Advanced LIGO and Virgo. In the case of sources, like pulsars, which rotational parameters are measured through electromagnetic observations, typical searches assume that the gravitational wave frequency is at a given known fixed ratio with respect to the star rotational frequency. For instance, for a neutron star rotating around one of its principal axis of inertia the gravitational signal frequency would be exactly two times the rotational frequency of the star. It is possible, however, that this assumption is wrong. This is why search algorithms able to take into account a possible small mismatch between the gravitational waves frequency and the frequency inferred from electromagnetic observations have been developed. In this paper we present an improved pipeline to perform such narrow-band searches for continuous gravitational waves from neutron stars, about three orders of magnitude faster than previous implementations. The algorithm that we have developed is based on the {\it 5-vectors} framework and is able to perform a fully coherent search over a frequency band of width $\mathcal{O}$(Hertz) and for hundreds of spin-down values running a few hours on a standard workstation. This new algorithm opens the possibility of long coherence time searches for objects which rotational parameters are highly uncertain.Comment: 19 pages, 8 figures, 6 tables, submitted to CQ

Finding Strong Gravitational Lenses in the Kilo Degree Survey with Convolutional Neural Networks

The volume of data that will be produced by new-generation surveys requires automatic classification methods to select and analyze sources. Indeed, this is the case for the search for strong gravitational lenses, where the population of the detectable lensed sources is only a very small fraction of the full source population. We apply for the first time a morphological classification method based on a Convolutional Neural Network (CNN) for recognizing strong gravitational lenses in $255$ square degrees of the Kilo Degree Survey (KiDS), one of the current-generation optical wide surveys. The CNN is currently optimized to recognize lenses with Einstein radii $\gtrsim 1.4$ arcsec, about twice the $r$-band seeing in KiDS. In a sample of $21789$ colour-magnitude selected Luminous Red Galaxies (LRG), of which three are known lenses, the CNN retrieves 761 strong-lens candidates and correctly classifies two out of three of the known lenses. The misclassified lens has an Einstein radius below the range on which the algorithm is trained. We down-select the most reliable 56 candidates by a joint visual inspection. This final sample is presented and discussed. A conservative estimate based on our results shows that with our proposed method it should be possible to find $\sim100$ massive LRG-galaxy lenses at z\lsim 0.4 in KiDS when completed. In the most optimistic scenario this number can grow considerably (to maximally $\sim$2400 lenses), when widening the colour-magnitude selection and training the CNN to recognize smaller image-separation lens systems.Comment: 24 pages, 17 figures. Published in MNRA

An Overview of LISA Data Analysis Algorithms

The development of search algorithms for gravitational wave sources in the LISA data stream is currently a very active area of research. It has become clear that not only does difficulty lie in searching for the individual sources, but in the case of galactic binaries, evaluating the fidelity of resolved sources also turns out to be a major challenge in itself. In this article we review the current status of developed algorithms for galactic binary, non-spinning supermassive black hole binary and extreme mass ratio inspiral sources. While covering the vast majority of algorithms, we will highlight those that represent the state of the art in terms of speed and accuracy.Comment: 21 pages. Invited highlight article appearing in issue 01 of Gravitational Waves Notes, "GW Notes", edited by Pau Amaro-Seoane and Bernard F. Schutz at: http://brownbag.lisascience.org/lisa-gw-notes