An Automated Scalable Framework for Distributing Radio Astronomy Processing Across Clusters and Clouds

The Low Frequency Array (LOFAR) radio telescope is an international aperture synthesis radio telescope used to study the Universe at low frequencies. One of the goals of the LOFAR telescope is to conduct deep wide-field surveys. Here we will discuss a framework for the processing of the LOFAR Two Meter Sky Survey (LoTSS). This survey will produce close to 50 PB of data within five years. These data rates require processing at locations with high-speed access to the archived data. To complete the LoTSS project, the processing software needs to be made portable and moved to clusters with a high bandwidth connection to the data archive. This work presents a framework that makes the LOFAR software portable, and is used to scale out LOFAR data reduction. Previous work was successful in pre-processing LOFAR data on a cluster of isolated nodes. This framework builds upon it and and is currently operational. It is designed to be portable, scalable, automated and general. This paper describes its design and high level operation and the initial results processing LoTSS data

The LOFAR Two-metre Sky Survey V. Second data release

In this data release from the ongoing LOw-Frequency ARray (LOFAR) Two-metre Sky Survey we present 120a 168 MHz images covering 27% of the northern sky. Our coverage is split into two regions centred at approximately 12h45m +44 30a and 1h00m +28 00a and spanning 4178 and 1457 square degrees respectively. The images were derived from 3451 h (7.6 PB) of LOFAR High Band Antenna data which were corrected for the direction-independent instrumental properties as well as direction-dependent ionospheric distortions during extensive, but fully automated, data processing. A catalogue of 4 396 228 radio sources is derived from our total intensity (Stokes I) maps, where the majority of these have never been detected at radio wavelengths before. At 6a resolution, our full bandwidth Stokes I continuum maps with a central frequency of 144 MHz have: a median rms sensitivity of 83 μJy beama 1; a flux density scale accuracy of approximately 10%; an astrometric accuracy of 0.2a; and we estimate the point-source completeness to be 90% at a peak brightness of 0.8 mJy beama 1. By creating three 16 MHz bandwidth images across the band we are able to measure the in-band spectral index of many sources, albeit with an error on the derived spectral index of > a ±a 0.2 which is a consequence of our flux-density scale accuracy and small fractional bandwidth. Our circular polarisation (Stokes V) 20a resolution 120a168 MHz continuum images have a median rms sensitivity of 95 μJy beama 1, and we estimate a Stokes I to Stokes V leakage of 0.056%. Our linear polarisation (Stokes Q and Stokes U) image cubes consist of 480a A a 97.6 kHz wide planes and have a median rms sensitivity per plane of 10.8 mJy beama 1 at 4a and 2.2 mJy beama 1 at 20a; we estimate the Stokes I to Stokes Q/U leakage to be approximately 0.2%. Here we characterise and publicly release our Stokes I, Q, U and V images in addition to the calibrated uv-data to facilitate the thorough scientific exploitation of this unique dataset

Building LOFAR as a Service

The LOFAR radio telescope is a low-frequency aperture synthesis radio telescope with headquarters in the Netherlands and stations across Europe. As a general purpose telescope, LOFAR produces petabytes of data each year serving a wide range of science cases. The data volumes produced are difficult or impossible to process on a single machine or even a small cluster at a scientific institute. We provide a layout for serving LOFAR processing to the astronomical community by providing access to LOFAR pipelines accelerated on a high throughput platform. We build this on our previous success with parallelizing the LOFAR Surveys pipeline and with creating automated LOFAR workflows on a distributed architecture. The LOFAR As A Service platform will serve the LOFAR Key Science Projects (KSPs), specifically the LOFAR Surveys KSP, which aims to provide science ready products to the scientific community. Additionally, this system will provide a robust method to re-process LOFAR data with a single click.Instrumentatio

Technology and Dementia Preconference

BACKGROUND: Various neurodegenerative and psychiatric diseases are related with changes in spoken language [1], although they have seldom been investigated. We evaluate the effectiveness of our language-agnostic Machine Learning (ML) system to detect subtle changes in spoken language that manifest early signs of cognitive decline, thus assisting with its diagnosis. We evaluate our methodology using recordings of speech samples from multiple languages obtained from patient cohorts in early stages of cognitive decline and matched healthy controls. METHOD: Our methodology involves capturing patient audio recordings while they are performing one or more predefined cognitive assessment tasks, involving, e.g., the description of a picture or recounting of an everyday activity. Afterwards, multi-language audio recordings and generated transcripts are analyzed with audio and NLP feature extraction methods [3], ranging from semantic, morpho-syntactic, phonological representations of the input, as well as, more sophisticated linguistic measures. The feature pool is filtered by a Pearson's rho threshold of 0.85. We build a Random Forest classifier out of 100 Decision trees, using the Gini impurity criterion, 5-fold cross-validation for training, elimination and composition-based feature selection, as well as post-selection retraining / fine-tuning. The model's diagnostic performance is evaluated on a test set unseen during training. RESULT: Our results are validated against the diagnosis that is provided by medical experts. Our performance in terms of accuracy (∼82%), f1 (84%) and ROC-AUC score (∼82%) are clear indicators of the effectiveness of speech analysis towards detecting cognitive decline. Moreover, our tree-based classifier produces probability scores that closely follow the proportion of pathological cases in the input data, with a correlation of 94%. CONCLUSION: In the current evaluation we verified our conjectures regarding the strong capacity of speech to predict cognitive decline. Audio analysis and machine learning are proven to be invaluable tools in the prediction of early signs of cognitive decline, which are coupled with a wide spectrum of neurodegenerative and psychiatric diseases. [1] Boschi, Veronica, et al., Frontiers in psychology 8 (2017): 269. [2] Vassiliki Rentoumi et al., Alzheimer's & Dementia, Wiley, volume 16, 2020. [3] Alberdi, Ane et al., Artificial intelligence in medicine 71 (2016): 1-29. © 2021 the Alzheimer's Association

The LOFAR Two-metre Sky Survey. II. First data release

International audienceThe LOFAR Two-metre Sky Survey (LoTSS) is an ongoing sensitive, high-resolution 120–168 MHz survey of the entire northern sky for which observations are now 20% complete. We present our first full-quality public data release. For this data release 424 square degrees, or 2% of the eventual coverage, in the region of the HETDEX Spring Field (right ascension 10h45m00s to 15h30m00s and declination 45°00′00″ to 57°00′00″) were mapped using a fully automated direction-dependent calibration and imaging pipeline that we developed. A total of 325 694 sources are detected with a signal of at least five times the noise, and the source density is a factor of ∼10 higher than the most sensitive existing very wide-area radio-continuum surveys. The median sensitivity is S144 MHz = 71 μJy beam−1 and the point-source completeness is 90% at an integrated flux density of 0.45 mJy. The resolution of the images is 6″ and the positional accuracy is within 0.2″. This data release consists of a catalogue containing location, flux, and shape estimates together with 58 mosaic images that cover the catalogued area. In this paper we provide an overview of the data release with a focus on the processing of the LOFAR data and the characteristics of the resulting images. In two accompanying papers we provide the radio source associations and deblending and, where possible, the optical identifications of the radio sources together with the photometric redshifts and properties of the host galaxies. These data release papers are published together with a further ∼20 articles that highlight the scientific potential of LoTSS.Key words: surveys / catalogs / radio continuum: general / techniques: image processing⋆ LoTSS.⋆⋆ The catalogue is only available at the CDS via anonymous ftp to cdsarc.u-strasbg.fr (130.79.128.5) or via http://cdsarc.u-strasbg.fr/viz-bin/qcat?J/A+A/622/A