Characterising Radio Telescope Software With the Workload Characterisation Framework

We present a modular framework, the Workload Characterisation Framework (WCF), that is developed to obtain, store and compare key characteristics of radio astronomy processing software in a reproducible way. As a demonstration, we discuss the experiences using the framework to characterise a LOFAR calibration and imaging pipeline.Instrumentatio

Astronomical data organization, management and access in Scientific Data Lakes

The data volumes stored in telescope archives is constantly increasing due to the development and improvements in the instrumentation. Often the archives need to be stored over a distributed storage architecture, provided by independent compute centres. Such a distributed data archive requires overarching data management orchestration. Such orchestration comprises of tools which handle data storage and cataloguing, and steering transfers integrating different storage systems and protocols, while being aware of data policies and locality. In addition, it needs a common Authorisation and Authentication Infrastructure (AAI) layer which is perceived as a single entity by end users and provides transparent data access. The scientific domain of particle physics also uses complex and distributed data management systems. The experiments at the Large Hadron Collider (LHC) accelerator at CERN generate several hundred petabytes of data per year. This data is globally distributed to partner sites and users using national compute facilities. Several innovative tools were developed to successfully address the distributed computing challenges in the context of the Worldwide LHC Computing Grid (WLCG). The work being carried out in the ESCAPE project and in the Data Infrastructure for Open Science (DIOS) work package is to prototype a Scientific Data Lake using the tools developed in the context of the WLCG, harnessing different physics scientific disciplines addressing FAIR standards and Open Data. We present how the Scientific Data Lake prototype is applied to address astronomical data use cases. We introduce the software stack and also discuss some of the differences between the domains

The LOFAR Two-metre Sky Survey

In this data release from the ongoing LOw-Frequency ARray (LOFAR) Two-metre Sky Survey we present 120–168 MHz images covering 27% of the northern sky. Our coverage is split into two regions centred at approximately 12h45m +44◦300 and 1h00m +28◦000 and spanning 4178 and 1457 square degrees respectively. The images were derived from 3451 h (7.6 PB) of LOFAR High Band Antenna data which were corrected for the direction-independent instrumental properties as well as direction-dependent ionospheric distortions during extensive, but fully automated, data processing. A catalogue of 4 396 228 radio sources is derived from our total intensity (Stokes I) maps, where the majority of these have never been detected at radio wavelengths before. At 600 resolution, our full bandwidth Stokes I continuum maps with a central frequency of 144 MHz have: a median rms sensitivity of 83 µJy beam−1 ; a flux density scale accuracy of approximately 10%; an astrometric accuracy of 0.200; and we estimate the point-source completeness to be 90% at a peak brightness of 0.8 mJy beam−1 . By creating three 16 MHz bandwidth images across the band we are able to measure the in-band spectral index of many sources, albeit with an error on the derived spectral index of >±0.2 which is a consequence of our flux-density scale accuracy and small fractional bandwidth. Our circular polarisation (Stokes V) 2000 resolution 120–168 MHz continuum images have a median rms sensitivity of 95 µJy beam−1 , and we estimate a Stokes I to Stokes V leakage of 0.056%. Our linear polarisation (Stokes Q and Stokes U) image cubes consist of 480 × 97.6 kHz wide planes and have a median rms sensitivity per plane of 10.8 mJy beam−1 at 40 and 2.2 mJy beam−1 at 2000; we estimate the Stokes I to Stokes Q/U leakage to be approximately 0.2%. Here we characterise and publicly release our Stokes I, Q, U and V images in addition to the calibrated uv-data to facilitate the thorough scientific exploitation of this unique dataset

LOFAR Two-metre Sky Survey (LoTSS) DR2

VizieR online Data Catalogue associated with article published in journal Astronomy & Astrophysics with title 'The LOFAR Two-metre Sky Survey (LoTSS). V. Second data release.' (bibcode: 2022A&A...659A...1S