We present a modular framework, the Workload Characterisation Framework (WCF), that is developed to obtain, store and compare key characteristics of radio astronomy processing software in a reproducible way. As a demonstration, we discuss the experiences using the framework to characterise a LOFAR calibration and imaging pipeline.Instrumentatio

Dijkema, T.J.

Emsley, I.

Grange, Y.G.

Lakhoo, R.

Mariani, G.

Mechev, A.P.

Petschow, M.

Veenboer, B.

Wu, C.

Leiden University Scholary Publications

arXiv:1612.00456v1  [astro-ph.IM]  1 Dec 2016 Characterising radio telescope software with the WorkloadCharacterisation FrameworkY. G. Grange1, R. Lakhoo2*, M. Petschow1, C. Wu3, B. Veenboer1, I. Emsley2,T. J. Dijkema1, A. P. Mechev4, and G. Mariani51 ASTRON, The Netherlands; (grange@astron.nl)2 Oxford e-Research Centre, University of Oxford, United Kingdom3 ICRAR, University of Western Australia, Australia4 Leiden Observatory and LIACS, Leiden University, The Netherlands5IBM Research, The NetherlandsAbstract. We present a modular framework, theWorkload Characterisation Frame-work (WCF), that is developed to reproducibly obtain, store and compare key char-acteristics of radio astronomy processing software. As a demonstration, we discussthe experiences using the framework to characterise a LOFARcalibration and imagingpipeline.1. IntroductionModern low-frequency radio interferometers, such as the LOw Frequency ARray (LO-FAR) (van Haarlem et al. 2013), consist of many antennas and dta processing is per-formed by software; this is reflected by the community termsoftware telescope. Suchsoftware telescopes are modular and under constant developm nt by large interdisci-plinary research collaborations, working across continens; consequently, obtaining aholistic view of the entire system becomes a challenging task. It is a considerable chal-lenge to manage, process and store such large datasets, within budgets and constraints,while “pushing the envelope” of technology. To achieve suchcost-performance opti-mum, the first aim is to obtain a quantitative understanding of the compute, energy anddata access behaviours exhibited by various radio astronomy data processing softwarepipelines and algorithms. Using low-level Linux kernel andhardware interfaces, theWorkload Characterisation Framework (WCF) can be used to support operations andsoftware development.To encourage best practices, the work undertaken requires to be reproducible: thesoftware should be open and publicly available, and it should allow for others to re-produce the software environment. In this work, we present the WCF that aims toprovide key metrics to characterise workloads in a standardand reproducible manner.The WCF is designed to be extensible and we present an exampleof such an extensionthat aims to provide further insight when identifying software bottlenecks for researchand development (R&D) purposes.12 Grange et al.2. The Workload Characterisation frameworkThe WCF is under development as part of the Local Monitoring ad Control (LMC)work package of the SKA Science Data Processor (SDP) consortium. The first purposeof the WCF is to provide the SDP compute resource scheduler with the essential work-load characteristics for each pipeline processing component, in order to make optimal(cost-effective) scheduling decisions. Secondly, the WCF aims to assist telescope de-velopers and operators to obtain a quantitative understanding of the compute, energyand data access behaviours exhibited by various pipelines,components and algorithms.Finally, the WCF aims to enable the micro-benchmarking of different compute plat-forms, which can be used for optimisation and comparison.The WCF prototypes have already been used, or tested for SKA precursor andpathfinders, including LOFAR, MWA, JVLA HDR, ASKAPSoft and the CHILIESproject.3. The LOFAR use caseThe WCF can be used for a variety of tasks; we focus on a bottleneck analysis – with theaim to support more targeted system development efforts (software and hardware). Inaddition to the typical WCF output data to assist this task, we consider non-time-seriesdata to create the link between system behaviour and the structure of the software.In this section, as an example use case, we consider a calibration and imagingprocessing pipeline, hereafter referred to as Clib, which creates sky images from theLOFAR telescope.3.1. The Calib pipelineThe Calib pipeline has been used to image HBA commissioning data of theGalacticdiffuse synchrotron emission (Iacobelli et al. 2013).For each frequency band, there are two pointings: one towards the target field andthe other towards the calibrator field. As depicted in Fig. 1,the two input measure-ment sets are fed into the pipeline. After a copy operation, the calibrator is used tosolve for the antenna gain amplitudes, which are then applied to the target. A phasecalibration is then performed on both the calibrator and thetarget data. Finally, skyimages are derived from the calibrated data sets. The entirecalibration is performedusing LOFAR’s DPPP (data preprocessing pipeline); the imagng is performed usingWSClean (Offringa et al. 2014).Figure 1. Work flow of the Calib pipeline.Characterising radio telescope software with the WCF 33.2. Goal-oriented system developmentWith goal-oriented or targeted system development, we promote the following workflow: (1) for a given micro-benchmark and platform, evaluatethe baseline performanceof both the hardware and software – ideally by replicating the results of measurementsstored in a data base; (2) use the WCF to analyse system behaviour nd identify bottle-necks; (3) concentrating on a specific part and possibly using other performance anal-ysis tools, optimize the code or change the underlying hardwe; (4) rerun the WCFmeasurements of the entire pipeline to assess the impact of the changes and store theresults in a database.With the WCF we assist a goal-oriented development in the following way: (1)Evaluating a pipeline’s performance over a long time requires a standardised storageformat of the measurements. (2) By standardising the measurment tools, the mea-surements become comparable over a wide range of use cases: comparing differentalgorithms and their implementations, different hardware, and different input data anddata formats. Among other advantages, this allows for meta-analysis of stored mea-surements for various configurations. (3) Quickly identifycomputational and otherresource bottlenecks. This information could be used to allevi te software bottlenecksand identify the most salient system features that determine performance. Besides fa-cilitating such a hardware-software co-design procedure,th insights can be used toinspire general research.4. Results020406080100CPU [%]01020Memory[GB]PhaseSolve010203040Mem bw[GB/s]0 1000 2000 3000 4000 5000 6000 7000Time [s]0100200300Disk bw[MB/s]Figure 2. Time series of system characteristics.In Fig. 2 we present time series data of a Clib pipeline run. We choose themetrics CPU and memory usage, memory bandwidth, and disk I/O bandwidth – allimportant parameters to identify performance bottlenecks. While a detailed analysis ofthe data is beyond the scope of this paper, we demonstrate itsuse for one specific step of4 Grange et al.the pipeline, PhaseSolve. This step requires roughly 55% of the entire execution time(Fig. 3; left). Further analysis indicates that the step exhibits poor scaling behaviour: theexecution using a 28 core dual-socket Intel Xeon compute nodis only 1.23 times fasterthan a sequential execution (Fig. 3; right). Using the OeRC SKA testbed, low-levelCPU characteristics were gathered, and showed less than optimal CPU core usage onaverage, a high number of CPU migrations (292/sec) and context switches (3,521/sec).The poor scaling characteristics were verified by configuring OMP affinity settings,using a Round-robin (RR) real-time CPU scheduler and CPU pinning, only resulted ina 8.2% increase in runtime using 10 fewer CPU cores than the default. A natural nextstep is to improve on the scalability of the PhaseSolve step of the pipeline. PhaseSolve Gain-CalApply-CalFlagging Phase-SolvePhase-CorrectImaging010002000300040005000Time [s]0.80x0.67x4.65x1.23x0.60x1.54xSequentialMultithreadedFigure 3. Breakdown of execution time and speedup due due parallel execution.5. ConclusionUsing a LOFAR calibration and imaging pipeline, we demonstrated the use of the WCFfor R&D radio telescope software and hardware development.Such structured ap-proach is especially relevant for the development of new instruments and algorithmsfor the SKA.Acknowledgments. We thank Marco Iacobelli for providing the Calib processingpipeline and the input data. This work is supported by the Dutch Ministry of EZ andthe province of Drenthe through the ASTRON-IBM Dome grant and the EU FP7 undergrant no ICT-610476 (DEEP-ER). The authors acknowledge theDepartment of Physicsand the OeRC at the University of Oxford, for use of their system resources and per-sonnel. The authors would like to acknowledge the use of the University of OxfordAdvanced Research Computing facility in carrying out this work.ReferencesIacobelli, M., et al. 2013, A&A, 558, A72.1308.2804Offringa, A. R., et al. 2014, MNRAS, 444, 606.1407.1943van Haarlem, M. P., et al. 2013, A&A, 556, A2.1305.3550

Characterising Radio Telescope Software With the Workload Characterisation Framework

https://scholarlypublications.universiteitleiden.nl/access/item%3A2913892/download

Characterising Radio Telescope Software With the Workload Characterisation Framework

Abstract

Similar works

Full text

Available Versions

Leiden University Scholary Publications