The Technologies Required for Fusing HPC and Real-Time Data to Support Urgent Computing

The use of High Performance Computing (HPC) to compliment urgent decision making in the event of disasters is an important future potential use of supercomputers. However, the usage modes involved are rather different from how HPC has been used traditionally. As such, there are many obstacles that need to be overcome, not least the unbounded wait times in the batch system queues, to make the use of HPC in disaster response practical. In this paper, we present how the VESTEC project plans to overcome these issues and develop a working prototype of an urgent computing control system. We describe the requirements for such a system and analyse the different technologies available that can be leveraged to successfully build such a system. We finally explore the design of the VESTEC system and discuss ongoing challenges that need to be addressed to realise a production level system.Comment: Preprint of paper in 2019 IEEE/ACM HPC for Urgent Decision Making (UrgentHPC

Predicting batch queue job wait times for informed scheduling of urgent HPC workloads

There is increasing interest in the use of HPC machines for urgent workloads to help tackle disasters as they unfold. Whilst batch queue systems are not ideal in supporting such workloads, many disadvantages can be worked around by accurately predicting when a waiting job will start to run. However there are numerous challenges in achieving such a prediction with high accuracy, not least because the queue's state can change rapidly and depend upon many factors. In this work we explore a novel machine learning approach for predicting queue wait times, hypothesising that such a model can capture the complex behaviour resulting from the queue policy and other interactions to generate accurate job start times. For ARCHER2 (HPE Cray EX), Cirrus (HPE 8600) and 4-cabinet (HPE Cray EX) we explore how different machine learning approaches and techniques improve the accuracy of our predictions, comparing against the estimation generated by Slurm. We demonstrate that our techniques deliver the most accurate predictions across our machines of interest, with the result of this work being the ability to predict job start times within one minute of the actual start time for around 65\% of jobs on ARCHER2 and 4-cabinet, and 76\% of jobs on Cirrus. When compared against what Slurm can deliver, this represents around 3.8 times better accuracy on ARCHER2 and 18 times better for Cirrus. Furthermore our approach can accurately predicting the start time for three quarters of all job within ten minutes of the actual start time on ARCHER2 and 4-cabinet, and for 90\% of jobs on Cirrus. Whilst the driver of this work has been to better facilitate placement of urgent workloads across HPC machines, the insights gained can be used to provide wider benefits to users and also enrich existing batch queue systems and inform policy too.Comment: Preprint of article at the 2022 Cray User Group (CUG

The formation and eruption of magnetic flux ropes in solar and stellar coronae

Flux ropes are magnetic structures commonly found in the solar corona. They are thought to play an important role in solar flares and coronal mass ejections. Understanding their formation and eruption is of paramount importance for our understanding of space weather. In this thesis the magnetofrictional method is applied to simulate the formation of flux ropes and track their evolution up to eruption both in solar and stellar coronae. Initially, the coronal magnetic field of a solar active region is simulated using observed magnetograms to drive the coronal evolution. From the sequence of magnetograms the formation of a flux rope is simulated, and compared with coronal observations. Secondly a procedure to produce proxy SOLIS synoptic magnetograms from SDO/HMI and SOHO/MDI magnetograms is presented. This procedure allows SOLIS-like synoptic magnetograms to be produced during times when SOLIS magnetograms are not available. Thirdly, a series of scaling laws for the formation and life-times of flux ropes in stellar coronae are determined as a function of stellar differential rotation and surface diffusion. These scaling laws can be used to infer the response of stellar coronae to the transport of magnetic fields at their surface. Finally, global long-term simulations of stellar corona are carried out to determine the coronal response to flux emergence and differential rotation. A bipole emergence model is developed and is used in conjunction with a surface flux transport model in order to drive the global coronal evolution. These global simulations allow the flux, energy and flux rope distributions to be studied as a function of a star’s differential rotation and flux emergence rate

Prevalence of liver fluke (Fasciola hepatica) in wild Red Deer (Cervus elaphus): coproantigen ELISA is a practicable alternative to faecal egg counting for surveillance in remote populations

Red deer (Cervus elaphus) are hosts of liver fluke (Fasciola hepatica); yet, prevalence is rarely quantified in wild populations. Testing fresh samples from remote regions by faecal examination (FE) can be logistically challenging; hence, we appraise frozen storage and the use of a coproantigen ELISA (cELISA) for F. hepatica surveillance. We also present cELISA surveillance data for red deer from the Highlands of Scotland. Diagnoses in faecal samples (207 frozen, 146 fresh) were compared using a cELISA and by FE. For each storage method (frozen or fresh), agreement between the two diagnostics was estimated at individual and population levels, where population prevalence was stratified into cohorts (e.g., by sampling location). To approximate sensitivity and specificity, 65 post-slaughter whole liver examinations were used as a reference. At the individual level, FE and cELISA diagnoses agreed moderately (κfrozen = 0.46; κfresh = 0.51), a likely reflection of their underlying principles. At the population level, FE and cELISA cohort prevalence correlated strongly (Pearson’s R = 0.89, p < 0.0001), reflecting good agreement on relative differences between cohort prevalence. In frozen samples, prevalence by cELISA exceeded FE overall (42.8% vs. 25.8%) and in 9/12 cohorts, alluding to differences in sensitivity; though, in fresh samples, no significant difference was found. In 959 deer tested by cELISA across the Scottish Highlands, infection prevalence ranged from 9.6% to 53% by sampling location. We highlight two key advantages of cELISA over FE: i) the ability to store samples long term (frozen) without apparent loss in diagnostic power; and ii) reduced labour and the ability to process large batches. Further evaluation of cELISA sensitivity in red deer, where a range of fluke burdens can be obtained, is desirable. In the interim, the cELISA is a practicable diagnostic for F. hepatica surveillance in red deer, and its application here has revealed considerable geographic, temporal, sex and age related differences in F. hepatica prevalence in wild Scottish Highland red deer

The role of interactive super-computing in using HPC for urgent decision making

Technological advances are creating exciting new opportunities that have the potential to move HPC well beyond traditional computational workloads. In this paper we focus on the potential for HPC to be instrumental in responding to disasters such as wildfires, hurricanes, extreme flooding, earthquakes, tsunamis, winter weather conditions, and accidents. Driven by the VESTEC EU funded H2020 project, our research looks to prove HPC as a tool not only capable of simulating disasters once they have happened, but also one which is able to operate in a responsive mode, supporting disaster response teams making urgent decisions in real-time. Whilst this has the potential to revolutionise disaster response, it requires the ability to drive HPC interactively, both from the user's perspective and also based upon the arrival of data. As such interactivity is a critical component in enabling HPC to be exploited in the role of supporting disaster response teams so that urgent decision makers can make the correct decision first time, every time

Peer collaboration as a way of developing effective pedagogy for including cultural, linguistic, and learning diversity

Teacher educators have begun to recognize that fundamental changes are needed to support teachers in meeting the challenges of increasing diversity in public schools. Using concept analysis to study our collaboration, we developed and implemented a framework that would move a consideration of diversity from the edges to the mainstream in teacher education would do this. We use a narrative strategy to identify five situations that led to the development and implementation of the Inclusive Pedagogy Framework. In our session, we will involve participants in analysis of five critical incidents as a strategy for exploring validity and interpretation in self-study data analysis

Activation of an Endogenous Retrovirus-Associated Long Non-Coding RNA in Human Adenocarcinoma

Background Long non-coding RNAs (lncRNAs) are emerging as molecules that significantly impact many cellular processes and have been associated with almost every human cancer. Compared to protein-coding genes, lncRNA genes are often associated with transposable elements, particularly with endogenous retroviral elements (ERVs). ERVs can have potentially deleterious effects on genome structure and function, so these elements are typically silenced in normal somatic tissues, albeit with varying efficiency. The aberrant regulation of ERVs associated with lncRNAs (ERV-lncRNAs), coupled with the diverse range of lncRNA functions, creates significant potential for ERV-lncRNAs to impact cancer biology. Methods We used RNA-seq analysis to identify and profile the expression of a novel lncRNA in six large cohorts, including over 7,500 samples from The Cancer Genome Atlas (TCGA). Results We identified the tumor-specific expression of a novel lncRNA that we have named Endogenous retroViral-associated ADenocarcinoma RNA or ‘EVADR’, by analyzing RNA-seq data derived from colorectal tumors and matched normal control tissues. Subsequent analysis of TCGA RNA-seq data revealed the striking association of EVADR with adenocarcinomas, which are tumors of glandular origin. Moderate to high levels of EVADR were detected in 25 to 53% of colon, rectal, lung, pancreas and stomach adenocarcinomas (mean = 30 to 144 FPKM), and EVADR expression correlated with decreased patient survival (Cox regression; hazard ratio = 1.47, 95% confidence interval = 1.06 to 2.04, P = 0.02). In tumor sites of non-glandular origin, EVADR expression was detectable at only very low levels and in less than 10% of patients. For EVADR, a MER48 ERV element provides an active promoter to drive its transcription. Genome-wide, MER48 insertions are associated with nine lncRNAs, but none of the MER48-associated lncRNAs other than EVADR were consistently expressed in adenocarcinomas, demonstrating the specific activation of EVADR. The sequence and structure of the EVADR locus is highly conserved among Old World monkeys and apes but not New World monkeys or prosimians, where the MER48 insertion is absent. Conservation of the EVADR locus suggests a functional role for this novel lncRNA in humans and our closest primate relatives. Conclusions Our results describe the specific activation of a highly conserved ERV-lncRNA in numerous cancers of glandular origin, a finding with diagnostic, prognostic and therapeutic implications

A Bespoke Workflow Management System for Data-Driven Urgent HPC

In this paper we present a workflow management system which permits the kinds of data-driven workflows required by urgent computing, namely where new data is integrated into the workflow as a disaster progresses in order refine the predictions as time goes on. This allows the workflow toadapt to new data at runtime, a capability that most workflow management systems do not possess. The workflow management system was developed for the EU-funded VESTEC project, which aims to fuse HPC with real-time data for supporting urgent decision making. We first describe an example workflow from the VESTEC project, and show why existing workflow technologies do not meet the needs of the project. We then go on to present the design of our Workflow Management System, describe how it is implemented into the VESTEC system, and provide an example of the workflow system in use for a test case

Spontaneous Clearance Of Vertically Acquired Hepatitis C Infection: Implications For Testing And Treatment

BACKGROUND: Current guidelines recommend that infants born to women with hepatitis C (HCV) viremia are screened for HCV antibody at age 18 months, and if positive, referred for RNA testing at 3 years to confirm chronic infection. This policy is based in part on analyses suggesting 25%-40% of vertically acquired HCV infections clear spontaneously within 4-5 years. METHODS: Data on 179 infants with HCV RNA and/or anti-HCV evidence of vertically acquired infection in three prospective European cohorts were investigated. Ages at clearance of infection were estimated taking account of interval censoring and delayed entry. We also investigated clearance in initially HCV RNA negative infants in whom RNA was not detectable until after 6 weeks. RESULTS: Clearance rates are initially high then decline slowly. Apparently, many infections clear before they can be confirmed. An estimated 65.9% (50.1-81.6) of confirmed infections cleared by 5 years, at a median 12.4 (7.1-18.9) months. If treatment began at age 6 months, 18 months or 3 years, at least 59.0% (42.0-76.9), 39.7% (17.9-65.9), and 20.9% (4.6-44.8) of those treated would clear without treatment. In seven (6.6%) confirmed infections, RNA was not detectable until after 6 weeks, and in 2 (1.9%) not until after 6 months. However, all such cases subsequently cleared. CONCLUSIONS: Most confirmed infection clears by age 3 years. Treatment before age 3, if it was available, would avoid loss to follow-up, but would result in substantial over-treatment

Overall vertical transmission of HCV, transmission net of clearance, and timing of transmission

Background: It is widely accepted that the risk of HCV vertical transmission (VT) is 5-6% in mono-infected women, and that 25-40% of HCV infection clears spontaneously within 5 years. However, there is no consensus on how VT rates should be estimated, and there is a lack of information on VT rates “net” of clearance. // Methods: We re-analysed data on 1749 children in 3 prospective cohorts to obtain coherent estimates of overall VT rate and VT rates “net” of clearance at different ages. Clearance rates were used to impute the proportion of uninfected children who had been infected and then cleared before testing negative. The proportion of transmission early in utero, late in utero and at delivery was estimated from data on the proportion of HCV RNA positive within three days of birth, and differences between elective caesarean and non-elective caesarean deliveries. // Findings: Overall VT rates were 7.2% (95% credible interval 5.6-8.9) in mothers who were HIV negative and 12.1% (8.6-16.8) in HIV-co-infected women. The corresponding rates net of clearance at 5 years were 2.4% (1.1-4.1) and 4.1% (1.7-7.3). We estimated that 24.8% (12.1-40.8) of infections occur early in utero, 66.0% (42.5-83.3) later in utero, and 9.3% (0.5-30.6) during delivery. // Conclusion: Overall VT rates are about 24% higher than previously assumed, but the risk of infection persisting beyond age 5 years is about 38% lower. The results can inform design of trials of to prevent or treat pediatric HCV infection, and strategies to manage children exposed in utero