Interactive data analysis of data from high energy physics experiments using Apache Spark

The primary goal of the project was to evaluate a set of Big Data tools for the analysis of the data from the TOTEM experiment which will enable interactive or semi-interactive work with large amounts of data. The product is a set of the analysis codes and notebooks written in a distributed model, together with their performance profiling, reports, and execution results. Our analysis application has several requirements to fulfill: correctness of the results, capacity to work interactively using data-science notebooks, scalability of the solution, simple and easy way to visualize the results, use of existing storage services. The significant characteristic of this analysis code is the application of the RDataFrame model. In the time of our work, RDataFrame was an innovation introduced in the newest release of ROOT and was still under heavy development. The concept is in general similar to the data frames known from other languages like Python or R and provides a high-level interface to work with data in ROOT format. Generally, it introduces a bit of a functional approach in the mostly iterative analyses done in C++ ROOT giving users interesting possibilities, like implicit multithreading. We run a sample analysis on 4.7TB of data from the TOTEM experiment, rewriting the analysis code to leverage the PyRoot and RDataFrame model and to take full advantage of the parallel processing capabilities offered by Apache Spark. The analysis was evaluated on a system that combines the use of public cloud infrastructure (Helix Nebula Science Cloud), and storage and processing services developed by CERN (Science Box)

Methods to optimize rare-event Monte Carlo reliability simulations for Large Hadron Collider Protection Systems

Machine Protection Systems of the Large Hadron Collider (LHC) are safety-critical systems that adhere to stringent reliability and availability requirements. High reliability is required because the energy of the particles beam and the energy stored in the magnets are high enough to damage the machine beyond repair. High availability is required to maximize the amount of data collected by the experiments. Both constraints will become even more critical for LHC upgrades (High Luminosity LHC) and a possible larger accelerator (Future Circular Collider). The field of availability and reliability studies provides a variety of tools well-suited for those purposes. Among them, Monte Carlo simulation is a remarkably flexible and versatile option for in-depth analyses of complex systems. This thesis reviews available methods for increasing computations efficiency in generic rare-event simulations framework used for reliability engineering purposes. Efforts to reduce the workload of MC simulations have been ongoing for a very long time, since the 1950s, and include methods such as importance sampling (IS), importance splitting (ISp), and randomized quasi-Monte Carlo (RQMC) method. More recent developments usually build on top of those methods. Based on a review of popular methods employed across the rare-event simulations field, ISp and RQMC methods were selected as promising solutions for our reliability and availability simulations. The empirical tests are a working proof of concept and show potential for remarkable improvements for reliability simulations of rare events with ISp and availability prediction with RQMC. Alongside the rare-event methods, the project also involved an analysis of HL-LHC Energy Extraction system reliability, which serves as an introduction for reliability modelling and is presented in a report attached to the thesis. The project involved contributions to the AvailSim4 MC framework developed at CERN

Reliability Analysis of the HL-LHC Energy Extraction System

The energy extraction systems for the protection of the new HL-LHC superconducting magnet circuits are based on vacuum breakers. This technology allows to significantly reduce the switch opening time and increases the overall system reliability with reduced maintenance needs. This paper presents the results of detailed reliability studies performed on these new energy extraction systems. The study quantifies the risk of a failure which prevents correct protection of a magnet circuit and identifies the most critical components of the system. For this, the model considers factors such as block or component level failure probabilities, different maintenance strategies and repair procedures. The reliability simulations have been performed with AvailSim4, a novel Monte Carlo code for availability and reliability simulations. The results are compared with the system reliability requirements and provides insights into the most critical components

Reliability studies for CERN’s new safe machine parameter system

The Safe Machine Parameter system (SMP) is a critical part of the machine protection system in CERN’s Large Hadron Collider (LHC) and the Super Proton Synchrotron (SPS). It broadcasts safety-critical parameters like beam energy, beam intensity, the beta functions and flags indicating safety levels of the beam to other machine protection elements. The current SMP will be replaced by a consolidated system during CERN’s Long Shutdown 3, foreseen to start in 2026. In this contribution the results of the reliability study of the new SMP system are presented. This study quantifies the criticality of end-users by identifying the hazard chains leading to potential damage of the involved equipment. Data-driven risk matrices are used to derive acceptable failure frequencies and reliability requirements. The study encompasses Monte Carlo simulations of sub-system level configurations to support the decision-making process in this project

Workshop on Dust Charging and Beam-Dust Interaction in Particle Accelerators

The effects of beam-dust interactions have been observed in particle accelerators and have been studied as early as the 1960s. The goal of this workshop was to: • Improve understanding of beam-dust interactions in particle accelerators, in particular, of dust-charging and release mechanisms. • Improve understanding of evolution of beam-dust interaction rate as a function of beam and other parameters. • Present modelling work on beam-dust interactions and their consequences. • Present research on dust issues in space applications. • Improve understanding of the behaviour of dust particles in accelerator hardware systems (Vacuum, RF, treated surfaces. . . ) and their consequences. • Improve understanding of mechanisms of dust migration into sensitive devices, such as high field gradient superconducting cavities, and ways to prevent this migration. • Identify synergies between particle-accelerator and space-research communities. • Define next research steps and possible collaborations. The workshop was held at CERN, Geneva, Switzerland, from June 13 to 15, 2023. All presentations given during the workshop can be found on the Indico website. In this report, we document the main conclusions and questions discussed during the workshop

Big Data Tools and Cloud Services for High Energy Physics Analysis in TOTEM Experiment

The High Energy Physics community has been developing dedicated solutions for processing experiment data over decades. However, with recent advancements in Big Data and Cloud Services, a question of application of such technologies in the domain of physics data analysis becomes relevant. In this paper, we present our initial experience with a system that combines the use of public cloud infrastructure (Helix Nebula Science Cloud), storage and processing services developed by CERN, and off-the-shelf Big Data frameworks. The system is completely decoupled from CERN main computing facilities and provides an interactive web-based interface based on Jupyter Notebooks as the main entry-point for the users. We run a sample analysis on 4.7 TB of data from the TOTEM experiment, rewriting the analysis code to leverage the PyRoot and RDataFrame model and to take full advantage of the parallel processing capabilities offered by Apache Spark. We report on the experience collected by embracing this new analysis model: preliminary scalability results show the processing time of our dataset can be reduced from 13 hrs on a single core to 7 mins on 248 cores