Uncertainty Aware Deep Learning for Particle Accelerators

Standard deep learning models for classification and regression applications are ideal for capturing complex system dynamics. However, their predictions can be arbitrarily inaccurate when the input samples are not similar to the training data. Implementation of distance aware uncertainty estimation can be used to detect these scenarios and provide a level of confidence associated with their predictions. In this paper, we present results from using Deep Gaussian Process Approximation (DGPA) methods for errant beam prediction at Spallation Neutron Source (SNS) accelerator (classification) and we provide an uncertainty aware surrogate model for the Fermi National Accelerator Lab (FNAL) Booster Accelerator Complex (regression).Comment: 6 pages, 2 figures, Neurips Physical Sciences Worksho

Hydra: Computer Vision for Data Quality Monitoring

Hydra is a system which utilizes computer vision to perform near real time data quality management, initially developed for Hall-D in 2019. Since then, it has been deployed across all experimental halls at Jefferson Lab, with the CLAS12 collaboration in Hall-B being the first outside of GlueX to fully utilize Hydra. The system comprises back end processes that manage the models, their inferences, and the data flow. The front-end components, accessible via web pages, allow detector experts and shift crews to view and interact with the system. This talk will give an overview of the Hydra system as well as highlight significant developments in Hydra's feature set, acute challenges with operating Hydra in all halls, and lessons learned along the way

Hydra: Computer Vision for Online Data Quality Monitoring

Hydra is a system utilizing computer vision for near real-time data quality monitoring. Currently operational across all of Jefferson Lab’s experimental halls, it reduces the workload of shift takers by autonomously monitoring diagnostic plots during experiments. Hydra uses "off-the-shelf" supervised learning technologies and is supported by a comprehensive MySQL database. To simplify access, web apps have been developed to facilitate both labeling and monitoring of Hydra’s inferences. Hydra can connect with the alarm system and incorporates complete historical tracking, enabling it to identify issues that shift takers could miss. When issues are detected, a natural first question is: "Why does Hydra think there is a problem?" To answer, Hydra employs Gradient-weighted Class Activation Maps (GradCAM) to identify regions of the image that are important for the specific classification. This interpretive layer enhances transparency and trustworthiness, which is essential for integration with experiment workflows and operation. The Hydra system, results, and sociological considerations for deployment will be discussed

Uncertainty Aware ML-based surrogate models for particle accelerators: A Study at the Fermilab Booster Accelerator Complex

Standard deep learning methods, such as Ensemble Models, Bayesian Neural Networks and Quantile Regression Models provide estimates to prediction uncertainties for data-driven deep learning models. However, they can be limited in their applications due to their heavy memory, inference cost, and ability to properly capture out-of-distribution uncertainties. Additionally, some of these models require post-training calibration which limits their ability to be used for continuous learning applications. In this paper, we present a new approach to provide prediction with calibrated uncertainties that includes out-of-distribution contributions and compare it to standard methods on the Fermi National Accelerator Laboratory (FNAL) Booster accelerator complex

Uncertainty aware anomaly detection to predict errant beam pulses in the SNS accelerator

High-power particle accelerators are complex machines with thousands of pieces of equipmentthat are frequently running at the cutting edge of technology. In order to improve the day-to-dayoperations and maximize the delivery of the science, new analytical techniques are being exploredfor anomaly detection, classification, and prognostications. As such, we describe the applicationof an uncertainty aware Machine Learning method, the Siamese neural network model, to predictupcoming errant beam pulses using the data from a single monitoring device. By predicting theupcoming failure, we can stop the accelerator before damage occurs. We describe the acceleratoroperation, related Machine Learning research, the prediction performance required to abort beamwhile maintaining operations, the monitoring device and its data, and the Siamese method andits results. These results show that the researched method can be applied to improve acceleratoroperations.Comment: 11 pages, 15 figures, for PR-A

Multi-module based CVAE to predict HVCM faults in the SNS accelerator

We present a multi-module framework based on Conditional Variational Autoencoder (CVAE) to detect anomalies in the power signals coming from multiple High Voltage Converter Modulators (HVCMs). We condition the model with the specific modulator type to capture different representations of the normal waveforms and to improve the sensitivity of the model to identify a specific type of fault when we have limited samples for a given module type. We studied several neural network (NN) architectures for our CVAE model and evaluated the model performance by looking at their loss landscape for stability and generalization. Our results for the Spallation Neutron Source (SNS) experimental data show that the trained model generalizes well to detecting multiple fault types for several HVCM module types. The results of this study can be used to improve the HVCM reliability and overall SNS uptim

AI4EIC Hackathon: PID with the ePIC dRICH

The inaugural AI4EIC Hackathon unfolded as a high-point satellite event during the second AI4EIC Workshop at William & Mary. The workshop itself boasted over two hundred participants in a hybrid format and delved into the myriad applications of Artificial Intelligence and Machine Learning (AI/ML) for the Electron-Ion Collider (EIC). This workshop aimed to catalyze advancements in AI/ML with applications ranging from advancements in accelerator and detector technologies—highlighted by the ongoing work on the ePIC detector and potential development of a second detector for the EIC—to data analytics, reconstruction, and particle identification, as well as the synergies between theoretical and experimental research. Complementing the technical agenda was an enriched educational outreach program that featured tutorials from leading AI/ML experts representing academia, national laboratories, and industry. The hackathon, held on the final day, showcased international participation with ten teams from around the globe. Each team, comprising up to four members, focused on the dual-radiator Ring Imaging Cherenkov (dRICH) detector, an integral part of the particle identification (PID) system in ePIC. The data for the hackathon were generated using the ePIC software suite. While the hackathon presented questions of increasing complexity, its challenges were designed with deliberate simplifications to serve as a preliminary step toward the integration of machine learning and deep learning techniques in PID with the dRICH detector. This article encapsulates the key findings and insights gained from this unique experience

Anomaly Detection and Feature Alignment for Time Series Data

Time series data are stemming from various applications that describe certain observations or quantities of interest over time. Their analysis typically involves the comparison (with reference data for anomaly detection) and feature alignment across different time series data sequences. General technique for anomaly detection via visualization is to compare a live signal along with reference sequences. Currently, the standard methods used in the industry are line/scatter plots. Due to limitations such as cluttering, lack of quantitative information etc., these plots are not effective. In this thesis, probabilistic envelope based technique is proposed for the visualization and anomaly detection of time series data. This technique provides quantitative information, is able to avoid the outliers in the reference data, and works well even with a large number of reference sequences. To demonstrate the practical use of the probabilistic envelope technique, it is applied in detection of over/under gauge of bore holes (wells). The implementation of gauge detection along with some results are also presented in this thesis. For feature alignment, the Dynamic Time Warping (DTW) is the standard approach to achieve an optimal alignment between two temporal signals. There are different variations of DTW proposed to address different needs of signal alignment or classifications. However, there lacks a comprehensive evaluation of their performance in these time series data processing tasks. Most DTW metrics are reported with good performance on certain types of time series data without a clear explanation of this performance. To address that, a synthesis framework is proposed to model the variation between two time series data sequences for comparison. The synthesis framework can produce realistic initial signal and deform it with controllable variation that mimics the real-world scenarios. With this synthesis framework, a large number of time series pairs with different but known variations can be produced, which are used to assess the performance of a number of well-known DTW measure in the tasks of alignment and classification. Their performance on different types of variations are reported and the proper DTW measure is suggested based on the type of variations between two time series sequences. This is the first time such a guideline for selecting proper DTW measure is presented.Computer Science, Department o

AI Enabled Data Quality Monitoring with Hydra

Data quality monitoring is critical to all experiments impacting the quality of any physics results. Traditionally, this is done through an alarm system, which detects low level faults, leaving higher level monitoring to human crews. Artificial Intelligence is beginning to find its way into scientific applications, but comes with difficulties, relying on the acquisition of new skill sets, either through education or acquisition, in data science. This paper will discuss the development and deployment of the Hydra monitoring system in production at Gluex. It will show how “off-the-shelf” technologies can be rapidly developed, as well as discuss what sociological hurdles must be overcome to successfully deploy such a system. Early results from production running of Hydra will also be shared as well as a future outlook for development of Hydra

Robust errant beam prognostics with conditional modeling for particle accelerators

Particle accelerators are complex and comprise thousands of components, with many pieces of equipment running at their peak power. Consequently, they can fault and abort operations for numerous reasons, lowering efficiency and science output. To avoid these faults, we apply anomaly detection techniques to predict unusual behavior and perform preemptive actions to improve the total availability. Supervised machine learning (ML) techniques such as siamese neural network models can outperform the often-used unsupervised or semi-supervised approaches for anomaly detection by leveraging the label information. One of the challenges specific to anomaly detection for particle accelerators is the data’s variability due to accelerator configuration changes within a production run of several months. ML models fail at providing accurate predictions when data changes due to changes in the configuration. To address this challenge, we include the configuration settings into our models and training to improve the results. Beam configurations are used as a conditional input for the model to learn any cross-correlation between the data from different conditions and retain its performance. We employ conditional siamese neural network (CSNN) models and conditional variational auto encoder (CVAE) models to predict errant beam pulses at the spallation neutron source under different system configurations and compare their performance. We demonstrate that CSNNs outperform CVAEs in our application