Policy gradient methods for free-electron laser and terahertz source optimization and stabilization at the FERMI free-electron laser at Elettra

In this article we report on the application of a model-free reinforcement learning method to the optimization of accelerator systems. We simplify a policy gradient algorithm to accelerator control from sophisticated algorithms that have recently been demonstrated to solve complex dynamic problems. After outlining a theoretical basis for the functioning of the algorithm, we explore the small hyperparameter space to develop intuition about said parameters using a simple number-guess environment. Finally, we demonstrate the algorithm optimizing both a free-electron laser and an accelerator-based terahertz source in-situ. The algorithm is applied to different accelerator control systems and optimizes the desired signals in a few hundred steps without any domain knowledge using up to five control parameters. In addition, the algorithm shows modest tolerance to accelerator fault conditions without any special preparation for such conditions

Free-electron laser spectrum evaluation and automatic optimization

The radiation generated by a seeded free-electron laser (FEL) is characterized by a high temporal coherence, which is close to the Fourier limit in the ideal case. The setup and optimization of a FEL is a non-trivial and challenging operation. This is due to the plethora of highly sensitive machine parameters and to the complex correlations between them. The fine tuning of the FEL process is normally supervised by physicists and is carried out by scanning various parameters with the aim of optimizing the spectrum of the emitted pulses in terms of intensity and line-width. In this article we introduce a novel quantitative method for the evaluation of the FEL spectrum via a quality index. Moreover, we investigate the possibility of optimization of the FEL parameters using this index as the objective function of an automatic procedure. We also present the results of the preliminary tests performed in the FERMI FEL focused on the effectiveness and ability of the automatic procedure to assist in the task of machine tuning and optimization

Test of machine learning at the CERN LINAC4

The CERN H− linear accelerator, LINAC4, served as a test bed for advanced algorithms during the CERN Long Shutdown 2 in the years 2019/20. One of the main goals was to show that reinforcement learning with all its benefits can be used as a replacement for numerical optimization and as a complement to classical control in the accelerator control context. Many of the algorithms used were prepared before- hand at the electron line of the AWAKE facility to make the best use of the limited time available at LINAC4. An overview of the algorithms and concepts tested at LINAC4 and AWAKE will be given and the results discussed.peer-reviewe

Feasibility Investigation on Several Reinforcement Learning Techniques to Improve the Performance of the FERMI Free-Electron Laser

The research carried out in particle accelerator facilities does not concern only particle and condensed matter physics, although these are the main topics covered in the field. Indeed, since a particle accelerator is composed of many different sub-systems, its proper functioning depends both on each of these parts and their interconnection. It follows that the study, implementation, and improvement of the various sub-systems are fundamental points of investigation too. In particular, an interesting aspect for the automation engineering community is the control of such systems that usually are complex, large, noise-affected, and non-linear. The doctoral project fits into this scope, investigating the introduction of new methods to automatically improve the performance of a specific type of particle accelerators: seeded free-electron lasers. The optimization of such systems is a challenging task, already faced in years by many different approaches in order to find and attain an optimal working point, keeping it optimally tuned despite drift or disturbances. Despite the good results achieved, better ones are always sought for. For this reason, several methods belonging to reinforcement learning, an area of machine learning that is attracting more and more attention in the scientific field, have been applied on FERMI, the free-electron laser facility at Elettra Sincrotrone Trieste. The research activity has been carried out by applying both model-free and model-based techniques belonging to reinforcement learning. Satisfactory preliminary results have been obtained, that present the first step toward a new fully automatic procedure for the alignment of the seed laser to the electron beam. In the meantime, at the Conseil Europ\ue9en pour la Recherche Nucl\ue9aire, CERN, a similar investigation was ongoing. In the last year of the doctoral course, a collaboration to share the knowledge on the topic took place. Some of the results collected on the largest particle physics laboratory in the world are presented in the doctoral dissertation

An Online Iterative Linear Quadratic Approach for a Satisfactory Working Point Attainment at FERMI

The attainment of a satisfactory operating point is one of the main problems in the tuning of particle accelerators. These are extremely complex facilities, characterized by the absence of a model that accurately describes their dynamics, and by an often persistent noise which, along with machine drifts, affects their behaviour in unpredictable ways. In this paper, we propose an online iterative Linear Quadratic Regulator (iLQR) approach to tackle this problem on the FERMI free-electron laser of Elettra Sincrotrone Trieste. It consists of a model identification performed by a neural network trained on data collected from the real facility, followed by the application of the iLQR in a Model-Predictive Control fashion. We perform several experiments, training the neural network with increasing amount of data, in order to understand what level of model accuracy is needed to accomplish the task. We empirically show that the online iLQR results, on average, in fewer steps than a simple gradient ascent (GA), and requires a less accurate neural network to achieve the goal

Towards the Automatic Setup of Longitudinal Emittance Blow-Up in the CERN SPS

Controlled longitudinal emittance blow-up in the CERN SPS is necessary to stabilize high-intensity beams for the High-Luminosity LHC (HL-LHC) by increasing the synchrotron frequency spread. The process consists of injecting bandwidth-limited noise into the main RF phase loop to diffuse particles in the core of the bunch. The setting up of the noise parameters, such as frequency band and amplitude, is a non-trivial and time-consuming procedure that has been performed manually so far. In this preliminary study, several optimization methods are investigated to set up the noise parameters automatically. We apply the CERN Common Optimization Interfaces as a generic framework for the optimization algorithm. Single-bunch profiles generated with the BLonD simulation code have been used to investigate the optimization algorithms offline. Furthermore, analysis has been carried out on measured bunch profiles in the SPS to define the problem constraints and properly formulate the objective function

Test of Machine Learning at the CERN LINAC4

The CERN H$^-$ linear accelerator, LINAC4, served as a test bed for advanced algorithms during the CERN Long Shutdown 2 in the years 2019/20. One of the main goals was to show that reinforcement learning with all its benefits can be used as a replacement for numerical optimization and as a complement to classical control in the accelerator control context. Many of the algorithms used were prepared beforehand at the electron line of the AWAKE facility to make the best use of the limited time available at LINAC4. An overview of the algorithms and concepts tested at LINAC4 and AWAKE will be given and the results discussed

Sample-efficient reinforcement learning for CERN accelerator control

Numerical optimization algorithms are already established tools to increase and stabilize the performance of particle accelerators. These algorithms have many advantages, are available out of the box, and can be adapted to a wide range of optimization problems in accelerator operation. The next boost in efficiency is expected to come from reinforcement learning algorithms that learn the optimal policy for a certain control problem and hence, once trained, can do without the time-consuming exploration phase needed for numerical optimizers. To investigate this approach, continuous model-free reinforcement learning with up to 16 degrees of freedom was developed and successfully tested at various facilities at CERN. The approach and algorithms used are discussed and the results obtained for trajectory steering at the AWAKE electron line and LINAC4 are presented. The necessary next steps, such as uncertainty aware model-based approaches, and the potential for future applications at particle accelerators are addressed

Coherent soft X-ray pulses from an echo-enabled harmonic generation free-electron laser

International audienceX-ray free-electron lasers (FELs), which amplify light emitted by a relativistic electron beam, are extending nonlinear optical techniques to shorter wavelengths, adding element specificity by exciting and probing electronic transitions from core levels. These techniques would benefit tremendously from having a stable FEL source, generating spectrally pure and wavelength-tunable pulses. We show that such requirements can be met by operating the FEL in the so-called echo-enabled harmonic generation (EEHG) configuration. Here, two external conventional lasers are used to precisely tailor the longitudinal phase space of the electron beam before emission of X-rays. We demonstrate high-gain EEHG lasing producing stable, intense, nearly fully coherent pulses at wavelengths as short as 5.9 nm (~211 eV) at the FERMI FEL user facility. Low sensitivity to electron-beam imperfections and observation of stable, narrow-band, coherent emission down to 2.6 nm (~474 eV) make the technique a prime candidate for generating laser-like pulses in the X-ray spectral region, opening the door to multidimensional coherent spectroscopies at short wavelengths