Free-form Flows: Make Any Architecture a Normalizing Flow

Normalizing Flows are generative models that directly maximize the likelihood. Previously, the design of normalizing flows was largely constrained by the need for analytical invertibility. We overcome this constraint by a training procedure that uses an efficient estimator for the gradient of the change of variables formula. This enables any dimension-preserving neural network to serve as a generative model through maximum likelihood training. Our approach allows placing the emphasis on tailoring inductive biases precisely to the task at hand. Specifically, we achieve excellent results in molecule generation benchmarks utilizing $E(n)$-equivariant networks. Moreover, our method is competitive in an inverse problem benchmark, while employing off-the-shelf ResNet architectures

A Normalized Autoencoder for LHC Triggers

Autoencoders are an effective analysis tool for the LHC, as they represent one of its main goal of finding physics beyond the Standard Model. The key challenge is that out-of-distribution anomaly searches based on the compressibility of features do not apply to the LHC, while existing density-based searches lack performance. We present the first autoencoder which identifies anomalous jets symmetrically in the directions of higher and lower complexity. The normalized autoencoder combines a standard bottleneck architecture with a well-defined probabilistic description. It works better than all available autoencoders for top vs QCD jets and reliably identifies different dark-jet signals.Comment: 26 pages, 11 figures; update based on referees repor

Maximum Likelihood Training of Autoencoders

Maximum likelihood training has favorable statistical properties and is popular for generative modeling, especially with normalizing flows. On the other hand, generative autoencoders promise to be more efficient than normalizing flows due to the manifold hypothesis. In this work, we introduce successful maximum likelihood training of unconstrained autoencoders for the first time, bringing the two paradigms together. To do so, we identify and overcome two challenges: Firstly, existing maximum likelihood estimators for free-form networks are unacceptably slow, relying on iteration schemes whose cost scales linearly with latent dimension. We introduce an improved estimator which eliminates iteration, resulting in constant cost (roughly double the runtime per batch of a vanilla autoencoder). Secondly, we demonstrate that naively applying maximum likelihood to autoencoders can lead to divergent solutions and use this insight to motivate a stable maximum likelihood training objective. We perform extensive experiments on toy, tabular and image data, demonstrating the competitive performance of the resulting model. We call our model the maximum likelihood autoencoder (MLAE)

Jet Diffusion versus JetGPT -- Modern Networks for the LHC

We introduce two diffusion models and an autoregressive transformer for LHC physics simulations. Bayesian versions allow us to control the networks and capture training uncertainties. After illustrating their different density estimation methods for simple toy models, we discuss their advantages for Z plus jets event generation. While diffusion networks excel through their precision, the transformer scales best with the phase space dimensionality. Given the different training and evaluation speed, we expect LHC physics to benefit from dedicated use cases for normalizing flows, diffusion models, and autoregressive transformers.Comment: 37 pages, 17 figure

An integrated methodology for stress-based fatigue assessment of steel railway bridges

Ageing infrastructure is a widespread problem, with potentially catastrophic consequences. Reliable structural integrity and remaining life assessment are essential for the resolution of the problem. Ageing railway bridges is a particularly difficult subset of the ageing infrastructure problem. There is a complex array of issues that face integrity and remaining life assessment of railway bridges. The structural conditions of railway bridges may change from site to site, even where bridges are supposedly of the same design. Structural conditions may also change over the life of any particular bridge. Dynamic interaction, between railway vehicles and railway bridge structures, has a dramatic effect on structural response and therefore remaining life. This dynamic interaction often requires complex modelling techniques in integrity assessment. For assessment of railway bridges, different assumptions and different methods of assessment are often used, making repeatable and verifiable assessment difficult. In this research, in order to work toward addressing the problem of ageing steel railway bridges, a clear, repeatable methodology for high-level structural assessment has been formulated. The method integrates state-of-the-art modelling, testing and fatigue code tools, and uses dynamic digital data for testing, modelling and assessment. The method is demonstrated by assessment of the steel girder approach spans of the Mullet Creek Railway Bridge, Dapto, NSW, Australia. The method developed in this research begins with a finite element sensitivity model, that is a finite element model that permits variations in joint fixity and support conditions. This model is tuned and validated, using transient digital data from the dynamic field-test response of a slow moving vehicle of known load, through a dynamic analytical model intermediate step. The dynamic field-test response of normal traffic is then recorded and axle loads are identified from the digitised measured response, using the dynamic analytical model as a transfer function. Identified loads are applied to the tuned finite element sensitivity model and the dynamic stress history generated for virtually any bridge component. Dynamic stress histories are then entered directly into a software system, which estimates remaining life via several international fatigue codes. After verification of the finite element sensitivity model against dynamic field-test results, loading and structural conditions may be adjusted and the integrity and remaining life of the structure evaluated for virtually any combination of structural and loading condition. In the demonstration of the method, three components of the Mullet Creek Railway Bridge approach spans have been chosen for structural integrity and remaining life assessment. Structural conditions and applied loading conditions have been altered and the impact of the changes investigated. From the structural integrity and remaining life assessment, two locations have been identified as fatigue critical and recommendations have been made for structural changes and ongoing inspection. The most significant contribution of this research is expected to be the complete methodology, its clarity and repeatability, its integration throughout and the way in which it deals with the difficult problem of true dynamic response. Other contributions have been made within individual steps of the methodology, where attempts have been made to extend current research. These contributions include: the development of a technique for modelling the dynamic response of structures in finite element software; a dynamic analytical model for beams with rotationally stiffened supports subjected to moving distributed loads; extension of load identification theory to distributed loads on non-simply supported beams; and a comparative study of several key international fatigue codes

A normalized autoencoder for LHC triggers

Autoencoders are an effective analysis tool for the LHC, as they represent one of its main goal of finding physics beyond the Standard Model. The key challenge is that out-of-distribution anomaly searches based on the compressibility of features do not apply to the LHC, while existing density-based searches lack performance. We present the first autoencoder which identifies anomalous jets symmetrically in the directions of higher and lower complexity. The normalized autoencoder combines a standard bottleneck architecture with a well-defined probabilistic description. It works better than all available autoencoders for top vs QCD jets and reliably identifies different dark-jet signals

Jet Diffusion versus JetGPT -- Modern Networks for the LHC

International audienceWe introduce two diffusion models and an autoregressive transformer for LHC physics simulations. Bayesian versions allow us to control the networks and capture training uncertainties. After illustrating their different density estimation methods for simple toy models, we discuss their advantages for Z plus jets event generation. While diffusion networks excel through their precision, the transformer scales best with the phase space dimensionality. Given the different training and evaluation speed, we expect LHC physics to benefit from dedicated use cases for normalizing flows, diffusion models, and autoregressive transformers