Modern Machine Learning for LHC Physicists

Modern machine learning is transforming particle physics, faster than we can follow, and bullying its way into our numerical tool box. For young researchers it is crucial to stay on top of this development, which means applying cutting-edge methods and tools to the full range of LHC physics problems. These lecture notes are meant to lead students with basic knowledge of particle physics and significant enthusiasm for machine learning to relevant applications as fast as possible. They start with an LHC-specific motivation and a non-standard introduction to neural networks and then cover classification, unsupervised classification, generative networks, and inverse problems. Two themes defining much of the discussion are well-defined loss functions reflecting the problem at hand and uncertainty-aware networks. As part of the applications, the notes include some aspects of theoretical LHC physics. All examples are chosen from particle physics publications of the last few years. Given that these notes will be outdated already at the time of submission, the week of ML4Jets 2022, they will be updated frequently.Comment: First version, we very much appreciate feedbac

Deep SELECTOR-JPEG: ADAPTIVE JPEG IMAGE COMPRESSION FOR COMPUTER VISION IN IMAGE CLASSIFICATION AND HUMAN VISION

Deep Neural Networks (DNNs) demonstrate excellent performance in many Computer Vision (CV) applications such as image classification. To meet storage/bandwidth requirements, the input images to these CV applications are compressed using lossy image compression standards, among which JPEG is the most common. Classical JPEG is designed to consider Human Vision (HV) and pays a little attention to CV, resulting in classification accuracy drop of DNNs, especially at high Compression Ratios (CRs). This work presents Deep Selector-JPEG, an adaptive JPEG compression method that simultaneously targets both image classification and HV. For each image, Deep Selector-JPEG selects a Quality Factor (QF) adaptively to compress the image so that a good trade-off between the Compression Ratio (CR) and DNN classifier Accuracy (Rate-Accuracy performance) can be achieved over a set of images for a variety of DNN classifiers while the PSNR of such compressed image is greater than a threshold value predetermined by HV with a high probability. Towards this end, Deep Selector-JPEG first defines a set of feasible QFs such that an image compressed at any QF within this set has PSNR greater than a predetermined threshold value with a high probability. For some images, multiple QFs within this set are suitable (ON) for compressing for a DNN classifier, which means compressing at these QFs at least maintains the ground truth rank of the original input for the DNN classifier. For a given image, Deep Selector-JPEG first determines the QFs that are ON among the set of feasible QFs. This problem is represented as a Multi-label Classification (MLC) problem since each image has multiple corresponding suitable QFs. We solve MLC using a binary relevance procedure, which involves training an independent binary DNN classifier for each QF within the feasible set to predict the ON/OFF labeling for each input image. Given a target CR, we empirically derive a subset of feasible QFs for this target CR and select the least QF that is ON in this set. Experimental results show that in comparison with the default JPEG, Deep Selector-JPEG indeed achieves better Rate-Accuracy performance over the entire ImageNet validation set for all tested DNN classifiers with gains in classification accuracy up to 1% at the same CRs, while satisfying HV constraints and keeping complexity under control

Neural density estimation and likelihood-free inference

I consider two problems in machine learning and statistics: the problem of estimating the joint probability density of a collection of random variables, known as density estimation, and the problem of inferring model parameters when their likelihood is intractable, known as likelihood-free inference. The contribution of the thesis is a set of new methods for addressing these problems that are based on recent advances in neural networks and deep learning. The first part of the thesis is about density estimation. The joint probability density of a collection of random variables is a useful mathematical description of their statistical properties, but can be hard to estimate from data, especially when the number of random variables is large. Traditional density-estimation methods such as histograms or kernel density estimators are effective for a small number of random variables, but scale badly as the number increases. In contrast, models for density estimation based on neural networks scale better with the number of random variables, and can incorporate domain knowledge in their design. My main contribution is Masked Autoregressive Flow, a new model for density estimation based on a bijective neural network that transforms random noise to data. At the time of its introduction, Masked Autoregressive Flow achieved state-of-the-art results in general-purpose density estimation. Since its publication, Masked Autoregressive Flow has contributed to the broader understanding of neural density estimation, and has influenced subsequent developments in the field. The second part of the thesis is about likelihood-free inference. Typically, a statistical model can be specified either as a likelihood function that describes the statistical relationship between model parameters and data, or as a simulator that can be run forward to generate data. Specifying a statistical model as a simulator can offer greater modelling flexibility and can produce more interpretable models, but can also make inference of model parameters harder, as the likelihood of the parameters may no longer be tractable. Traditional techniques for likelihood-free inference such as approximate Bayesian computation rely on simulating data from the model, but often require a large number of simulations to produce accurate results. In this thesis, I cast the problem of likelihood-free inference as a density-estimation problem, and address it with neural density models. My main contribution is the introduction of two new methods for likelihood-free inference: Sequential Neural Posterior Estimation (Type A), which estimates the posterior, and Sequential Neural Likelihood, which estimates the likelihood. Both methods use a neural density model to estimate the posterior/likelihood, and a sequential training procedure to guide simulations. My experiments show that the proposed methods produce accurate results, and are often orders of magnitude faster than alternative methods based on approximate Bayesian computation

Role of deep learning in infant brain MRI analysis

Deep learning algorithms and in particular convolutional networks have shown tremendous success in medical image analysis applications, though relatively few methods have been applied to infant MRI data due numerous inherent challenges such as inhomogenous tissue appearance across the image, considerable image intensity variability across the first year of life, and a low signal to noise setting. This paper presents methods addressing these challenges in two selected applications, specifically infant brain tissue segmentation at the isointense stage and presymptomatic disease prediction in neurodevelopmental disorders. Corresponding methods are reviewed and compared, and open issues are identified, namely low data size restrictions, class imbalance problems, and lack of interpretation of the resulting deep learning solutions. We discuss how existing solutions can be adapted to approach these issues as well as how generative models seem to be a particularly strong contender to address them