Learning a Consensus Sub-Network with Polarization Regularization and One Pass Training

The subject of green AI has been gaining attention within the deep learning community given the recent trend of ever larger and more complex neural network models. Existing solutions for reducing the computational load of training at inference time usually involve pruning the network parameters. Pruning schemes often create extra overhead either by iterative training and fine-tuning for static pruning or repeated computation of a dynamic pruning graph. We propose a new parameter pruning strategy for learning a lighter-weight sub-network that minimizes the energy cost while maintaining comparable performance to the fully parameterised network on given downstream tasks. Our proposed pruning scheme is green-oriented, as it only requires a one-off training to discover the optimal static sub-networks by dynamic pruning methods. The pruning scheme consists of a binary gating module and a novel loss function to uncover sub-networks with user-defined sparsity. Our method enables pruning and training simultaneously, which saves energy in both the training and inference phases and avoids extra computational overhead from gating modules at inference time. Our results on CIFAR-10 and CIFAR-100 suggest that our scheme can remove 50% of connections in deep networks with less than 1% reduction in classification accuracy. Compared to other related pruning methods, our method demonstrates a lower drop in accuracy for equivalent reductions in computational cost

Topical: Learning Repository Embeddings from Source Code using Attention

Machine learning on source code (MLOnCode) promises to transform how software is delivered. By mining the context and relationship between software artefacts, MLOnCode augments the software developers capabilities with code auto-generation, code recommendation, code auto-tagging and other data-driven enhancements. For many of these tasks a script level representation of code is sufficient, however, in many cases a repository level representation that takes into account various dependencies and repository structure is imperative, for example, auto-tagging repositories with topics or auto-documentation of repository code etc. Existing methods for computing repository level representations suffer from (a) reliance on natural language documentation of code (for example, README files) (b) naive aggregation of method/script-level representation, for example, by concatenation or averaging. This paper introduces Topical a deep neural network to generate repository level embeddings of publicly available GitHub code repositories directly from source code. Topical incorporates an attention mechanism that projects the source code, the full dependency graph and the script level textual information into a dense repository-level representation. To compute the repository-level representations, Topical is trained to predict the topics associated with a repository, on a dataset of publicly available GitHub repositories that were crawled along with their ground truth topic tags. Our experiments show that the embeddings computed by Topical are able to outperform multiple baselines, including baselines that naively combine the method-level representations through averaging or concatenation at the task of repository auto-tagging.Comment: Pre-print, under revie

On the Utility of Prediction Sets in Human-AI Teams

Research on human-AI teams usually provides experts with a single label, which ignores the uncertainty in a model's recommendation. Conformal prediction (CP) is a well established line of research that focuses on building a theoretically grounded, calibrated prediction set, which may contain multiple labels. We explore how such prediction sets impact expert decision-making in human-AI teams. Our evaluation on human subjects finds that set valued predictions positively impact experts. However, we notice that the predictive sets provided by CP can be very large, which leads to unhelpful AI assistants. To mitigate this, we introduce D-CP, a method to perform CP on some examples and defer to experts. We prove that D-CP can reduce the prediction set size of non-deferred examples. We show how D-CP performs in quantitative and in human subject experiments ($n=120$). Our results suggest that CP prediction sets improve human-AI team performance over showing the top-1 prediction alone, and that experts find D-CP prediction sets are more useful than CP prediction sets.The Alan Turing Institute Leverhulme Trust via CFI DeepMind Mozilla Foundatio

Measurement of the double-differential inclusive jet cross section in proton-proton collisions at s\sqrt{s} = 5.02 TeV

International audienceThe inclusive jet cross section is measured as a function of jet transverse momentum $p_\mathrm{T}$ and rapidity $y$. The measurement is performed using proton-proton collision data at $\sqrt{s}$ = 5.02 TeV, recorded by the CMS experiment at the LHC, corresponding to an integrated luminosity of 27.4 pb$^{-1}$. The jets are reconstructed with the anti-$k_\mathrm{T}$ algorithm using a distance parameter of $R$ = 0.4, within the rapidity interval $\lvert y\rvert$$\lt$ 2, and across the kinematic range 0.06 $\lt$$p_\mathrm{T}$$\lt$ 1 TeV. The jet cross section is unfolded from detector to particle level using the determined jet response and resolution. The results are compared to predictions of perturbative quantum chromodynamics, calculated at both next-to-leading order and next-to-next-to-leading order. The predictions are corrected for nonperturbative effects, and presented for a variety of parton distribution functions and choices of the renormalization/factorization scales and the strong coupling $\alpha_\mathrm{S}$

Measurement of the double-differential inclusive jet cross section in proton-proton collisions at s= \sqrt{s} = 5.02 TeV

The inclusive jet cross section is measured as a function of jet transverse momentum $p_{\mathrm{T}}$ and rapidity $y$. The measurement is performed using proton-proton collision data at $\sqrt{s} =$ 5.02 TeV, recorded by the CMS experiment at the LHC, corresponding to an integrated luminosity of 27.4$\,\text{pb}^{-1}$. The jets are reconstructed with the anti-$k_{\mathrm{T}}$ algorithm using a distance parameter of $R=$ 0.4, within the rapidity interval $|y| <$ 2, and across the kinematic range 0.06 $< p_{\mathrm{T}} <$ 1 TeV. The jet cross section is unfolded from detector to particle level using the determined jet response and resolution. The results are compared to predictions of perturbative quantum chromodynamics, calculated at both next-to-leading order and next-to-next-to-leading order. The predictions are corrected for nonperturbative effects, and presented for a variety of parton distribution functions and choices of the renormalization/factorization scales and the strong coupling $\alpha_\mathrm{S}$.The inclusive jet cross section is measured as a function of jet transverse momentum $p_\mathrm{T}$ and rapidity $y$. The measurement is performed using proton-proton collision data at $\sqrt{s}$ = 5.02 TeV, recorded by the CMS experiment at the LHC, corresponding to an integrated luminosity of 27.4 pb$^{-1}$. The jets are reconstructed with the anti-$k_\mathrm{T}$ algorithm using a distance parameter of $R$ = 0.4, within the rapidity interval $\lvert y\rvert$$\lt$ 2, and across the kinematic range 0.06 $\lt$$p_\mathrm{T}$$\lt$ 1 TeV. The jet cross section is unfolded from detector to particle level using the determined jet response and resolution. The results are compared to predictions of perturbative quantum chromodynamics, calculated at both next-to-leading order and next-to-next-to-leading order. The predictions are corrected for nonperturbative effects, and presented for a variety of parton distribution functions and choices of the renormalization/factorization scales and the strong coupling $\alpha_\mathrm{S}$

Measurement of the double-differential inclusive jet cross section in proton-proton collisions at s\sqrt{s} = 5.02 TeV

International audienceThe inclusive jet cross section is measured as a function of jet transverse momentum $p_\mathrm{T}$ and rapidity $y$. The measurement is performed using proton-proton collision data at $\sqrt{s}$ = 5.02 TeV, recorded by the CMS experiment at the LHC, corresponding to an integrated luminosity of 27.4 pb$^{-1}$. The jets are reconstructed with the anti-$k_\mathrm{T}$ algorithm using a distance parameter of $R$ = 0.4, within the rapidity interval $\lvert y\rvert$$\lt$ 2, and across the kinematic range 0.06 $\lt$$p_\mathrm{T}$$\lt$ 1 TeV. The jet cross section is unfolded from detector to particle level using the determined jet response and resolution. The results are compared to predictions of perturbative quantum chromodynamics, calculated at both next-to-leading order and next-to-next-to-leading order. The predictions are corrected for nonperturbative effects, and presented for a variety of parton distribution functions and choices of the renormalization/factorization scales and the strong coupling $\alpha_\mathrm{S}$

Two-particle Bose-Einstein correlations and their Lévy parameters in PbPb collisions at sNN\sqrt{s_\mathrm{NN}} = 5.02 TeV

Two-particle Bose-Einstein momentum correlation functions are studied for charged-hadron pairs in lead-lead collisions at a center-of-mass energy per nucleon pair of $\sqrt{s_\mathrm{NN}}$ = 5.02 TeV. The data sample, containing 4.27$\times10^{9}$ minimum bias events corresponding to an integrated luminosity of 0.607 nb$^{-1}$, was collected by the CMS experiment in 2018. The experimental results are discussed in terms of a L\'evy-type source distribution. The parameters of this distribution are extracted as functions of particle pair average transverse mass and collision centrality. These parameters include the L\'evy index or shape parameter ($\alpha$), the L\'evy scale parameter ($R$), and the correlation strength parameter ($\lambda$). The source shape, characterized by $\alpha$, is found to be neither Cauchy nor Gaussian, implying the need for a full L\'evy analysis. Similarly to what was previously found for systems characterized by Gaussian source radii, a hydrodynamical scaling is observed for the L\'evy $R$ parameter. The $\lambda$ parameter is studied in terms of the core-halo model.Comment: Submitted to Physical Review C. All figures and tables can be found at http://cms-results.web.cern.ch/cms-results/public-results/publications/HIN-21-011 (CMS Public Pages

Measurement of the double-differential inclusive jet cross section in proton-proton collisions at s\sqrt{s} = 5.02 TeV

International audienceThe inclusive jet cross section is measured as a function of jet transverse momentum $p_\mathrm{T}$ and rapidity $y$. The measurement is performed using proton-proton collision data at $\sqrt{s}$ = 5.02 TeV, recorded by the CMS experiment at the LHC, corresponding to an integrated luminosity of 27.4 pb$^{-1}$. The jets are reconstructed with the anti-$k_\mathrm{T}$ algorithm using a distance parameter of $R$ = 0.4, within the rapidity interval $\lvert y\rvert$$\lt$ 2, and across the kinematic range 0.06 $\lt$$p_\mathrm{T}$$\lt$ 1 TeV. The jet cross section is unfolded from detector to particle level using the determined jet response and resolution. The results are compared to predictions of perturbative quantum chromodynamics, calculated at both next-to-leading order and next-to-next-to-leading order. The predictions are corrected for nonperturbative effects, and presented for a variety of parton distribution functions and choices of the renormalization/factorization scales and the strong coupling $\alpha_\mathrm{S}$

Measurement of the double-differential inclusive jet cross section in proton-proton collisions at s\sqrt{s} = 5.02 TeV

International audienceThe inclusive jet cross section is measured as a function of jet transverse momentum $p_\mathrm{T}$ and rapidity $y$. The measurement is performed using proton-proton collision data at $\sqrt{s}$ = 5.02 TeV, recorded by the CMS experiment at the LHC, corresponding to an integrated luminosity of 27.4 pb$^{-1}$. The jets are reconstructed with the anti-$k_\mathrm{T}$ algorithm using a distance parameter of $R$ = 0.4, within the rapidity interval $\lvert y\rvert$$\lt$ 2, and across the kinematic range 0.06 $\lt$$p_\mathrm{T}$$\lt$ 1 TeV. The jet cross section is unfolded from detector to particle level using the determined jet response and resolution. The results are compared to predictions of perturbative quantum chromodynamics, calculated at both next-to-leading order and next-to-next-to-leading order. The predictions are corrected for nonperturbative effects, and presented for a variety of parton distribution functions and choices of the renormalization/factorization scales and the strong coupling $\alpha_\mathrm{S}$