Model Fusion via Optimal Transport

Combining different models is a widely used paradigm in machine learning applications. While the most common approach is to form an ensemble of models and average their individual predictions, this approach is often rendered infeasible by given resource constraints in terms of memory and computation, which grow linearly with the number of models. We present a layer-wise model fusion algorithm for neural networks that utilizes optimal transport to (soft-) align neurons across the models before averaging their associated parameters. We show that this can successfully yield "one-shot" knowledge transfer (i.e, without requiring any retraining) between neural networks trained on heterogeneous non-i.i.d. data. In both i.i.d. and non-i.i.d. settings , we illustrate that our approach significantly outperforms vanilla averaging, as well as how it can serve as an efficient replacement for the ensemble with moderate fine-tuning, for standard convolutional networks (like VGG11), residual networks (like ResNet18), and multi-layer perceptrons on CIFAR10, CIFAR100, and MNIST. Finally, our approach also provides a principled way to combine the parameters of neural networks with different widths, and we explore its application for model compression. The code is available at the following link, https://github.com/sidak/otfusion.Comment: NeurIPS 2020 conference proceedings (early version featured in the Optimal Transport & Machine Learning workshop, NeurIPS 2019

Optimisation of confinement in a fusion reactor using a nonlinear turbulence model

The confinement of heat in the core of a magnetic fusion reactor is optimised using a multidimensional optimisation algorithm. For the first time in such a study, the loss of heat due to turbulence is modelled at every stage using first-principles nonlinear simulations which accurately capture the turbulent cascade and large-scale zonal flows. The simulations utilise a novel approach, with gyrofluid treatment of the small-scale drift waves and gyrokinetic treatment of the large-scale zonal flows. A simple near-circular equilibrium with standard parameters is chosen as the initial condition. The figure of merit, fusion power per unit volume, is calculated, and then two control parameters, the elongation and triangularity of the outer flux surface, are varied, with the algorithm seeking to optimise the chosen figure of merit. A two-fold increase in the plasma power per unit volume is achieved by moving to higher elongation and strongly negative triangularity.Comment: 32 pages, 8 figures, accepted to JP

Fast ignition of fusion targets by laser-driven electrons

We present hybrid PIC simulations of fast electron transport and energy deposition in pre-compressed fusion targets, taking full account of collective magnetic effects and the hydrodynamic response of the background plasma. Results on actual ignition of an imploded fast ignition configuration are shown accounting for the increased beam divergence found in recent experiments [J.S. Green et al., Phys. Rev. Lett. 100, 015003 (2008)] and the reduction of the electron kinetic energy due to profile steepening predicted by advanced PIC simulations [B. Chrisman et al. Phys. Plasmas 15, 056309 (2008)]. Target ignition is studied as a function of injected electron energy, distance of cone-tip to dense core, initial divergence and kinetic energy of the relativistic electron beam. We found that beam collimation reduces substantially the ignition energies of the cone-guided fuel configuration assumed here.Comment: 15 pages, 9 figures. accepted for publication in Plasma Physics and Controlled Fusio

Fast-ignition design transport studies: realistic electron source, integrated PIC-hydrodynamics, imposed magnetic fields

Transport modeling of idealized, cone-guided fast ignition targets indicates the severe challenge posed by fast-electron source divergence. The hybrid particle-in-cell [PIC] code Zuma is run in tandem with the radiation-hydrodynamics code Hydra to model fast-electron propagation, fuel heating, and thermonuclear burn. The fast electron source is based on a 3D explicit-PIC laser-plasma simulation with the PSC code. This shows a quasi two-temperature energy spectrum, and a divergent angle spectrum (average velocity-space polar angle of 52 degrees). Transport simulations with the PIC-based divergence do not ignite for > 1 MJ of fast-electron energy, for a modest 70 micron standoff distance from fast-electron injection to the dense fuel. However, artificially collimating the source gives an ignition energy of 132 kJ. To mitigate the divergence, we consider imposed axial magnetic fields. Uniform fields ~50 MG are sufficient to recover the artificially collimated ignition energy. Experiments at the Omega laser facility have generated fields of this magnitude by imploding a capsule in seed fields of 50-100 kG. Such imploded fields are however more compressed in the transport region than in the laser absorption region. When fast electrons encounter increasing field strength, magnetic mirroring can reflect a substantial fraction of them and reduce coupling to the fuel. A hollow magnetic pipe, which peaks at a finite radius, is presented as one field configuration which circumvents mirroring.Comment: 16 pages, 17 figures, submitted to Phys. Plasma