Least-Squares and Other Residual Based Techniques for Radiation Transport Calculations

In this dissertation, we develop several novel methods based on or related to least-squares transport residual for solving deterministic radiation transport problems. For the first part of this dissertation a nonlinear spherical harmonics (PN) closure (TPN) was developed based on analysis of the least-squares residual for time-dependent PN equations in 1D slab geometry. The TPN closure suppresses the oscillations induced by Gibbs phenomenon in time-dependent transport calculations effectively. Simultaneously, a nonlinear viscosity term based on the spatial and temporal variations is realized and used in the extension to filtered PN method (NFPN). NFPN determines the angular viscosity on the fly and potentially fixed the issue existed in linear FPN that filtering strength needs to be predefined by iteratively solving the problem. We further developed another type of NFPN and demonstrate both of the two NFPN preserve the thick diffusion limit for thermal radiative transfer problems theoretically and numerically. We also developed several novel methods along with error analyses for steady-state neutron transport calculations based on least-squares methods. Firstly, a relaxed L1 finite element method was developed based on nonlinearly weighting the least-squares formulation by the pointwise transport residual. In problems such as void and near-void situations where least-squares accuracy is poor, the L1 method improves the solution. Further, a non-converged RL1 still can present comparable accuracy. We then developed a least-squares method based on a novel contiguous-discontinuous functional. A proof is provided for the conservation preservation for such a method, which is significant for problems such as k-eigenvalue calculations. Also, a second order accuracy is observed with much lower error magnitudes in several quantities of interest for heterogeneous problems compared with self-adjoint angular flux (SAAF) solution. Lastly, we extended the CD methodology with 1/σt-weighted least-squares functional to derive a CD-SAAF method and developed a SN-PN angular hybrid scheme. The hybrid scheme can employ high order SN in regions with strong transport feature to couple with low order PN in regions with diffusive flux. In k-eigenvalue calculations, it shows superb accuracy with low degrees of freedom

Synergies between Numerical Methods for Kinetic Equations and Neural Networks

The overarching theme of this work is the efficient computation of large-scale systems. Here we deal with two types of mathematical challenges, which are quite different at first glance but offer similar opportunities and challenges upon closer examination. Physical descriptions of phenomena and their mathematical modeling are performed on diverse scales, ranging from nano-scale interactions of single atoms to the macroscopic dynamics of the earth\u27s atmosphere. We consider such systems of interacting particles and explore methods to simulate them efficiently and accurately, with a focus on the kinetic and macroscopic description of interacting particle systems. Macroscopic governing equations describe the time evolution of a system in time and space, whereas the more fine-grained kinetic description additionally takes the particle velocity into account. The study of discretizing kinetic equations that depend on space, time, and velocity variables is a challenge due to the need to preserve physical solution bounds, e.g. positivity, avoiding spurious artifacts and computational efficiency. In the pursuit of overcoming the challenge of computability in both kinetic and multi-scale modeling, a wide variety of approximative methods have been established in the realm of reduced order and surrogate modeling, and model compression. For kinetic models, this may manifest in hybrid numerical solvers, that switch between macroscopic and mesoscopic simulation, asymptotic preserving schemes, that bridge the gap between both physical resolution levels, or surrogate models that operate on a kinetic level but replace computationally heavy operations of the simulation by fast approximations. Thus, for the simulation of kinetic and multi-scale systems with a high spatial resolution and long temporal horizon, the quote by Paul Dirac is as relevant as it was almost a century ago. The first goal of the dissertation is therefore the development of acceleration strategies for kinetic discretization methods, that preserve the structure of their governing equations. Particularly, we investigate the use of convex neural networks, to accelerate the minimal entropy closure method. Further, we develop a neural network-based hybrid solver for multi-scale systems, where kinetic and macroscopic methods are chosen based on local flow conditions. Furthermore, we deal with the compression and efficient computation of neural networks. In the meantime, neural networks are successfully used in different forms in countless scientific works and technical systems, with well-known applications in image recognition, and computer-aided language translation, but also as surrogate models for numerical mathematics. Although the first neural networks were already presented in the 1950s, the scientific discipline has enjoyed increasing popularity mainly during the last 15 years, since only now sufficient computing capacity is available. Remarkably, the increasing availability of computing resources is accompanied by a hunger for larger models, fueled by the common conception of machine learning practitioners and researchers that more trainable parameters equal higher performance and better generalization capabilities. The increase in model size exceeds the growth of available computing resources by orders of magnitude. Since $2012$, the computational resources used in the largest neural network models doubled every $3.4$ months\footnote{\url{https://openai.com/blog/ai-and-compute/}}, opposed to Moore\u27s Law that proposes a $2$-year doubling period in available computing power. To some extent, Dirac\u27s statement also applies to the recent computational challenges in the machine-learning community. The desire to evaluate and train on resource-limited devices sparked interest in model compression, where neural networks are sparsified or factorized, typically after training. The second goal of this dissertation is thus a low-rank method, originating from numerical methods for kinetic equations, to compress neural networks already during training by low-rank factorization. This dissertation thus considers synergies between kinetic models, neural networks, and numerical methods in both disciplines to develop time-, memory- and energy-efficient computational methods for both research areas

Improved Fully-Implicit Spherical Harmonics Methods for First and Second Order Forms of the Transport Equation Using Galerkin Finite Element

In this dissertation, we focus on solving the linear Boltzmann equation -- or transport equation -- using spherical harmonics (PN) expansions with fully-implicit time-integration schemes and Galerkin Finite Element spatial discretizations within the Multiphysics Object Oriented Simulation Environment (MOOSE) framework. The presentation is composed of two main ensembles. On one hand, we study the first-order form of the transport equation in the context of Thermal Radiation Transport (TRT). This nonlinear application physically necessitates to maintain a positive material temperature while the PN approximation tends to create oscillations and negativity in the solution. To mitigate these flaws, we provide a fully-implicit implementation of the Filtered PN (FPN) method and investigate local filtering strategies. After analyzing its effect on the conditioning of the system and showing that it improves the convergence properties of the iterative solver, we numerically investigate the error estimates derived in the linear setting and observe that they hold in the non-linear case. Then, we illustrate the benefits of the method on a standard test problem and compare it with implicit Monte Carlo (IMC) simulations. On the other hand, we focus on second-order forms of the transport equation for neutronics applications. We mostly consider the Self-Adjoint Angular Flux (SAAF) and Least-Squares (LS) formulations, the former being globally conservative but void incompatible and the latter having -- in all generality -- the opposite properties. We study the relationship between these two methods based on the weakly-imposed LS boundary conditions. Equivalences between various parity-based PN methods are also established, in particular showing that second-order filters are not an appropriate fix to retrieve void compatibility. The importance of global conservation is highlighted on a heterogeneous multigroup k-eigenvalue test problem. Based on these considerations, we propose a new method that is both globally conservative and compatible with voids. The main idea is to solve the LS form in the void regions and the SAAF form elsewhere. For the LS form to be conservative in void, a non-symmetric fix is required, yielding the Conservative LS (CLS) formulation. From there, a hybrid SAAF-- CLS method can be derived, having the desired properties. We also show how to extend it to near-void regions and time-dependent problems. While such a second-order form already existed for discrete-ordinates (SN) discretizations (Wang et al. 2014), we believe that this method is the first of its kind, being well-suited to both SN and PN discretizations