Systematic construction of efficient six-stage fifth-order explicit Runge-Kutta embedded pairs without standard simplifying assumptions

This thesis examines methodologies and software to construct explicit Runge-Kutta (ERK) pairs for solving initial value problems (IVPs) by constructing efficient six-stage fifth-order ERK pairs without standard simplifying assumptions. The problem of whether efficient higher-order ERK pairs can be constructed algebraically without the standard simplifying assumptions dates back to at least the 1960s, with Cassity's complete solution of the six-stage fifth-order order conditions. Although RK methods based on the six-stage fifth-order order conditions have been widely studied and have continuing practical importance, prior to this thesis, the aforementioned complete solution to these order conditions has no published usage beyond the original series of publications by Cassity in the 1960s. The complete solution of six-stage fifth-order ERK order conditions published by Cassity in 1969 is not in a formulation that can easily be used for practical purposes, such as a software implementation. However, it is shown in this thesis that when the order conditions are solved and formulated appropriately using a computer algebra system (CAS), the generated code can be used for practical purposes and the complete solution is readily extended to ERK pairs. The condensed matrix form of the order conditions introduced by Cassity in 1969 is shown to be an ideal methodology, which probably has wider applicability, for solving order conditions using a CAS. The software package OCSage developed for this thesis, in order to solve the order conditions and study the properties of the resulting methods, is built on top of the Sage CAS. However, in order to effectively determine that the constructed ERK pairs without standard simplifying assumptions are in fact efficient by some well-defined criteria, the process of selecting the coefficients of ERK pairs is re-examined in conjunction with a sufficient amount of performance data. The pythODE software package developed for this thesis is used to generate a large amount of performance data from a large selection of candidate ERK pairs found using OCSage. In particular, it is shown that there is unlikely to be a well-defined methodology for selecting optimal pairs for general-purpose use, other than avoiding poor choices of certain properties and ensuring the error coefficients are as small as possible. However, for IVPs from celestial mechanics, there are obvious optimal pairs that have specific values of a small subset of the principal error coefficients (PECs). Statements seen in the literature that the best that can be done is treating all PECs equally do not necessarily apply to at least some broad classes of IVPs. By choosing ERK pairs based on specific values of individual PECs, not only are ERK pairs that are 20-30% more efficient than comparable published pairs found for test sets of IVPs from celestial mechanics, but the variation in performance between the best and worst ERK pairs that otherwise would seem to have similar properties is reduced from a factor of 2 down to as low as 15%. Based on observations of the small number of IVPs of other classes in common IVP test sets, there are other classes of IVPs that have different optimal values of the PECs. A more general contribution of this thesis is that it specifically demonstrates how specialized software tools and a larger amount of performance data than is typical can support novel empirical insights into numerical methods

Diagonally Implicit Runge-Kutta Methods for Ordinary Differential Equations. A Review

A review of diagonally implicit Runge-Kutta (DIRK) methods applied to rst-order ordinary di erential equations (ODEs) is undertaken. The goal of this review is to summarize the characteristics, assess the potential, and then design several nearly optimal, general purpose, DIRK-type methods. Over 20 important aspects of DIRKtype methods are reviewed. A design study is then conducted on DIRK-type methods having from two to seven implicit stages. From this, 15 schemes are selected for general purpose application. Testing of the 15 chosen methods is done on three singular perturbation problems. Based on the review of method characteristics, these methods focus on having a stage order of two, sti accuracy, L-stability, high quality embedded and dense-output methods, small magnitudes of the algebraic stability matrix eigenvalues, small values of aii, and small or vanishing values of the internal stability function for large eigenvalues of the Jacobian. Among the 15 new methods, ESDIRK4(3)6L[2]SA is recommended as a good default method for solving sti problems at moderate error tolerances

Computation of saddle type slow manifolds using iterative methods

This paper presents an alternative approach for the computation of trajectory segments on slow manifolds of saddle type. This approach is based on iterative methods rather than collocation-type methods. Compared to collocation methods, that require mesh refinements to ensure uniform convergence with respect to $\epsilon$, appropriate estimates are directly attainable using the method of this paper. The method is applied to several examples including: A model for a pair of neurons coupled by reciprocal inhibition with two slow and two fast variables and to the computation of homoclinic connections in the FitzHugh-Nagumo system.Comment: To appear in SIAM Journal of Applied Dynamical System

A Symbolic-Numeric Approach to the Solution of the Butcher Equations

Abstract A new approach based on the introduction of new simplifying assumptions of a novel kind is introduced. The approach is based on the construction of a graduated finite-dimensional algebra for a given Butcher tableau. This approach allowed us to discover some new families of Runge-Kutta (RK) methods of orders less than or equal to 8. Most of the methods constructed have new features different from those of previously known methods. A new order 9 method has been found having only 13 stages. For all of these families we have found representatives numerically and introduced a method to find their local dimensions. Using numerical information we additionally derive analytical solutions in some cases

A Unified Approach to Spurious Solutions Introduced by Time Discretisation. Part I: Basic Theory

The asymptotic states of numerical methods for initial value problems are examined. In particular, spurious steady solutions, solutions with period 2 in the timestep, and spurious invariant curves are studied. A numerical method is considered as a dynamical system parameterised by the timestep h. It is shown that the three kinds of spurious solutions can bifurcate from genuine steady solutions of the numerical method (which are inherited from the differential equation) as h is varied. Conditions under which these bifurcations occur are derived for Runge–Kutta schemes, linear multistep methods, and a class of predictor-corrector methods in a PE(CE)^M implementation. The results are used to provide a unifying framework to various scattered results on spurious solutions which already exist in the literature. Furthermore, the implications for choice of numerical scheme are studied. In numerical simulation it is desirable to minimise the effect of spurious solutions. Classes of methods with desirable dynamical properties are described and evaluated

Automated Translation and Accelerated Solving of Differential Equations on Multiple GPU Platforms

We demonstrate a high-performance vendor-agnostic method for massively parallel solving of ensembles of ordinary differential equations (ODEs) and stochastic differential equations (SDEs) on GPUs. The method is integrated with a widely used differential equation solver library in a high-level language (Julia's DifferentialEquations.jl) and enables GPU acceleration without requiring code changes by the user. Our approach achieves state-of-the-art performance compared to hand-optimized CUDA-C++ kernels, while performing $20-100\times$ faster than the vectorized-map (\texttt{vmap}) approach implemented in JAX and PyTorch. Performance evaluation on NVIDIA, AMD, Intel, and Apple GPUs demonstrates performance portability and vendor-agnosticism. We show composability with MPI to enable distributed multi-GPU workflows. The implemented solvers are fully featured, supporting event handling, automatic differentiation, and incorporating of datasets via the GPU's texture memory, allowing scientists to take advantage of GPU acceleration on all major current architectures without changing their model code and without loss of performance.Comment: 11 figure

Confederated Modular Differential Equation APIs for Accelerated Algorithm Development and Benchmarking

Performant numerical solving of differential equations is required for large-scale scientific modeling. In this manuscript we focus on two questions: (1) how can researchers empirically verify theoretical advances and consistently compare methods in production software settings and (2) how can users (scientific domain experts) keep up with the state-of-the-art methods to select those which are most appropriate? Here we describe how the confederated modular API of DifferentialEquations.jl addresses these concerns. We detail the package-free API which allows numerical methods researchers to readily utilize and benchmark any compatible method directly in full-scale scientific applications. In addition, we describe how the complexity of the method choices is abstracted via a polyalgorithm. We show how scientific tooling built on top of DifferentialEquations.jl, such as packages for dynamical systems quantification and quantum optics simulation, both benefit from this structure and provide themselves as convenient benchmarking tools.Comment: 4 figures, 3 algorithm