Probabilistic ODE Solvers with Runge-Kutta Means

Runge-Kutta methods are the classic family of solvers for ordinary differential equations (ODEs), and the basis for the state of the art. Like most numerical methods, they return point estimates. We construct a family of probabilistic numerical methods that instead return a Gauss-Markov process defining a probability distribution over the ODE solution. In contrast to prior work, we construct this family such that posterior means match the outputs of the Runge-Kutta family exactly, thus inheriting their proven good properties. Remaining degrees of freedom not identified by the match to Runge-Kutta are chosen such that the posterior probability measure fits the observed structure of the ODE. Our results shed light on the structure of Runge-Kutta solvers from a new direction, provide a richer, probabilistic output, have low computational cost, and raise new research questions.Comment: 18 pages (9 page conference paper, plus supplements); appears in Advances in Neural Information Processing Systems (NIPS), 201

On the Lie enveloping algebra of a post-Lie algebra

We consider pairs of Lie algebras $g$ and $\bar{g}$, defined over a common vector space, where the Lie brackets of $g$ and $\bar{g}$ are related via a post-Lie algebra structure. The latter can be extended to the Lie enveloping algebra $U(g)$. This permits us to define another associative product on $U(g)$, which gives rise to a Hopf algebra isomorphism between $U(\bar{g})$ and a new Hopf algebra assembled from $U(g)$ with the new product. For the free post-Lie algebra these constructions provide a refined understanding of a fundamental Hopf algebra appearing in the theory of numerical integration methods for differential equations on manifolds. In the pre-Lie setting, the algebraic point of view developed here also provides a concise way to develop Butcher's order theory for Runge--Kutta methods.Comment: 25 page

Trees, bialgebras and intrinsic numerical algorithms

Preliminary work about intrinsic numerical integrators evolving on groups is described. Fix a finite dimensional Lie group G; let g denote its Lie algebra, and let Y(sub 1),...,Y(sub N) denote a basis of g. A class of numerical algorithms is presented that approximate solutions to differential equations evolving on G of the form: dot-x(t) = F(x(t)), x(0) = p is an element of G. The algorithms depend upon constants c(sub i) and c(sub ij), for i = 1,...,k and j is less than i. The algorithms have the property that if the algorithm starts on the group, then it remains on the group. In addition, they also have the property that if G is the abelian group R(N), then the algorithm becomes the classical Runge-Kutta algorithm. The Cayley algebra generated by labeled, ordered trees is used to generate the equations that the coefficients c(sub i) and c(sub ij) must satisfy in order for the algorithm to yield an rth order numerical integrator and to analyze the resulting algorithms

Evolutionary Design of Numerical Methods: Generating Finite Difference and Integration Schemes by Differential Evolution

Classical and new numerical schemes are generated using evolutionary computing. Differential Evolution is used to find the coefficients of finite difference approximations of function derivatives, and of single and multi-step integration methods. The coefficients are reverse engineered based on samples from a target function and its derivative used for training. The Runge-Kutta schemes are trained using the order condition equations. An appealing feature of the evolutionary method is the low number of model parameters. The population size, termination criterion and number of training points are determined in a sensitivity analysis. Computational results show good agreement between evolved and analytical coefficients. In particular, a new fifth-order Runge-Kutta scheme is computed which adheres to the order conditions with a sum of absolute errors of order 10^-14. Execution of the evolved schemes proved the intended orders of accuracy. The outcome of this study is valuable for future developments in the design of complex numerical methods that are out of reach by conventional means.Comment: 19 pages, 7 figures, 10 tables, 4 appendice