774 research outputs found
Probabilistic ODE Solvers with Runge-Kutta Means
Runge-Kutta methods are the classic family of solvers for ordinary
differential equations (ODEs), and the basis for the state of the art. Like
most numerical methods, they return point estimates. We construct a family of
probabilistic numerical methods that instead return a Gauss-Markov process
defining a probability distribution over the ODE solution. In contrast to prior
work, we construct this family such that posterior means match the outputs of
the Runge-Kutta family exactly, thus inheriting their proven good properties.
Remaining degrees of freedom not identified by the match to Runge-Kutta are
chosen such that the posterior probability measure fits the observed structure
of the ODE. Our results shed light on the structure of Runge-Kutta solvers from
a new direction, provide a richer, probabilistic output, have low computational
cost, and raise new research questions.Comment: 18 pages (9 page conference paper, plus supplements); appears in
Advances in Neural Information Processing Systems (NIPS), 201
Probabilistic Solutions to Differential Equations and their Application to Riemannian Statistics
We study a probabilistic numerical method for the solution of both boundary
and initial value problems that returns a joint Gaussian process posterior over
the solution. Such methods have concrete value in the statistics on Riemannian
manifolds, where non-analytic ordinary differential equations are involved in
virtually all computations. The probabilistic formulation permits marginalising
the uncertainty of the numerical solution such that statistics are less
sensitive to inaccuracies. This leads to new Riemannian algorithms for mean
value computations and principal geodesic analysis. Marginalisation also means
results can be less precise than point estimates, enabling a noticeable
speed-up over the state of the art. Our approach is an argument for a wider
point that uncertainty caused by numerical calculations should be tracked
throughout the pipeline of machine learning algorithms.Comment: 11 page (9 page conference paper, plus supplements
A quantitative probabilistic investigation into the accumulation of rounding errors in numerical ODE solution.
We examine numerical rounding errors of some deterministic solvers for systems of ordinary differential equations (ODEs) from a probabilistic viewpoint. We show that the accumulation of rounding errors results in a solution which is inherently random and we obtain the theoretical distribution of the trajectory as a function of time, the step size and the numerical precision of the computer. We consider, in particular, systems which amplify the effect of the rounding errors so that over long time periods the solutions exhibit divergent behaviour. By performing multiple repetitions with different values of the time step size, we observe numerically the random distributions predicted theoretically. We mainly focus on the explicit Euler and fourth order RungeāKutta methods but also briefly consider more complex algorithms such as the implicit solvers VODE and RADAU5 in order to demonstrate that the observed effects are not specific to a particular method
Probabilistic Numerics and Uncertainty in Computations
We deliver a call to arms for probabilistic numerical methods: algorithms for
numerical tasks, including linear algebra, integration, optimization and
solving differential equations, that return uncertainties in their
calculations. Such uncertainties, arising from the loss of precision induced by
numerical calculation with limited time or hardware, are important for much
contemporary science and industry. Within applications such as climate science
and astrophysics, the need to make decisions on the basis of computations with
large and complex data has led to a renewed focus on the management of
numerical uncertainty. We describe how several seminal classic numerical
methods can be interpreted naturally as probabilistic inference. We then show
that the probabilistic view suggests new algorithms that can flexibly be
adapted to suit application specifics, while delivering improved empirical
performance. We provide concrete illustrations of the benefits of probabilistic
numeric algorithms on real scientific problems from astrometry and astronomical
imaging, while highlighting open problems with these new algorithms. Finally,
we describe how probabilistic numerical methods provide a coherent framework
for identifying the uncertainty in calculations performed with a combination of
numerical algorithms (e.g. both numerical optimisers and differential equation
solvers), potentially allowing the diagnosis (and control) of error sources in
computations.Comment: Author Generated Postprint. 17 pages, 4 Figures, 1 Tabl
Stochastic rounding and reduced-precision fixed-point arithmetic for solving neural ordinary differential equations
Although double-precision floating-point arithmetic currently dominates
high-performance computing, there is increasing interest in smaller and simpler
arithmetic types. The main reasons are potential improvements in energy
efficiency and memory footprint and bandwidth. However, simply switching to
lower-precision types typically results in increased numerical errors. We
investigate approaches to improving the accuracy of reduced-precision
fixed-point arithmetic types, using examples in an important domain for
numerical computation in neuroscience: the solution of Ordinary Differential
Equations (ODEs). The Izhikevich neuron model is used to demonstrate that
rounding has an important role in producing accurate spike timings from
explicit ODE solution algorithms. In particular, fixed-point arithmetic with
stochastic rounding consistently results in smaller errors compared to single
precision floating-point and fixed-point arithmetic with round-to-nearest
across a range of neuron behaviours and ODE solvers. A computationally much
cheaper alternative is also investigated, inspired by the concept of dither
that is a widely understood mechanism for providing resolution below the least
significant bit (LSB) in digital signal processing. These results will have
implications for the solution of ODEs in other subject areas, and should also
be directly relevant to the huge range of practical problems that are
represented by Partial Differential Equations (PDEs).Comment: Submitted to Philosophical Transactions of the Royal Society
Convergence Rates of Gaussian ODE Filters
A recently-introduced class of probabilistic (uncertainty-aware) solvers for
ordinary differential equations (ODEs) applies Gaussian (Kalman) filtering to
initial value problems. These methods model the true solution and its first
derivatives \emph{a priori} as a Gauss--Markov process ,
which is then iteratively conditioned on information about . This
article establishes worst-case local convergence rates of order for a
wide range of versions of this Gaussian ODE filter, as well as global
convergence rates of order in the case of and an integrated Brownian
motion prior, and analyses how inaccurate information on coming from
approximate evaluations of affects these rates. Moreover, we show that, in
the globally convergent case, the posterior credible intervals are well
calibrated in the sense that they globally contract at the same rate as the
truncation error. We illustrate these theoretical results by numerical
experiments which might indicate their generalizability to .Comment: 26 pages, 5 figure
- ā¦