9,718 research outputs found
Probabilistic Interpretation of Linear Solvers
This manuscript proposes a probabilistic framework for algorithms that
iteratively solve unconstrained linear problems with positive definite
for . The goal is to replace the point estimates returned by existing
methods with a Gaussian posterior belief over the elements of the inverse of
, which can be used to estimate errors. Recent probabilistic interpretations
of the secant family of quasi-Newton optimization algorithms are extended.
Combined with properties of the conjugate gradient algorithm, this leads to
uncertainty-calibrated methods with very limited cost overhead over conjugate
gradients, a self-contained novel interpretation of the quasi-Newton and
conjugate gradient algorithms, and a foundation for new nonlinear optimization
methods.Comment: final version, in press at SIAM J Optimizatio
Probabilistic Linear Solvers: A Unifying View
Several recent works have developed a new, probabilistic interpretation for
numerical algorithms solving linear systems in which the solution is inferred
in a Bayesian framework, either directly or by inferring the unknown action of
the matrix inverse. These approaches have typically focused on replicating the
behavior of the conjugate gradient method as a prototypical iterative method.
In this work surprisingly general conditions for equivalence of these disparate
methods are presented. We also describe connections between probabilistic
linear solvers and projection methods for linear systems, providing a
probabilistic interpretation of a far more general class of iterative methods.
In particular, this provides such an interpretation of the generalised minimum
residual method. A probabilistic view of preconditioning is also introduced.
These developments unify the literature on probabilistic linear solvers, and
provide foundational connections to the literature on iterative solvers for
linear systems
Probabilistic ODE Solvers with Runge-Kutta Means
Runge-Kutta methods are the classic family of solvers for ordinary
differential equations (ODEs), and the basis for the state of the art. Like
most numerical methods, they return point estimates. We construct a family of
probabilistic numerical methods that instead return a Gauss-Markov process
defining a probability distribution over the ODE solution. In contrast to prior
work, we construct this family such that posterior means match the outputs of
the Runge-Kutta family exactly, thus inheriting their proven good properties.
Remaining degrees of freedom not identified by the match to Runge-Kutta are
chosen such that the posterior probability measure fits the observed structure
of the ODE. Our results shed light on the structure of Runge-Kutta solvers from
a new direction, provide a richer, probabilistic output, have low computational
cost, and raise new research questions.Comment: 18 pages (9 page conference paper, plus supplements); appears in
Advances in Neural Information Processing Systems (NIPS), 201
Probabilistic Numerics and Uncertainty in Computations
We deliver a call to arms for probabilistic numerical methods: algorithms for
numerical tasks, including linear algebra, integration, optimization and
solving differential equations, that return uncertainties in their
calculations. Such uncertainties, arising from the loss of precision induced by
numerical calculation with limited time or hardware, are important for much
contemporary science and industry. Within applications such as climate science
and astrophysics, the need to make decisions on the basis of computations with
large and complex data has led to a renewed focus on the management of
numerical uncertainty. We describe how several seminal classic numerical
methods can be interpreted naturally as probabilistic inference. We then show
that the probabilistic view suggests new algorithms that can flexibly be
adapted to suit application specifics, while delivering improved empirical
performance. We provide concrete illustrations of the benefits of probabilistic
numeric algorithms on real scientific problems from astrometry and astronomical
imaging, while highlighting open problems with these new algorithms. Finally,
we describe how probabilistic numerical methods provide a coherent framework
for identifying the uncertainty in calculations performed with a combination of
numerical algorithms (e.g. both numerical optimisers and differential equation
solvers), potentially allowing the diagnosis (and control) of error sources in
computations.Comment: Author Generated Postprint. 17 pages, 4 Figures, 1 Tabl
- …