Probabilistic Numerics and Uncertainty in Computations

We deliver a call to arms for probabilistic numerical methods: algorithms for numerical tasks, including linear algebra, integration, optimization and solving differential equations, that return uncertainties in their calculations. Such uncertainties, arising from the loss of precision induced by numerical calculation with limited time or hardware, are important for much contemporary science and industry. Within applications such as climate science and astrophysics, the need to make decisions on the basis of computations with large and complex data has led to a renewed focus on the management of numerical uncertainty. We describe how several seminal classic numerical methods can be interpreted naturally as probabilistic inference. We then show that the probabilistic view suggests new algorithms that can flexibly be adapted to suit application specifics, while delivering improved empirical performance. We provide concrete illustrations of the benefits of probabilistic numeric algorithms on real scientific problems from astrometry and astronomical imaging, while highlighting open problems with these new algorithms. Finally, we describe how probabilistic numerical methods provide a coherent framework for identifying the uncertainty in calculations performed with a combination of numerical algorithms (e.g. both numerical optimisers and differential equation solvers), potentially allowing the diagnosis (and control) of error sources in computations.Comment: Author Generated Postprint. 17 pages, 4 Figures, 1 Tabl

On the computation of Gaussian quadrature rules for Chebyshev sets of linearly independent functions

We consider the computation of quadrature rules that are exact for a Chebyshev set of linearly independent functions on an interval $[a,b]$. A general theory of Chebyshev sets guarantees the existence of rules with a Gaussian property, in the sense that $2l$ basis functions can be integrated exactly with just $l$ points and weights. Moreover, all weights are positive and the points lie inside the interval $[a,b]$. However, the points are not the roots of an orthogonal polynomial or any other known special function as in the case of regular Gaussian quadrature. The rules are characterized by a nonlinear system of equations, and earlier numerical methods have mostly focused on finding suitable starting values for a Newton iteration to solve this system. In this paper we describe an alternative scheme that is robust and generally applicable for so-called complete Chebyshev sets. These are ordered Chebyshev sets where the first $k$ elements also form a Chebyshev set for each $k$. The points of the quadrature rule are computed one by one, increasing exactness of the rule in each step. Each step reduces to finding the unique root of a univariate and monotonic function. As such, the scheme of this paper is guaranteed to succeed. The quadrature rules are of interest for integrals with non-smooth integrands that are not well approximated by polynomials

Fast matrix computations for pair-wise and column-wise commute times and Katz scores

We first explore methods for approximating the commute time and Katz score between a pair of nodes. These methods are based on the approach of matrices, moments, and quadrature developed in the numerical linear algebra community. They rely on the Lanczos process and provide upper and lower bounds on an estimate of the pair-wise scores. We also explore methods to approximate the commute times and Katz scores from a node to all other nodes in the graph. Here, our approach for the commute times is based on a variation of the conjugate gradient algorithm, and it provides an estimate of all the diagonals of the inverse of a matrix. Our technique for the Katz scores is based on exploiting an empirical localization property of the Katz matrix. We adopt algorithms used for personalized PageRank computing to these Katz scores and theoretically show that this approach is convergent. We evaluate these methods on 17 real world graphs ranging in size from 1000 to 1,000,000 nodes. Our results show that our pair-wise commute time method and column-wise Katz algorithm both have attractive theoretical properties and empirical performance.Comment: 35 pages, journal version of http://dx.doi.org/10.1007/978-3-642-18009-5_13 which has been submitted for publication. Please see http://www.cs.purdue.edu/homes/dgleich/publications/2011/codes/fast-katz/ for supplemental code

On the convergence of spectral deferred correction methods

In this work we analyze the convergence properties of the Spectral Deferred Correction (SDC) method originally proposed by Dutt et al. [BIT, 40 (2000), pp. 241--266]. The framework for this high-order ordinary differential equation (ODE) solver is typically described wherein a low-order approximation (such as forward or backward Euler) is lifted to higher order accuracy by applying the same low-order method to an error equation and then adding in the resulting defect to correct the solution. Our focus is not on solving the error equation to increase the order of accuracy, but on rewriting the solver as an iterative Picard integral equation solver. In doing so, our chief finding is that it is not the low-order solver that picks up the order of accuracy with each correction, but it is the underlying quadrature rule of the right hand side function that is solely responsible for picking up additional orders of accuracy. Our proofs point to a total of three sources of errors that SDC methods carry: the error at the current time point, the error from the previous iterate, and the numerical integration error that comes from the total number of quadrature nodes used for integration. The second of these two sources of errors is what separates SDC methods from Picard integral equation methods; our findings indicate that as long as difference between the current and previous iterate always gets multiplied by at least a constant multiple of the time step size, then high-order accuracy can be found even if the underlying "solver" is inconsistent the underlying ODE. From this vantage, we solidify the prospects of extending spectral deferred correction methods to a larger class of solvers to which we present some examples.Comment: 29 page