116,177 research outputs found
Probabilistic Numerics and Uncertainty in Computations
We deliver a call to arms for probabilistic numerical methods: algorithms for
numerical tasks, including linear algebra, integration, optimization and
solving differential equations, that return uncertainties in their
calculations. Such uncertainties, arising from the loss of precision induced by
numerical calculation with limited time or hardware, are important for much
contemporary science and industry. Within applications such as climate science
and astrophysics, the need to make decisions on the basis of computations with
large and complex data has led to a renewed focus on the management of
numerical uncertainty. We describe how several seminal classic numerical
methods can be interpreted naturally as probabilistic inference. We then show
that the probabilistic view suggests new algorithms that can flexibly be
adapted to suit application specifics, while delivering improved empirical
performance. We provide concrete illustrations of the benefits of probabilistic
numeric algorithms on real scientific problems from astrometry and astronomical
imaging, while highlighting open problems with these new algorithms. Finally,
we describe how probabilistic numerical methods provide a coherent framework
for identifying the uncertainty in calculations performed with a combination of
numerical algorithms (e.g. both numerical optimisers and differential equation
solvers), potentially allowing the diagnosis (and control) of error sources in
computations.Comment: Author Generated Postprint. 17 pages, 4 Figures, 1 Tabl
Probabilistic Interpretation of Linear Solvers
This manuscript proposes a probabilistic framework for algorithms that
iteratively solve unconstrained linear problems with positive definite
for . The goal is to replace the point estimates returned by existing
methods with a Gaussian posterior belief over the elements of the inverse of
, which can be used to estimate errors. Recent probabilistic interpretations
of the secant family of quasi-Newton optimization algorithms are extended.
Combined with properties of the conjugate gradient algorithm, this leads to
uncertainty-calibrated methods with very limited cost overhead over conjugate
gradients, a self-contained novel interpretation of the quasi-Newton and
conjugate gradient algorithms, and a foundation for new nonlinear optimization
methods.Comment: final version, in press at SIAM J Optimizatio
A unified framework for solving a general class of conditional and robust set-membership estimation problems
In this paper we present a unified framework for solving a general class of
problems arising in the context of set-membership estimation/identification
theory. More precisely, the paper aims at providing an original approach for
the computation of optimal conditional and robust projection estimates in a
nonlinear estimation setting where the operator relating the data and the
parameter to be estimated is assumed to be a generic multivariate polynomial
function and the uncertainties affecting the data are assumed to belong to
semialgebraic sets. By noticing that the computation of both the conditional
and the robust projection optimal estimators requires the solution to min-max
optimization problems that share the same structure, we propose a unified
two-stage approach based on semidefinite-relaxation techniques for solving such
estimation problems. The key idea of the proposed procedure is to recognize
that the optimal functional of the inner optimization problems can be
approximated to any desired precision by a multivariate polynomial function by
suitably exploiting recently proposed results in the field of parametric
optimization. Two simulation examples are reported to show the effectiveness of
the proposed approach.Comment: Accpeted for publication in the IEEE Transactions on Automatic
Control (2014
Control Variates for Reversible MCMC Samplers
A general methodology is introduced for the construction and effective
application of control variates to estimation problems involving data from
reversible MCMC samplers. We propose the use of a specific class of functions
as control variates, and we introduce a new, consistent estimator for the
values of the coefficients of the optimal linear combination of these
functions. The form and proposed construction of the control variates is
derived from our solution of the Poisson equation associated with a specific
MCMC scenario. The new estimator, which can be applied to the same MCMC sample,
is derived from a novel, finite-dimensional, explicit representation for the
optimal coefficients. The resulting variance-reduction methodology is primarily
applicable when the simulated data are generated by a conjugate random-scan
Gibbs sampler. MCMC examples of Bayesian inference problems demonstrate that
the corresponding reduction in the estimation variance is significant, and that
in some cases it can be quite dramatic. Extensions of this methodology in
several directions are given, including certain families of Metropolis-Hastings
samplers and hybrid Metropolis-within-Gibbs algorithms. Corresponding
simulation examples are presented illustrating the utility of the proposed
methods. All methodological and asymptotic arguments are rigorously justified
under easily verifiable and essentially minimal conditions.Comment: 44 pages; 6 figures; 5 table
Bootstrapping spectral statistics in high dimensions
Statistics derived from the eigenvalues of sample covariance matrices are
called spectral statistics, and they play a central role in multivariate
testing. Although bootstrap methods are an established approach to
approximating the laws of spectral statistics in low-dimensional problems,
these methods are relatively unexplored in the high-dimensional setting. The
aim of this paper is to focus on linear spectral statistics as a class of
prototypes for developing a new bootstrap in high-dimensions --- and we refer
to this method as the Spectral Bootstrap. In essence, the method originates
from the parametric bootstrap, and is motivated by the notion that, in high
dimensions, it is difficult to obtain a non-parametric approximation to the
full data-generating distribution. From a practical standpoint, the method is
easy to use, and allows the user to circumvent the difficulties of complex
asymptotic formulas for linear spectral statistics. In addition to proving the
consistency of the proposed method, we provide encouraging empirical results in
a variety of settings. Lastly, and perhaps most interestingly, we show through
simulations that the method can be applied successfully to statistics outside
the class of linear spectral statistics, such as the largest sample eigenvalue
and others.Comment: 42 page
- …