52,927 research outputs found
Scalable iterative methods for sampling from massive Gaussian random vectors
Sampling from Gaussian Markov random fields (GMRFs), that is multivariate
Gaussian ran- dom vectors that are parameterised by the inverse of their
covariance matrix, is a fundamental problem in computational statistics. In
this paper, we show how we can exploit arbitrarily accu- rate approximations to
a GMRF to speed up Krylov subspace sampling methods. We also show that these
methods can be used when computing the normalising constant of a large
multivariate Gaussian distribution, which is needed for both any
likelihood-based inference method. The method we derive is also applicable to
other structured Gaussian random vectors and, in particu- lar, we show that
when the precision matrix is a perturbation of a (block) circulant matrix, it
is still possible to derive O(n log n) sampling schemes.Comment: 17 Pages, 4 Figure
String and Membrane Gaussian Processes
In this paper we introduce a novel framework for making exact nonparametric
Bayesian inference on latent functions, that is particularly suitable for Big
Data tasks. Firstly, we introduce a class of stochastic processes we refer to
as string Gaussian processes (string GPs), which are not to be mistaken for
Gaussian processes operating on text. We construct string GPs so that their
finite-dimensional marginals exhibit suitable local conditional independence
structures, which allow for scalable, distributed, and flexible nonparametric
Bayesian inference, without resorting to approximations, and while ensuring
some mild global regularity constraints. Furthermore, string GP priors
naturally cope with heterogeneous input data, and the gradient of the learned
latent function is readily available for explanatory analysis. Secondly, we
provide some theoretical results relating our approach to the standard GP
paradigm. In particular, we prove that some string GPs are Gaussian processes,
which provides a complementary global perspective on our framework. Finally, we
derive a scalable and distributed MCMC scheme for supervised learning tasks
under string GP priors. The proposed MCMC scheme has computational time
complexity and memory requirement , where
is the data size and the dimension of the input space. We illustrate the
efficacy of the proposed approach on several synthetic and real-world datasets,
including a dataset with millions input points and attributes.Comment: To appear in the Journal of Machine Learning Research (JMLR), Volume
1
Investigating dynamic dependence using copulae
A general methodology for time series modelling is developed which works down from distributional
properties to implied structural models including the standard regression relationship. This
general to specific approach is important since it can avoid spurious assumptions such as linearity
in the form of the dynamic relationship between variables. It is based on splitting the multivariate
distribution of a time series into two parts: (i) the marginal unconditional distribution, (ii) the
serial dependence encompassed in a general function , the copula. General properties of the class of
copula functions that fulfill the necessary requirements for Markov chain construction are exposed.
Special cases for the gaussian copula with AR(p) dependence structure and for archimedean copulae
are presented. We also develop copula based dynamic dependency measures — auto-concordance
in place of autocorrelation. Finally, we provide empirical applications using financial returns and
transactions based forex data. Our model encompasses the AR(p) model and allows non-linearity.
Moreover, we introduce non-linear time dependence functions that generalize the autocorrelation
function
Ellipsoidal Prediction Regions for Multivariate Uncertainty Characterization
While substantial advances are observed in probabilistic forecasting for
power system operation and electricity market applications, most approaches are
still developed in a univariate framework. This prevents from informing about
the interdependence structure among locations, lead times and variables of
interest. Such dependencies are key in a large share of operational problems
involving renewable power generation, load and electricity prices for instance.
The few methods that account for dependencies translate to sampling scenarios
based on given marginals and dependence structures. However, for classes of
decision-making problems based on robust, interval chance-constrained
optimization, necessary inputs take the form of polyhedra or ellipsoids.
Consequently, we propose a systematic framework to readily generate and
evaluate ellipsoidal prediction regions, with predefined probability and
minimum volume. A skill score is proposed for quantitative assessment of the
quality of prediction ellipsoids. A set of experiments is used to illustrate
the discrimination ability of the proposed scoring rule for misspecification of
ellipsoidal prediction regions. Application results based on three datasets
with wind, PV power and electricity prices, allow us to assess the skill of the
resulting ellipsoidal prediction regions, in terms of calibration, sharpness
and overall skill.Comment: 8 pages, 7 Figures, Submitted to IEEE Transactions on Power System
A New Monte Carlo Based Algorithm for the Gaussian Process Classification Problem
Gaussian process is a very promising novel technology that has been applied
to both the regression problem and the classification problem. While for the
regression problem it yields simple exact solutions, this is not the case for
the classification problem, because we encounter intractable integrals. In this
paper we develop a new derivation that transforms the problem into that of
evaluating the ratio of multivariate Gaussian orthant integrals. Moreover, we
develop a new Monte Carlo procedure that evaluates these integrals. It is based
on some aspects of bootstrap sampling and acceptancerejection. The proposed
approach has beneficial properties compared to the existing Markov Chain Monte
Carlo approach, such as simplicity, reliability, and speed
- …