13,157 research outputs found
Kernel Analog Forecasting: Multiscale Test Problems
Data-driven prediction is becoming increasingly widespread as the volume of
data available grows and as algorithmic development matches this growth. The
nature of the predictions made, and the manner in which they should be
interpreted, depends crucially on the extent to which the variables chosen for
prediction are Markovian, or approximately Markovian. Multiscale systems
provide a framework in which this issue can be analyzed. In this work kernel
analog forecasting methods are studied from the perspective of data generated
by multiscale dynamical systems. The problems chosen exhibit a variety of
different Markovian closures, using both averaging and homogenization;
furthermore, settings where scale-separation is not present and the predicted
variables are non-Markovian, are also considered. The studies provide guidance
for the interpretation of data-driven prediction methods when used in practice.Comment: 30 pages, 14 figures; clarified several ambiguous parts, added
references, and a comparison with Lorenz' original method (Sec. 4.5
Efficient state-space inference of periodic latent force models
Latent force models (LFM) are principled approaches to incorporating solutions to differen-tial equations within non-parametric inference methods. Unfortunately, the developmentand application of LFMs can be inhibited by their computational cost, especially whenclosed-form solutions for the LFM are unavailable, as is the case in many real world prob-lems where these latent forces exhibit periodic behaviour. Given this, we develop a newsparse representation of LFMs which considerably improves their computational efficiency,as well as broadening their applicability, in a principled way, to domains with periodic ornear periodic latent forces. Our approach uses a linear basis model to approximate onegenerative model for each periodic force. We assume that the latent forces are generatedfrom Gaussian process priors and develop a linear basis model which fully expresses thesepriors. We apply our approach to model the thermal dynamics of domestic buildings andshow that it is effective at predicting day-ahead temperatures within the homes. We alsoapply our approach within queueing theory in which quasi-periodic arrival rates are mod-elled as latent forces. In both cases, we demonstrate that our approach can be implemented efficiently using state-space methods which encode the linear dynamic systems via LFMs.Further, we show that state estimates obtained using periodic latent force models can re-duce the root mean squared error to 17% of that from non-periodic models and 27% of thenearest rival approach which is the resonator model (S ̈arkk ̈a et al., 2012; Hartikainen et al.,2012.
Efficient State-Space Inference of Periodic Latent Force Models
Latent force models (LFM) are principled approaches to incorporating
solutions to differential equations within non-parametric inference methods.
Unfortunately, the development and application of LFMs can be inhibited by
their computational cost, especially when closed-form solutions for the LFM are
unavailable, as is the case in many real world problems where these latent
forces exhibit periodic behaviour. Given this, we develop a new sparse
representation of LFMs which considerably improves their computational
efficiency, as well as broadening their applicability, in a principled way, to
domains with periodic or near periodic latent forces. Our approach uses a
linear basis model to approximate one generative model for each periodic force.
We assume that the latent forces are generated from Gaussian process priors and
develop a linear basis model which fully expresses these priors. We apply our
approach to model the thermal dynamics of domestic buildings and show that it
is effective at predicting day-ahead temperatures within the homes. We also
apply our approach within queueing theory in which quasi-periodic arrival rates
are modelled as latent forces. In both cases, we demonstrate that our approach
can be implemented efficiently using state-space methods which encode the
linear dynamic systems via LFMs. Further, we show that state estimates obtained
using periodic latent force models can reduce the root mean squared error to
17% of that from non-periodic models and 27% of the nearest rival approach
which is the resonator model.Comment: 61 pages, 13 figures, accepted for publication in JMLR. Updates from
earlier version occur throughout article in response to JMLR review
How priors of initial hyperparameters affect Gaussian process regression models
The hyperparameters in Gaussian process regression (GPR) model with a
specified kernel are often estimated from the data via the maximum marginal
likelihood. Due to the non-convexity of marginal likelihood with respect to the
hyperparameters, the optimization may not converge to the global maxima. A
common approach to tackle this issue is to use multiple starting points
randomly selected from a specific prior distribution. As a result the choice of
prior distribution may play a vital role in the predictability of this
approach. However, there exists little research in the literature to study the
impact of the prior distributions on the hyperparameter estimation and the
performance of GPR. In this paper, we provide the first empirical study on this
problem using simulated and real data experiments. We consider different types
of priors for the initial values of hyperparameters for some commonly used
kernels and investigate the influence of the priors on the predictability of
GPR models. The results reveal that, once a kernel is chosen, different priors
for the initial hyperparameters have no significant impact on the performance
of GPR prediction, despite that the estimates of the hyperparameters are very
different to the true values in some cases
Gaussian process models for periodicity detection
We consider the problem of detecting and quantifying the periodic component
of a function given noise-corrupted observations of a limited number of
input/output tuples. Our approach is based on Gaussian process regression which
provides a flexible non-parametric framework for modelling periodic data. We
introduce a novel decomposition of the covariance function as the sum of
periodic and aperiodic kernels. This decomposition allows for the creation of
sub-models which capture the periodic nature of the signal and its complement.
To quantify the periodicity of the signal, we derive a periodicity ratio which
reflects the uncertainty in the fitted sub-models. Although the method can be
applied to many kernels, we give a special emphasis to the Mat\'ern family,
from the expression of the reproducing kernel Hilbert space inner product to
the implementation of the associated periodic kernels in a Gaussian process
toolkit. The proposed method is illustrated by considering the detection of
periodically expressed genes in the arabidopsis genome.Comment: in PeerJ Computer Science, 201
Predictability on finite horizon for processes with exponential decrease of energy on higher frequencies
The paper presents sufficient conditions of predictability for continuous
time processes in deterministic setting. We found that processes with
exponential decay on energy for higher frequencies are predictable in some weak
sense on some finite time horizon defined by the rate of decay. Moreover, this
predictability can be achieved uniformly over classes of processes. Some
explicit formulas for predictors are suggested.Comment: 11 page
- …