Efficient State-Space Inference of Periodic Latent Force Models

Latent force models (LFM) are principled approaches to incorporating solutions to differential equations within non-parametric inference methods. Unfortunately, the development and application of LFMs can be inhibited by their computational cost, especially when closed-form solutions for the LFM are unavailable, as is the case in many real world problems where these latent forces exhibit periodic behaviour. Given this, we develop a new sparse representation of LFMs which considerably improves their computational efficiency, as well as broadening their applicability, in a principled way, to domains with periodic or near periodic latent forces. Our approach uses a linear basis model to approximate one generative model for each periodic force. We assume that the latent forces are generated from Gaussian process priors and develop a linear basis model which fully expresses these priors. We apply our approach to model the thermal dynamics of domestic buildings and show that it is effective at predicting day-ahead temperatures within the homes. We also apply our approach within queueing theory in which quasi-periodic arrival rates are modelled as latent forces. In both cases, we demonstrate that our approach can be implemented efficiently using state-space methods which encode the linear dynamic systems via LFMs. Further, we show that state estimates obtained using periodic latent force models can reduce the root mean squared error to 17% of that from non-periodic models and 27% of the nearest rival approach which is the resonator model.Comment: 61 pages, 13 figures, accepted for publication in JMLR. Updates from earlier version occur throughout article in response to JMLR review

Efficient state-space inference of periodic latent force models

Latent force models (LFM) are principled approaches to incorporating solutions to differen-tial equations within non-parametric inference methods. Unfortunately, the developmentand application of LFMs can be inhibited by their computational cost, especially whenclosed-form solutions for the LFM are unavailable, as is the case in many real world prob-lems where these latent forces exhibit periodic behaviour. Given this, we develop a newsparse representation of LFMs which considerably improves their computational efficiency,as well as broadening their applicability, in a principled way, to domains with periodic ornear periodic latent forces. Our approach uses a linear basis model to approximate onegenerative model for each periodic force. We assume that the latent forces are generatedfrom Gaussian process priors and develop a linear basis model which fully expresses thesepriors. We apply our approach to model the thermal dynamics of domestic buildings andshow that it is effective at predicting day-ahead temperatures within the homes. We alsoapply our approach within queueing theory in which quasi-periodic arrival rates are mod-elled as latent forces. In both cases, we demonstrate that our approach can be implemented efficiently using state-space methods which encode the linear dynamic systems via LFMs.Further, we show that state estimates obtained using periodic latent force models can re-duce the root mean squared error to 17% of that from non-periodic models and 27% of thenearest rival approach which is the resonator model (S ̈arkk ̈a et al., 2012; Hartikainen et al.,2012.

A reduced-order strategy for 4D-Var data assimilation

This paper presents a reduced-order approach for four-dimensional variational data assimilation, based on a prior EO F analysis of a model trajectory. This method implies two main advantages: a natural model-based definition of a mul tivariate background error covariance matrix $\textbf{B}_r$, and an important decrease of the computational burden o f the method, due to the drastic reduction of the dimension of the control space. % An illustration of the feasibility and the effectiveness of this method is given in the academic framework of twin experiments for a model of the equatorial Pacific ocean. It is shown that the multivariate aspect of $\textbf{B}_r$ brings additional information which substantially improves the identification procedure. Moreover the computational cost can be decreased by one order of magnitude with regard to the full-space 4D-Var method

Reliability-based design optimization using kriging surrogates and subset simulation

The aim of the present paper is to develop a strategy for solving reliability-based design optimization (RBDO) problems that remains applicable when the performance models are expensive to evaluate. Starting with the premise that simulation-based approaches are not affordable for such problems, and that the most-probable-failure-point-based approaches do not permit to quantify the error on the estimation of the failure probability, an approach based on both metamodels and advanced simulation techniques is explored. The kriging metamodeling technique is chosen in order to surrogate the performance functions because it allows one to genuinely quantify the surrogate error. The surrogate error onto the limit-state surfaces is propagated to the failure probabilities estimates in order to provide an empirical error measure. This error is then sequentially reduced by means of a population-based adaptive refinement technique until the kriging surrogates are accurate enough for reliability analysis. This original refinement strategy makes it possible to add several observations in the design of experiments at the same time. Reliability and reliability sensitivity analyses are performed by means of the subset simulation technique for the sake of numerical efficiency. The adaptive surrogate-based strategy for reliability estimation is finally involved into a classical gradient-based optimization algorithm in order to solve the RBDO problem. The kriging surrogates are built in a so-called augmented reliability space thus making them reusable from one nested RBDO iteration to the other. The strategy is compared to other approaches available in the literature on three academic examples in the field of structural mechanics.Comment: 20 pages, 6 figures, 5 tables. Preprint submitted to Springer-Verla

State Space Methods in Ox/SsfPack

The use of state space models and their inference is illustrated using the package SsfPack for Ox. After a rather long introduction that explains the use of SsfPack and many of its functions, four case-studies illustrate the practical implementation of the software to real world problems through short sample programs. The first case consists in the analysis of the well-known (at least to time series analysis experts) Nile data with a local level model. The other case-studies deal with ARIMA and RegARIMA models applied to the (also well-known) Airline time series, structural time series models applied to the Italian industrial production index and stochastic volatility models applied to the FTSE100 index. In all applications inference on the model (hyper-) parameters is carried out by maximum likelihood, but in one case (stochastic volatility) also an MCMC-based approach is illustrated. Cubic splines are covered in a very short example as well.

Structural Drift: The Population Dynamics of Sequential Learning

We introduce a theory of sequential causal inference in which learners in a chain estimate a structural model from their upstream teacher and then pass samples from the model to their downstream student. It extends the population dynamics of genetic drift, recasting Kimura's selectively neutral theory as a special case of a generalized drift process using structured populations with memory. We examine the diffusion and fixation properties of several drift processes and propose applications to learning, inference, and evolution. We also demonstrate how the organization of drift process space controls fidelity, facilitates innovations, and leads to information loss in sequential learning with and without memory.Comment: 15 pages, 9 figures; http://csc.ucdavis.edu/~cmg/compmech/pubs/sdrift.ht

A spatiotemporal stochastic framework of groundwater fluctuation analysis on the South - Eastern part of the great Hungarian plain

The current study was performed on a Hungarian area where the groundwater has been highly affected in the past 40 years by climate change. The stochastic estimation framework of groundwater as a spatiotemporally varying dynamic phenomenon is proposed. The probabilistic estimation of the water depth is performed as a joint realization of spatially correlated hydrographs, where parametric temporal trend models are fitted to the measured time series thereafter regionalized in space. Two types of trend models are evaluated. Due to its simplicity the purely mathematical trend can be used to analyze long-term groundwater trends, the average water fluctuation range and to determine the most probable date of peak groundwater level. The one which takes advantage of the knowledge of expected groundwater changes, clearly over performed the purely mathematical model, and it is selected for the construction of a spatiotemporal trend. Model fitting error values are considered as a set of stochastic time series which expresses short-term anomalies of the groundwater, and they are modelled as joint space-time distribution. The resulting spatiotemporal residual field is added to the trend field, thus resulting 125 simulated realizations, which are evaluated probabilistically. The high number of joint spatiotemporal realizations provides alternative groundwater datasets as boundary conditions for a wide variety of environmental models, while the presented procedure behaves more robust over non-complete datasets