A Bayesian Nonparametric Markovian Model for Nonstationary Time Series

Stationary time series models built from parametric distributions are, in general, limited in scope due to the assumptions imposed on the residual distribution and autoregression relationship. We present a modeling approach for univariate time series data, which makes no assumptions of stationarity, and can accommodate complex dynamics and capture nonstandard distributions. The model for the transition density arises from the conditional distribution implied by a Bayesian nonparametric mixture of bivariate normals. This implies a flexible autoregressive form for the conditional transition density, defining a time-homogeneous, nonstationary, Markovian model for real-valued data indexed in discrete-time. To obtain a more computationally tractable algorithm for posterior inference, we utilize a square-root-free Cholesky decomposition of the mixture kernel covariance matrix. Results from simulated data suggest the model is able to recover challenging transition and predictive densities. We also illustrate the model on time intervals between eruptions of the Old Faithful geyser. Extensions to accommodate higher order structure and to develop a state-space model are also discussed

Time Series Forecasting: The Case for the Single Source of Error State Space

The state space approach to modelling univariate time series is now widely used both in theory and in applications. However, the very richness of the framework means that quite different model formulations are possible, even when they purport to describe the same phenomena. In this paper, we examine the single source of error [SSOE] scheme, which has perfectly correlated error components. We then proceed to compare SSOE to the more common version of the state space models, for which all the error terms are independent; we refer to this as the multiple source of error [MSOE] scheme. As expected, there are many similarities between the MSOE and SSOE schemes, but also some important differences. Both have ARIMA models as their reduced forms, although the mapping is more transparent for SSOE. Further, SSOE does not require a canonical form to complete its specification. An appealing feature of SSOE is that the estimates of the state variables converge in probability to their true values, thereby leading to a formal inferential structure for the ad-hoc exponential smoothing methods for forecasting. The parameter space for SSOE models may be specified to match that of the corresponding ARIMA scheme, or it may be restricted to meaningful sub-spaces, as for MSOE but with somewhat different outcomes. The SSOE formulation enables straightforward extensions to certain classes of non-linear models, including a linear trend with multiplicative seasonals version that underlies the Holt-Winters forecasting method. Conditionally heteroscedastic models may be developed in a similar manner. Finally we note that smoothing and decomposition, two crucial practical issues, may be performed within the SSOE framework.ARIMA, Dynamic Linear Models, Equivalence, Exponential Smoothing, Forecasting, GARCH, Holt's Method, Holt-Winters Method, Kalman Filter, Prediction Intervals.

Data-driven model reduction-based nonlinear MPC for large-scale distributed parameter systems

This is the author accepted manuscript. The final version is available from Elsevier via the DOI in this recordModel predictive control (MPC) has been effectively applied in process industries since the 1990s. Models in the form of closed equation sets are normally needed for MPC, but it is often difficult to obtain such formulations for large nonlinear systems. To extend nonlinear MPC (NMPC) application to nonlinear distributed parameter systems (DPS) with unknown dynamics, a data-driven model reduction-based approach is followed. The proper orthogonal decomposition (POD) method is first applied off-line to compute a set of basis functions. Then a series of artificial neural networks (ANNs) are trained to effectively compute POD time coefficients. NMPC, using sequential quadratic programming is then applied. The novelty of our methodology lies in the application of POD's highly efficient linear decomposition for the consequent conversion of any distributed multi-dimensional space-state model to a reduced 1-dimensional model, dependent only on time, which can be handled effectively as a black-box through ANNs. Hence we construct a paradigm, which allows the application of NMPC to complex nonlinear high-dimensional systems, even input/output systems, handled by black-box solvers, with significant computational efficiency. This paradigm combines elements of gain scheduling, NMPC, model reduction and ANN for effective control of nonlinear DPS. The stabilization/destabilization of a tubular reactor with recycle is used as an illustrative example to demonstrate the efficiency of our methodology. Case studies with inequality constraints are also presented.The authors would like to acknowledge the financial support of the EC FP6 Project: CONNECT [COOP-2006-31638] and the EC FP7 project CAFE [KBBE-212754]

Multiscale Information Decomposition: Exact Computation for Multivariate Gaussian Processes

Exploiting the theory of state space models, we derive the exact expressions of the information transfer, as well as redundant and synergistic transfer, for coupled Gaussian processes observed at multiple temporal scales. All of the terms, constituting the frameworks known as interaction information decomposition and partial information decomposition, can thus be analytically obtained for different time scales from the parameters of the VAR model that fits the processes. We report the application of the proposed methodology firstly to benchmark Gaussian systems, showing that this class of systems may generate patterns of information decomposition characterized by mainly redundant or synergistic information transfer persisting across multiple time scales or even by the alternating prevalence of redundant and synergistic source interaction depending on the time scale. Then, we apply our method to an important topic in neuroscience, i.e., the detection of causal interactions in human epilepsy networks, for which we show the relevance of partial information decomposition to the detection of multiscale information transfer spreading from the seizure onset zone

Metastable Decomposition of High-Dimensional Meteorological Data with Gaps

This paper presents an extension of the recently developed method for simultaneous dimension reduction and metastability analysis of high-dimensional time series. The modified approach is based on a combination of ensembles of hidden Markov models (HMMs) with state-specific principal component analysis (PCA) in extended space (guaranteeing that the overall dynamics will be Markovian). The main advantage of the modified method is its ability to deal with the gaps in the high-dimensional observation data. The proposed method allows for (i) the separation of the data according to the metastable states, (ii) a hierarchical decomposition of these sets into metastable substates, and (iii) calculation of the state-specific extended empirical orthogonal functions simultaneously with identification of the underlying Markovian dynamics switching between those metastable substates. The authors discuss the introduced model assumptions, explain how the quality of the resulting reduced representation can be assessed, and show what kind of additional insight into the underlying dynamics such a reduced Markovian representation can give (e.g., in the form of transition probabilities, statistical weights, mean first exit times, and mean first passage times). The performance of the new method analyzing 500-hPa geopotential height fields [daily mean values from the 40-yr ECMWF Re-Analysis (ERA-40) dataset for a period of 44 winters] is demonstrated and the results are compared with information gained from a numerically expensive but assumption-free method (Wavelets–PCA), and the identified metastable states are interpreted w.r.t. the blocking events in the atmosphere