Exploring ICA for time series decomposition

In this paper, we apply independent component analysis (ICA) for prediction and signal extraction in multivariate time series data. We compare the performance of three different ICA procedures, JADE, SOBI, and FOTBI that estimate the components exploiting either the non-Gaussianity, or the temporal structure of the data, or combining both, non-Gaussianity as well as temporal dependence. Some Monte Carlo simulation experiments are carried out to investigate the performance of these algorithms in order to extract components such as trend, cycle, and seasonal components. Moreover, we empirically test the performance of those three ICA procedures on capturing the dynamic relationships among the industrial production index (IPI) time series of four European countries. We also compare the accuracy of the IPI time series forecasts using a few JADE, SOBI, and FOTBI components, at different time horizons. According to the results, FOTBI seems to be a good starting point for automatic time series signal extraction procedures, and it also provides quite accurate forecasts for the IPIs.ICA, Signal extraction, Multivariate time series, Forecasting

Optimized Forecasting of Dominant U.S. Stock Market Equities Using Univariate and Multivariate Time Series Analysis Methods

This dissertation documents an investigation into forecasting U.S. stock market equities via two very different time series analysis techniques: 1) autoregressive integrated moving average (ARIMA), and 2) singular spectrum analysis (SSA). Approximately 40% of the S&P 500 stocks are analyzed. Forecasts are generated for one and five days ahead using daily closing prices. Univariate and multivariate structures are applied and results are compared. One objective is to explore the hypothesis that a multivariate model produces superior performance over a univariate configuration. Another objective is to compare the forecasting performance of ARIMA to SSA, as SSA is a relatively recent development and has shown much potential. Stochastic characteristics of stock market data are analyzed and found to be definitely not Gaussian, but instead better fit to a generalized t-distribution. Probability distribution models are validated with goodness-of-fit tests. For analysis, stock data is segmented into non-overlapping time “windows” to support unconditional statistical evaluation. Univariate and multivariate ARIMA and SSA time series models are evaluated for independence. ARIMA models are found to be independent, but SSA models are not able to reach independence. Statistics for out-of-sample forecasts are computed for every stock in every window, and multivariate-univariate confidence interval shrinkages are examined. Results are compared for univariate, bivariate, and trivariate combinations of highly-correlated stocks. Effects are found to be mixed. Bivariate modeling and forecasting with three different covariates are investigated. Examination of results with covariates of trading volume, principal component analysis (PCA), and volatility reveal that PCA exhibits the best overall forecasting accuracy in the entire field of investigated elements, including univariate models. Bivariate-PCA structures are applied in a back-testing environment to evaluate economic significance and robustness of the methods. Initial results of back-testing yielded similar results to those from earlier independent testing. Inconsistent performance across test intervals inspired the development of a second technique that yields improved results and positive economic significance. Robustness is validated through back-testing across multiple market trends

Multivariate streamflow forecasting using independent component analysis

Seasonal forecasting of streamflow provides many benefits to society, by improving our ability to plan and adapt to changing water supplies. A common approach to developing these forecasts is to use statistical methods that link a set of predictors representing climate state as it relates to historical streamflow, and then using this model to project streamflow one or more seasons in advance based on current or a projected climate state. We present an approach for forecasting multivariate time series using independent component analysis (ICA) to transform the multivariate data to a set of univariate time series that are mutually independent, thereby allowing for the much broader class of univariate models to provide seasonal forecasts for each transformed series. Uncertainty is incorporated by bootstrapping the error component of each univariate model so that the probability distribution of the errors is maintained. Although all analyses are performed on univariate time series, the spatial dependence of the streamflow is captured by applying the inverse ICA transform to the predicted univariate series. We demonstrate the technique on a multivariate streamflow data set in Colombia, South America, by comparing the results to a range of other commonly used forecasting methods. The results show that the ICA-based technique is significantly better at representing spatial dependence, while not resulting in any loss of ability in capturing temporal dependence. As such, the ICA-based technique would be expected to yield considerable advantages when used in a probabilistic setting to manage large reservoir systems with multiple inflows or data collection points.Seth Westra, Ashish Sharma, Casey Brown and Upmanu Lal

Forecasting multiple functional time series in a group structure: an application to mortality’

When modeling sub-national mortality rates, we should consider three features: (1) how to incorporate any possible correlation among sub-populations to potentially improve forecast accuracy through multi-population joint modeling; (2) how to reconcile sub-national mortality forecasts so that they aggregate adequately across various levels of a group structure; (3) among the forecast reconciliation methods, how to combine their forecasts to achieve improved forecast accuracy. To address these issues, we introduce an extension of grouped univariate functional time series method. We first consider a multivariate functional time series method to jointly forecast multiple related series. We then evaluate the impact and benefit of using forecast combinations among the forecast reconciliation methods. Using the Japanese regional age-specific mortality rates, we investigate one-step-ahead to 15-step-ahead point and interval forecast accuracies of our proposed extension and make recommendations

Machine Learning Based Data Driven Modelling of Time Series of Power Plant Data

Accurate modeling and simulation of data collected from a power plant system are important factors in the strategic planning and maintenance of the unit. Several non-linearities and multivariable couplings are associated with real-world plants. Therefore, it becomes almost impossible to model the system using conventional mathematical equations. Statistical models such as ARIMA, ARMA are potential solutions but their linear nature cannot very well t a system with non-linear, multivariate time series data. Recently, deep learning methods such as Arti cial Neural Networks (ANNs) have been extensively applied for time series forecasting. ANNs in contrast to stochastic models such as ARIMA can uncover the non-linearities present underneath the data. In this thesis, we analyze the real-time temperature data obtained from a nuclear power plant, and discover the patterns and characteristics of the sensory data. Principal Component Analysis (PCA) followed by Linear Discriminant Analysis (LDA) is used to extract features from the time series data; k-means clustering is applied to label the data instances. Finite state machine representation formulated from the clustered data is then used to model the behaviour of nuclear power plants using system states and state transitions. Dependent and independent parameters of the system are de ned based on co-relation among themselves. Various forecasting models are then applied over multivariate time-stamped data. We discuss thoroughly the implementation of a key architecture of neural networks, Long Short-Term Neural Networks (LSTMs). LSTM can capture nonlinear relationships in a dynamic system using its memory connections. This further aids them to counter the problem of back-propagated error decay through memory blocks. Poly-regression is applied to represent the working of the plant by de ning an association between independent and dependent parameters. This representation is then used to forecast dependent variates based on the observed values of independent variates. Principle of sensitivity analysis is used for optimisation of number of parameters used for predicting. It helps in making a compromise between number of parameters used and level of accuracy achieved in forecasting. The objective of this thesis is to examine the feasibility of the above-mentioned forecasting techniques in the modeling of a complex time series of data, and predicting system parameters such as Reactor Temperature and Linear Power based on past information. It also carries out a comparative analysis of forecasts obtained in each approach

A multivariate generalized independent factor GARCH model with an application to financial stock returns

We propose a new multivariate factor GARCH model, the GICA-GARCH model , where the data are assumed to be generated by a set of independent components (ICs). This model applies independent component analysis (ICA) to search the conditionally heteroskedastic latent factors. We will use two ICA approaches to estimate the ICs. The first one estimates the components maximizing their non-gaussianity, and the second one exploits the temporal structure of the data. After estimating the ICs, we fit an univariate GARCH model to the volatility of each IC. Thus, the GICA-GARCH reduces the complexity to estimate a multivariate GARCH model by transforming it into a small number of univariate volatility models. We report some simulation experiments to show the ability of ICA to discover leading factors in a multivariate vector of financial data. An empirical application to the Madrid stock market will be presented, where we compare the forecasting accuracy of the GICA-GARCH model versus the orthogonal GARCH one

Multivariate Bayesian Predictive Synthesis in Macroeconomic Forecasting

We develop the methodology and a detailed case study in use of a class of Bayesian predictive synthesis (BPS) models for multivariate time series forecasting. This extends the recently introduced foundational framework of BPS to the multivariate setting, with detailed application in the topical and challenging context of multi-step macroeconomic forecasting in a monetary policy setting. BPS evaluates-- sequentially and adaptively over time-- varying forecast biases and facets of miscalibration of individual forecast densities, and-- critically-- of time-varying inter-dependencies among them over multiple series. We develop new BPS methodology for a specific subclass of the dynamic multivariate latent factor models implied by BPS theory. Structured dynamic latent factor BPS is here motivated by the application context-- sequential forecasting of multiple US macroeconomic time series with forecasts generated from several traditional econometric time series models. The case study highlights the potential of BPS to improve of forecasts of multiple series at multiple forecast horizons, and its use in learning dynamic relationships among forecasting models or agents

Posterior mean and variance approximation for regression and time series problems

This paper develops a methodology for approximating the posterior first two moments of the posterior distribution in Bayesian inference. Partially specified probability models that are defined only by specifying means and variances, are constructed based upon second-order conditional independence in order to facilitate posterior updating and prediction of required distributional quantities. Such models are formulated particularly for multivariate regression and time series analysis with unknown observational variance-covariance components. The similarities and differences of these models with the Bayes linear approach are established. Several subclasses of important models, including regression and time series models with errors following multivariate t, inverted multivariate t and Wishart distributions, are discussed in detail. Two numerical examples consisting of simulated data and of US investment and change in inventory data illustrate the proposed methodology