12,500 research outputs found
Exploring Interpretable LSTM Neural Networks over Multi-Variable Data
For recurrent neural networks trained on time series with target and
exogenous variables, in addition to accurate prediction, it is also desired to
provide interpretable insights into the data. In this paper, we explore the
structure of LSTM recurrent neural networks to learn variable-wise hidden
states, with the aim to capture different dynamics in multi-variable time
series and distinguish the contribution of variables to the prediction. With
these variable-wise hidden states, a mixture attention mechanism is proposed to
model the generative process of the target. Then we develop associated training
methods to jointly learn network parameters, variable and temporal importance
w.r.t the prediction of the target variable. Extensive experiments on real
datasets demonstrate enhanced prediction performance by capturing the dynamics
of different variables. Meanwhile, we evaluate the interpretation results both
qualitatively and quantitatively. It exhibits the prospect as an end-to-end
framework for both forecasting and knowledge extraction over multi-variable
data.Comment: Accepted to International Conference on Machine Learning (ICML), 201
Time series forecasting with the WARIMAX-GARCH method
It is well-known that causal forecasting methods that include appropriately chosen Exogenous Variables (EVs) very often present improved forecasting performances over univariate methods. However, in practice, EVs are usually difficult to obtain and in many cases are not available at all. In this paper, a new causal forecasting approach, called Wavelet Auto-Regressive Integrated Moving Average with eXogenous variables and Generalized Auto-Regressive Conditional Heteroscedasticity (WARIMAX-GARCH) method, is proposed to improve predictive performance and accuracy but also to address, at least in part, the problem of unavailable EVs. Basically, the WARIMAX-GARCH method obtains Wavelet âEVsâ (WEVs) from Auto-Regressive Integrated Moving Average with eXogenous variables and Generalized Auto-Regressive Conditional Heteroscedasticity (ARIMAX-GARCH) models applied to Wavelet Components (WCs) that are initially determined from the underlying time series. The WEVs are, in fact, treated by the WARIMAX-GARCH method as if they were conventional EVs. Similarly to GARCH and ARIMA-GARCH models, the WARIMAX-GARCH method is suitable for time series exhibiting non-linear characteristics such as conditional variance that depends on past values of observed data. However, unlike those, it can explicitly model frequency domain patterns in the series to help improve predictive performance. An application to a daily time series of dam displacement in Brazil shows the WARIMAX-GARCH method to remarkably outperform the ARIMA-GARCH method, as well as the (multi-layer perceptron) Artificial Neural Network (ANN) and its wavelet version referred to as Wavelet Artificial Neural Network (WANN) as in [1], on statistical measures for both in-sample and out-of-sample forecasting
- âŠ