A factor augmented vector autoregressive model and a stacked de-noising auto-encoders forecast combination to predict the price of oil

The following dissertation aims to show the benefits of a forecast combination between an econometric and a deep learning approach. On one side, a Factor Augmented Vector Autoregressive Model (FAVAR) with naming variables identification following Stock and Watson (2016)1; on the other side, a Stacked De-noising Auto-Encoder with Bagging (SDAE-B) following Zhao, Li and Yu (2017)2 are implemented. From January 2010 to September 2018 Two-hundred-eighty-one monthly series are used to predict the price of the West Texas Intermediate (WTI). The model performance is analysed by Root Mean Squared Error (RMSE), Mean Absolute Percentage Error (MAPE) and Directional Accuracy (DA). The combination benefits from both SDAE-B’s high accuracy and FAVAR’s interpretation features through impulse response functions (IRFs) and forecast error variance decomposition (FEVD)

Improved sparse autoencoder based artificial neural network approach for prediction of heart disease

Abstract:In this paper a two stage method is proposed to effectively predict heart disease. The first stage involves training an improved sparse autoencoder (SAE), an unsupervised neural network, to learn the best representation of the training data. The second stage involves using an artificial neural network (ANN) to predict the health status based on the learned records. The SAE was optimized so as to train an efficient model. The experimental result shows that the proposed method improves the performance of the ANN classifier, and is more robust as compared to other methods and similar scholarly works

다 변수 시계열 예측을 위한 주의 기반 LSTM 모델 연구

학위논문(석사)--서울대학교 대학원 :공과대학 컴퓨터공학부,2019. 8. Kang, U.Given previous observations of a multivariate time-series, how can we accurately predict the future value of several steps ahead? With the continuous development of sensor systems and computer systems, time-series prediction techniques are playing more and more important roles in various fields, such as finance, energy, and traffics. Many models have been proposed for time-series prediction tasks, such as Autoregressive model, Vector Autoregressive model, and Recurrent Neural Networks (RNNs). However, these models still have limitations like failure in modeling non-linearity and long-term dependencies in time-series. Among all the proposed approaches, the Temporal Pattern Attention (TPA), which is an attention-based LSTM model, achieves state-of-the-art performance on several real-world multivariate time-series datasets. In this thesis, we study three factors that effect the prediction performance of TPA model, which are the Recurrent Neural Network RNN layer, the attention mechanism, and the Convolutional Neural Network for temporal patter detection. For recurrent layer, we implement bi-directional LSTMs that can extract information from the input sequence in both forward and backward directions. In addition, we design two attention mechanisms, each of which assigns attention weights in different directions. We study the effect of both attention mechanisms on TPA model. Finally, to validate the Convolutional Neural Network (CNN) for temporal pattern detection, we implement a TPA model without CNN. We test all of these factors using several real-world time-series datasets from different fields. The experimental results indicate the validity of these factors.I. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 II. Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.1 Long Short-term Memory . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.2 Typical Attention Mechanism . . . . . . . . . . . . . . . . . . . . . . 5 2.3 Temporal Pattern Attention Model . . . . . . . . . . . . . . . . . . . . 6 III. Study on Temporal Pattern Attention Model . . . . . . . . . . . . . . 9 3.1 Problem Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.2 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.3 Recurrent Neural Network Layer . . . . . . . . . . . . . . . . . . . . . 10 3.4 Vertical v.s. Horizontal Attention Mechanism . . . . . . . . . . . . . . 12 3.5 Temporal Pattern Attention Model without CNN . . . . . . . . . . . . 14 IV. Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 4.1 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 4.2 Performance Comparison (Q1) . . . . . . . . . . . . . . . . . . . . . . 18 4.3 Effects of Bi-directional LSTM (Q2) . . . . . . . . . . . . . . . . . . . 20 4.4 Effects of CNN for Temporal Pattern Detection (Q3) . . . . . . . . . . 22 4.5 Which Attention Direction Is Better (Q4) . . . . . . . . . . . . . . . . 23 V. RelatedWorks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 VI. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30Maste