Time series prediction and forecasting using Deep learning Architectures

Nature brings time series data everyday and everywhere, for example, weather data, physiological signals and biomedical signals, financial and business recordings. Predicting the future observations of a collected sequence of historical observations is called time series forecasting. Forecasts are essential, considering the fact that they guide decisions in many areas of scientific, industrial and economic activity such as in meteorology, telecommunication, finance, sales and stock exchange rates. A massive amount of research has already been carried out by researchers over many years for the development of models to improve the time series forecasting accuracy. The major aim of time series modelling is to scrupulously examine the past observation of time series and to develop an appropriate model which elucidate the inherent behaviour and pattern existing in time series. The behaviour and pattern related to various time series may possess different conventions and infact requires specific countermeasures for modelling. Consequently, retaining the neural networks to predict a set of time series of mysterious domain remains particularly challenging. Time series forecasting remains an arduous problem despite the fact that there is substantial improvement in machine learning approaches. This usually happens due to some factors like, different time series may have different flattering behaviour. In real world time series data, the discriminative patterns residing in the time series are often distorted by random noise and affected by high-frequency perturbations. The major aim of this thesis is to contribute to the study and expansion of time series prediction and multistep ahead forecasting method based on deep learning algorithms. Time series forecasting using deep learning models is still in infancy as compared to other research areas for time series forecasting.Variety of time series data has been considered in this research. We explored several deep learning architectures on the sequential data, such as Deep Belief Networks (DBNs), Stacked AutoEncoders (SAEs), Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs). Moreover, we also proposed two different new methods based on muli-step ahead forecasting for time series data. The comparison with state of the art methods is also exhibited. The research work conducted in this thesis makes theoretical, methodological and empirical contributions to time series prediction and multi-step ahead forecasting by using Deep Learning Architectures

다차원 거래 데이터와 딥러닝 모형을 이용한 금융 시장 복잡계 네트워크 분석

학위논문(박사) -- 서울대학교대학원 : 공과대학 산업공학과, 2021.8. 장우진.주식은 독립적 요소가 아닌, 다른 자산들을 포함한 다양한 요소들과 상호 연관성이 있다. 따라서 주식의 상호 연관성을 분석하는 것은 주식 시장의 구조를 묘사하고 이해하기 위한 필수적 요소일 뿐만 아니라, 투자와 위험관리와 같은 실무적 관점에서도 중요한 요소이다. 주식 시장은 개별 요인의 상호 작용을 통해 형성된 복잡계 네트워크라는 관점하에, 많은 연구들이 주식 간의 상호 연관성을 분석하기 위해 경제물리학의 네트워크 분석을 사용하고 있다. 주식 간의 상호 관계를 정의 할 때, 많은 연구에서 주가 시계열에만 초점을 둔다. 그러나, 주식을 나타내는 정보는 주가뿐만 아니라 거래량, 변동성, 그리고 회계 정보와 같은 다양한 데이터가 있다. 따라서 주가만을 이용하여 주식 간의 상호 연관성을 분석하는 것은, 금융 시장에 대한 단편적인 시각만을 의미할 수 있다. 게다가, 딥러닝과 같은 기술력의 발전에 따라, 학문과 데이터의 종류의 경계를 허무는 학제간 융합의 중요성이 대두되고 있다. 이러한 맥락에서 본 논문의 목적은 경제 물리학과 딥러닝의 학제 간 융합을 통해 금융 시장 분석에 대한 새로운 시각을 제시하는데 있다. 우리는 S\&P500 주식 시장을 다변량 관점에서 분석하기 위해 이차원 히스토그램(Bi-dimensional histogram)과 오토인코더(Autoencoder)에 기반한 딥러닝과 경제물리학이 융합된 프레임워크(Framework)를 제안한다. 먼저, 우리는 주식을 다변량으로 표현하기 위해 주가와 거래량을 통해 구성된 이차원 히스토그램을 정의한다. 각 주식 별 이차원 히스토그램은 일별 주가 시계열과 거래량 시계열로 구성되며, 주식을 분포 관점에서 직관적으로 표현하는데 목적이 있다. 그 다음, 이차원 히스토그램의 차원을 축소하고 의미있는 변수를 추출하기 위해 오토인코더가 적용된다. 추출된 변수를 통해 이차원 히스토그램 간의 거리 행렬(Distance matrix)이 정의되고, 거리행렬에 경제물리학 방법론인 PMFG(Planar Maximally Filtered Graph) 알고리즘을 적용하여 금융 시장을 나타내는 히스토그램 네트워크가 구축된다. 구축된 네트워크는 이차원 히스토그램의 잠재 공간(Latent space)를 의미하며, 금융 시장의 구조적 특성을 분석하기 위한 네트워크 분석이 수행된다. 우리는 히스토그램에 기반한 네트워크의 구조적 특성이 히스토그램의 산포도와 연관이 있음을 발견하였다. 이를 통해 우리는 오토인코더가 이차원 히스토그램의 내재된 변수를 추출하는데 효과적임을 확인하였다. 또한 우리는 히스토그램 네트워크는 기존 방법론인 주가 시계열 기반의 네트워크와 위상학적 특징이 서로 상이함을 발견했다. 추가적으로, 히스토그램 네트워크는 기존 방법론 보다 변동성과 시장 베타(Beta)와 같은 금융 시장에 대한 외생적인 특성이 유사한 주식들을 구조적으로 군집화하는데 더 효과가 있다. 더 나아가, 우리는 히스토그램 네트워크에 기반한 그래프 뉴럴 네트워크(Graph Neural Network) 주가 추세 예측 모형을 제안한다. 주가 추세 예측은 실무적 중요성을 바탕으로, 계량적 모형부터 기계학습 모형과 딥러닝 모형까지 광범위하게 연구되고 있는 주제이다. 최근 그래프 뉴럴 네트워크 모형은 요소 간의 상호 연관성을 반영 할 수 있는 특징을 바탕으로 시계열 예측 분야에서 주목 받고 있다. 최근, 여러 연구들이 그래프 뉴럴 네트워크에 주식 간의 상호 연관성을 반영하여 주가 추세 예측에 대한 연구를 진행헀다. 하지만, 이러한 선행 연구들은 주식 간의 상호 연관성을 정립하는데 산업구조, 비즈니스 영역과 같은 정적이고 사전 정의가 요구되는 관계만을 이용한다. 하지만 많은 연구들을 통해 금융 시장의 구조 및 주식 간의 상호 연관성의 시변 속성(Time-varying property)이 입증되었다. 따라서 우리는 금융 시장의 시변 속성을 모형에 반영하고자, 히스토그램 네트워크를 주식 간 상호 관계로 활용하는 그래프 뉴럴 네트워크 모형을 제안한다. 주가 추세 예측 실험은 10일과 20일 뒤의 주가의 방향 분류(Classification) 문제로 이루어진다. 실험 결과, 우리는 본 학위 논문에서 제안하는 모형이 벤치마크(Benchmark) 대비 우월한 예측 정확도와, F1-Score를 갖는 것을 확인했다. 마지막으로 우리는 주가 추세 예측 모형의 실무적 활용도를 검증하기 위해 예측 모형에 기반한 투자 전략을 제안한다. 제안하는 투자 전략은 전체 주식을 대상으로 추세 예측을 수행했을 때, 상승 확률이 가장 높은 $N$개의 주식을 투자 포트폴리오로 구성하는 것이다. 즉, 우리는 히스토그램 네트워크에 기반한 그래프 뉴럴 네트워크 모형으로 추세 예측을 수행하고, 예측 결과를 활용하여 투자 포트폴리오를 구성한다. 투자 실험 결과, 제안하는 투자 전략은 전통적 투자 전략 대비 샤프 지수(Sharpe ratio), 소르티노 지수(Sortino ratio)와 같은 수익성 지표가 뛰어난 전략일 뿐만 아니라, 하락 추세에서 가장 빠르게 벗어날 수 있는 투자 전략이다. 정리하면, 본 학위 논문은 경제물리학과 딥러닝의 융합에 기반한 금융 시장 구조화 방법론 및 예측 모델을 제안하는데 의의가 있으며, 군집화, 예측, 그리고 투자를 통해 그 유용성을 검증하였다.Stocks are correlated with a variety of factors, such as other financial assets and economic factors. Therefore, analyzing the relationship of stocks is an essential element for understanding the stock market structure and an essential element from a practical point of view, such as investment and risk management. Stock markets that fluctuate chaotically are modeled as complex networks formed through the interaction of individual factors. Many studies have used network analysis in econophysics to investigate the relationship between stocks. In defining interactions between stocks, many studies highlighted stock price time series. However, information on stocks is not only contained in the price, but also in various data such as trading volumes, volatility, and accounting variables. Therefore, the relationship between stocks using only the stock price can show a fragmentary view of the financial market. In addition, with the development of technologies such as deep learning, the boundaries between disciplines and data types are collapsing, and interdisciplinary convergence is becoming more significant. In this context, the purpose of this dissertation is to present a new perspective on financial market analysis through the interdisciplinary convergence of econophysics and deep learning. We propose a framework that combines deep learning and econophysics consisting of a bi-dimensional histogram and an autoencoder to analyze the S\&P500 stock market from a multivariate perspective. We utilize a bi-dimensional histogram to intuitively represent stock trading volume information as well as stock price information. Then, an autoencoder is applied to reduce the dimension of the bi-dimensional histogram and extract the latent vector. A distance matrix between bi-dimensional histograms is defined through the latent vectors, and a histogram network representing the financial market is constructed by applying the Planar Maximally Filtered Graph(PMFG) algorithm, an econophysics methodology, to the distance matrix. The constructed network implies the latent space of the bi-dimensional histogram, and network analysis is performed to analyze the structural properties of the financial market. We reveal that the structural properties of the histogram network are related to the dispersion of histogram, which means that the autoencoder is effective in extracting the latent vector of the bi-dimensional histogram. Additionally, the histogram network effectively clusters stocks with similar exogenous characteristics to financial markets, such as volatility and market beta. Furthermore, we propose a stock price prediction model based on a Graph Neural Network(GNN) and histogram network. Based on its practical importance, stock price trend prediction is a research topic studied extensively, from econometric models to machine learning and deep learning. Recently, GNN models are in the spotlight in time series prediction based on their properties that can reflect the relationship between elements. However, previous studies use only static and predefined relationships such as industrial structures to establish the relationship between stocks. However, many studies have demonstrated the time-varying property of the financial market. Therefore, we propose a graph neural network model that uses the histogram network as a relationship between stocks to reflect the time-varying properties of the financial market. The trend prediction experiment is performed with a classification problem of the stock price after 10 and 20 days. As a result, we confirmed that the proposed model has superior prediction accuracy and F1-Score compared to the benchmark. Finally, we propose an investment strategy based on the predictive model to verify the practical utility of the proposed model. The proposed investment strategy is to construct an investment portfolio of $N$ stocks that are most likely to rise in price in the future. In other words, we perform trend prediction with a graph neural network model based on a histogram network and construct an investment portfolio using the prediction results. Our results show that the proposed investment strategy is outstanding in profitability, including the Sharpe ratio and Sortino ratio and the cumulative return. In addition, our proposed investment strategy can recover from a downtrend faster than traditional investment strategies based on Modern Portfolio Theory. In summary, the novelty of the dissertation is that we construct a stock market PMFG network and build a GNN model based on the interdisciplinary convergence of econophysics and deep learning. The usefulness of the dissertation is verified through clustering, prediction, and investment experiments.Chapter 1 Introduction 1 1.1 Research Motivation and Purpose 1 1.2 Organization of the Research 8 Chapter 2 Literature Review 11 2.1 Stock market network 11 2.2 Autoencoder in Stock market 12 2.3 Predictive model in Stock market 13 Chapter 3 Bi-dimensional Histogram Network 17 3.1 Architecture of Bi-dimensional Histogram Network 17 3.1.1 Bi-dimensional Histogram 17 3.1.2 Auto-encoder 21 3.1.3 Stock market PMFG network 25 3.2 Data description 28 3.3 Experimental Results 29 3.3.1 Properties of bi-dimensional histogram 29 3.3.2 Dimensional reduction result of bi-dimensional histogram 34 3.3.3 Properties of bi-dimensional histogram network 36 3.3.4 Comparison of histogram network with price network 40 3.4 Summary and Discussion 47 Chapter 4 A hybrid graph neural network for time series trend prediction 49 4.1 Proposed Model 49 4.2 Benchmark Algorithms 58 4.3 Data Description 63 4.4 Experimental Procedure 64 4.5 Experimental Result 68 4.5.1 Properties of High-Low volatility PMFG 68 4.5.2 Prediction Performance 74 4.5.3 Robustness Test for Prediction Performance 79 4.6 Summary and Discussion 82 Chapter 5 Investment Strategy with Trend Prediction Model 85 5.1 Proposed Strategy 85 5.2 Benchmark Algorithms 87 5.3 Experiment Procedure 91 5.4 Experimental Result 94 5.5 Summary and Discussion 110 Chapter 6 Conclusion 113 6.1 Conclusions 113 6.2 Future Works 117 Appendix 119 Bibliography 129 국문초록 145박

DeepLOB: Deep Convolutional Neural Networks for Limit Order Books

We develop a large-scale deep learning model to predict price movements from limit order book (LOB) data of cash equities. The architecture utilises convolutional filters to capture the spatial structure of the limit order books as well as LSTM modules to capture longer time dependencies. The proposed network outperforms all existing state-of-the-art algorithms on the benchmark LOB dataset [1]. In a more realistic setting, we test our model by using one year market quotes from the London Stock Exchange and the model delivers a remarkably stable out-of-sample prediction accuracy for a variety of instruments. Importantly, our model translates well to instruments which were not part of the training set, indicating the model's ability to extract universal features. In order to better understand these features and to go beyond a "black box" model, we perform a sensitivity analysis to understand the rationale behind the model predictions and reveal the components of LOBs that are most relevant. The ability to extract robust features which translate well to other instruments is an important property of our model which has many other applications.Comment: 12 pages, 9 figure

Enhanced news sentiment analysis using deep learning methods

We explore the predictive power of historical news sentiments based on financial market performance to forecast financial news sentiments. We define news sentiments based on stock price returns averaged over one minute right after a news article has been released. If the stock price exhibits positive (negative) return, we classify the news article released just prior to the observed stock return as positive (negative). We use Wikipedia and Gigaword five corpus articles from 2014 and we apply the global vectors for word representation method to this corpus to create word vectors to use as inputs into the deep learning TensorFlow network. We analyze high-frequency (intraday) Thompson Reuters News Archive as well as the high-frequency price tick history of the Dow Jones Industrial Average (DJIA 30) Index individual stocks for the period between 1/1/2003 and 12/30/2013. We apply a combination of deep learning methodologies of recurrent neural network with long short-term memory units to train the Thompson Reuters News Archive Data from 2003 to 2012, and we test the forecasting power of our method on 2013 News Archive data. We find that the forecasting accuracy of our methodology improves when we switch from random selection of positive and negative news to selecting the news with highest positive scores as positive news and news with highest negative scores as negative news to create our training data set.Published versio