LSTM-SDM: An integrated framework of LSTM implementation for sequential data modeling[Formula presented]

LSTM-SDM is a python-based integrated computational framework built on the top of Tensorflow/Keras and written in the Jupyter notebook. It provides several object-oriented functionalities for implementing single layer and multilayer LSTM models for sequential data modeling and time series forecasting. Multiple subroutines are blended to create a conducive user-friendly environment that facilitates data exploration and visualization, normalization and input preparation, hyperparameter tuning, performance evaluations, visualization of results, and statistical analysis. We utilized the LSTM-SDM framework in predicting the stock market index and observed impressive results. The framework can be generalized to solve several other real-world time series problems

Concatenation of fundamental and financial news data.

Concatenation of fundamental and financial news data.</p

Scatter plots between the actual closing price and predicted closing price of test data corresponding to the models: (a) LSTM-News, (b) GRU-News, (c) LSTM, and (d) GRU.

The best-fit linear equation (y = x) is represented by the red dotted line.</p

Overall schematic diagram of the proposed research framework.

Overall schematic diagram of the proposed research framework.</p

Hypothesis together with p-values of the normality test.

Hypothesis together with p-values of the normality test.</p

Model performance metrics of four models obtained using test datasets.

Model performance metrics of four models obtained using test datasets.</p

Long short-term memory(LSTM) architecture [56].

The accelerated progress in artificial intelligence encourages sophisticated deep learning methods in predicting stock prices. In the meantime, easy accessibility of the stock market in the palm of one’s hand has made its behavior more fuzzy, volatile, and complex than ever. The world is looking at an accurate and reliable model that uses text and numerical data which better represents the market’s highly volatile and non-linear behavior in a broader spectrum. A research gap exists in accurately predicting a target stock’s closing price utilizing the combined numerical and text data. This study uses long short-term memory (LSTM) and gated recurrent unit (GRU) to predict the stock price using stock features alone and incorporating financial news data in conjunction with stock features. The comparative study carried out under identical conditions dispassionately evaluates the importance of incorporating financial news in stock price prediction. Our experiment concludes that incorporating financial news data produces better prediction accuracy than using the stock fundamental features alone. The performances of the model architecture are compared using the standard assessment metrics —Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE), and Correlation Coefficient (R). Furthermore, statistical tests are conducted to further verify the models’ robustness and reliability.</div

P-values of the Mann-Whitney test for pairwise comparisons within the models.

P-values of the Mann-Whitney test for pairwise comparisons within the models.</p

Time series plots between the actual closing price and predicted closing price of test data corresponding to the models: (a) LSTM-News, (b) GRU-News, (c) LSTM, and (d) GRU.

Time series plots between the actual closing price and predicted closing price of test data corresponding to the models: (a) LSTM-News, (b) GRU-News, (c) LSTM, and (d) GRU.</p

Overall description of the datasets.

The accelerated progress in artificial intelligence encourages sophisticated deep learning methods in predicting stock prices. In the meantime, easy accessibility of the stock market in the palm of one’s hand has made its behavior more fuzzy, volatile, and complex than ever. The world is looking at an accurate and reliable model that uses text and numerical data which better represents the market’s highly volatile and non-linear behavior in a broader spectrum. A research gap exists in accurately predicting a target stock’s closing price utilizing the combined numerical and text data. This study uses long short-term memory (LSTM) and gated recurrent unit (GRU) to predict the stock price using stock features alone and incorporating financial news data in conjunction with stock features. The comparative study carried out under identical conditions dispassionately evaluates the importance of incorporating financial news in stock price prediction. Our experiment concludes that incorporating financial news data produces better prediction accuracy than using the stock fundamental features alone. The performances of the model architecture are compared using the standard assessment metrics —Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE), and Correlation Coefficient (R). Furthermore, statistical tests are conducted to further verify the models’ robustness and reliability.</div