FOOD AND BEVERAGE STOCK PRICE PREDICTION USING STACKED LSTM BASED ON DATA PIPELINE WITH APACHE SPARK AND APACHE AIRFLOW

Abstract

Stock price movements in the food and beverage sector exhibit time series characteristics that are fluctuative, nonlinear, and influenced by historical patterns, making them difficult to predict using conventional methods. In addition, processing large-scale stock data requires an integrated and automated system. This study aims to design a stock price prediction system using the Stacked Long Short-Term Memory (LSTM) model integrated with a data pipeline based on Apache Spark and Apache Airflow. The research method includes the development of a data pipeline (data lake, staging, preprocessing) and the implementation of a three-layer stacked LSTM model. Model evaluation was conducted using four LSTM unit configuration patterns (128-64-32, 64-64-32, 128-128-32, and 200-200-200). Time series analysis using ADF and ACF-PACF tests indicates that the data becomes stationary after differencing (p-value 0.0000). Temporal patterns are captured through lag features with optimal lag values of 1 (GOOD), 4 (ICBP), and 3 (MYOR), and a global lag of 4.The results show that the model achieves good performance with relatively low error rates. The best performance for MYOR stock is obtained using pattern 2 with RMSE of 65.50, MAE of 50.47, and MAPE of 2.28%. For ICBP stock, the best performance is also achieved using pattern 2 with RMSE of 277.48, MAE of 218.84, and MAPE of 2.36%. Meanwhile, for GOOD stock, the best performance is achieved using pattern 3 with RMSE of 11.68, MAE of 9.51, and MAPE of 2.57%. Overall, the MAPE values range from 2% to 4%, indicating that the model has good prediction accuracy. In addition, the data pipeline runs automatically with a 100% task success rate and efficient processing time.In conclusion, the Stacked LSTM model with appropriate pattern configurations is capable of effectively capturing temporal patterns and producing an accurate, efficient, and automated stock price prediction system