Performance Forecasting of Share Market using Machine Learning Techniques: A Review

Forecasting share performance becomes more challenging issue due to the enormous amount of valuable trading data stored in the stock database. Currently, existing forecasting methods are insufficient to analyze the share performance accurately. There are two main reasons for that: First, the study of existing forecasting methods is still insufficient to identify the most suitable methods for share price prediction. Second, the lack of investigations made on the factors affecting the share performance. In this regard, this study presents a systematic review of the last fifteen years on various machine learning techniques in order to analyze share performance accurately. The only objective of this study is to provide an overview of the machine learning techniques that have been used to forecast share performance. This paper also highlights a how the prediction algorithms can be used to identify the most important variables in a share market dataset. Finally, we could have succeeded to analyze share performance effectively. It could bring benefits and impacts to researchers, society, brokers and financial analysts

The Performance of Maximum Likelihood Factor Analysis on South African Stock Price Performance

Abstract: The purpose of this paper is to explore the effectiveness and applicability of Maximum Likelihood Factor Analysis (MLFA) method on stock price performance. This method identifies the variables according to their co-movement and variability and builds a model that can be useful for prediction and ranking or classification. The results of factor analysis in this study provide a guide as far as investment decision is concerned. Stock price performance of the seven well-known and biggest companies listed in the Johannesburg stock exchange (JSE) was used as an experimental unit. Monthly data was available for the period 2010 to 2014.Details of a trivariate factor model is: Factor 1 comprises of Absa and Standard Bank (Financial sectors), Factor 2 has Shoprite and Pick ‘n Pay (Retail sectors) while Factor 3 collected Vodacom MTN and Sasol (Industrial sectors). The companies contribute 46.9%, 12.7% and 10.8% respectively to the three sectors and these findings are confirmed by a Chi-square and the Akaike information criterion to be valid. The three factors are also diverse and reliable according to Tucker and Lewis and Cronbach’s coefficients. The findings of this study give economic significance and the study is relevant as it gives investors and portfolio manager’s sensible investment reference.Keywords: Maximum Likelihood Factor Analysis, stock price

A Robust Regression-Based Stock Exchange Forecasting and Determination of Correlation between Stock Markets

Knowledge-based decision support systems for financial management are an important part of investment plans. Investors are avoiding investing in traditional investment areas such as banks due to low return on investment. The stock exchange is one of the major areas for investment presently. Various non-linear and complex factors affect the stock exchange. A robust stock exchange forecasting system remains an important need. From this line of research, we evaluate the performance of a regression-based model to check the robustness over large datasets. We also evaluate the effect of top stock exchange markets on each other. We evaluate our proposed model on the top 4 stock exchanges—New York, London, NASDAQ and Karachi stock exchange. We also evaluate our model on the top 3 companies—Apple, Microsoft, and Google. A huge (Big Data) historical data is gathered from Yahoo finance consisting of 20 years. Such huge data creates a Big Data problem. The performance of our system is evaluated on a 1-step, 6-step, and 12-step forecast. The experiments show that the proposed system produces excellent results. The results are presented in terms of Mean Absolute Error (MAE) and Root Mean Square Error (RMSE)

Improving Stock Trading Decisions Based on Pattern Recognition Using Machine Learning Technology

PRML, a novel candlestick pattern recognition model using machine learning methods, is proposed to improve stock trading decisions. Four popular machine learning methods and 11 different features types are applied to all possible combinations of daily patterns to start the pattern recognition schedule. Different time windows from one to ten days are used to detect the prediction effect at different periods. An investment strategy is constructed according to the identified candlestick patterns and suitable time window. We deploy PRML for the forecast of all Chinese market stocks from Jan 1, 2000 until Oct 30, 2020. Among them, the data from Jan 1, 2000 to Dec 31, 2014 is used as the training data set, and the data set from Jan 1, 2015 to Oct 30, 2020 is used to verify the forecasting effect. Empirical results show that the two-day candlestick patterns after filtering have the best prediction effect when forecasting one day ahead; these patterns obtain an average annual return, an annual Sharpe ratio, and an information ratio as high as 36.73%, 0.81, and 2.37, respectively. After screening, three-day candlestick patterns also present a beneficial effect when forecasting one day ahead in that these patterns show stable characteristics. Two other popular machine learning methods, multilayer perceptron network and long short-term memory neural networks, are applied to the pattern recognition framework to evaluate the dependency of the prediction model. A transaction cost of 0.2% is considered on the two-day patterns predicting one day ahead, thus confirming the profitability. Empirical results show that applying different machine learning methods to two-day and three-day patterns for one-day-ahead forecasts can be profitable

An empirical study on the various stock market prediction methods

Investment in the stock market is one of the much-admired investment actions. However, prediction of the stock market has remained a hard task because of the non-linearity exhibited. The non-linearity is due to multiple affecting factors such as global economy, political situations, sector performance, economic numbers, foreign institution investment, domestic institution investment, and so on. A proper set of such representative factors must be analyzed to make an efficient prediction model. Marginal improvement of prediction accuracy can be gainful for investors. This review provides a detailed analysis of research papers presenting stock market prediction techniques. These techniques are assessed in the time series analysis and sentiment analysis section. A detailed discussion on research gaps and issues is presented. The reviewed articles are analyzed based on the use of prediction techniques, optimization algorithms, feature selection methods, datasets, toolset, evaluation matrices, and input parameters. The techniques are further investigated to analyze relations of prediction methods with feature selection algorithm, datasets, feature selection methods, and input parameters. In addition, major problems raised in the present techniques are also discussed. This survey will provide researchers with deeper insight into various aspects of current stock market prediction methods

Generating Buy/Sell Signals for an Equity Share Using Machine Learning

This study proposes a novel model for predicting 5 days’ ahead share price direction of GARAN (Garanti Bankasi A.Ş.), an equity share that is the top traded stock in BIST100, Istanbul Stock Exchange -Turkey. The first model includes global macroeconomic indicators as well as local inputs whereas the second model is focused more on local inputs. The performances of the two models are tested using Support Vector Machines (SVM), Neural Network with Back-Propagation (BPN), and Decision Tree (DT) algorithms. Though BPN and SVM have previously been used to predict BIST100 Index movement, DT has not been utilized before with this purpose. Forecasting is carried out tested for a time span of about 6 months on a rolling horizon basis, that is, algorithms are re-run weekly with updated data to generate daily buy/sell signals for the next week. A simple trading strategy is implemented based on buy/sell signals to calculate the rate of return on investment during the testing period. The results illustrate that DT having 80% prediction accuracy outperforms BPN and SVM that achieve 60% accuracy. Consequently, DT achieves a higher rate of return

Essays on Financial Applications of Nonlinear Models

In this thesis, we examine the relationship between news and the stock market. Further, we explore methods and build new nonlinear models for forecasting stock price movement and portfolio optimization based on past stock prices and on one type of big data, news items, which are obtained through the RavenPack News Analytics Global Equities editions. The thesis consists of three essays. In Essay 1, we investigate the relationship between news items and stock prices using the artificial neural network (ANN) model. First, we use Granger causality to ascertain how news items affect stock prices. The results show that news volume is not the Granger cause of stock price change; rather, news sentiment is. Second, we test the semi–strong form efficient market hypothesis, whereas most existing research testing efficient market hypothesis focuses on the weak–form version. Our ANN strategies consistently outperform the passive buy–and–hold strategy and this finding is apparently at odds with the notion of the efficient market hypothesis. Finally, using news sentiment analytics from RavenPack Dow Jones News Analytics, we show positive profitability with out–of–sample prediction using the proposed ANN strategies for Google Inc. (NASDAQ: GOOG). In Essay 2, we expand the utility of the information from news volume and news sentiments to encompass portfolio diversification. For the Dow Jones Industrial Average (DJIA) components, we assign different weights to build portfolios according to their weekly news volumes or news sentiments. Our results show that news volume contributes to portfolio variance both in–sample and out–of–sample: positive news sentiment contributes to the portfolio return in–sample, while negative contributes to the portfolio return out–of–sample, which is a consequence of investors overreacting to the news sentiment. Further, we propose a novel approach to portfolio diversification using the k–Nearest Neighbors (kNN) algorithm based on the idea that news sentiment correlates with stock returns. Out–of–sample results indicate that such strategy dominates the benchmark DJIA index portfolio. In Essay 3, we propose a new model called the Combined Markov and Hidden Markov Model (CMHMM), in which observation is affected by a Markov model and an HMM (Hidden Markov Model) model. The three fundamental questions of the CMHMM are discussed. Further, the application of the CMHMM, in which the news sentiment is one observation and the stock return is the other, is discussed. The empirical results of the trading strategy based on the CMHMM show the potential applications of the proposed model in finance. This thesis contributes to the literature in a number of ways. First, it extends the literature on financial applications of nonlinear models. We explore the applications of the ANNs and kNN in the financial market. Besides, the proposed new CMHMM model adheres to the nature of the stock market and has better potential prediction ability. Second, the empirical results from this dissertation contribute to the understanding of the relationship between news and the stock market. For instance, our research found that news volume contributes to the portfolio return and that investors overreact to news sentiment—a phenomenon that has been discussed by other scholars from different angles

Machine Learning: A Potential Forecasting Toll

Technical analysis involves predicting asset price movements from analysis of historical prices. Many studies have been conducted to determine the profitability of technical analysis. A composite prediction is considered here by using the buy and sell signals from technical indicators as inputs. Both machine learning methods like neural networks and statistical methods like logistic regression are used to get composite forecasts. Signals from trend-following and mean-reversal technical indicators are used in addition to variance of prices as inputs. Variance is added to help technical indicators switch between trend-following and mean-reversal systems. Five commodities from agricultural, livestock and foreign exchange futures markets are selected to test the hypothesis of profitability of technical indicators. Special care is taken to avoid data snooping error. None of the individual indicators or machine learning models generate significant profit in single day forecasts. In twenty-day forecasts, only random forest and pipeline models are profitable. Neural networks and statistical models both failed to deliver here. The out of sample failure of the neural networks is partly due to the relatively large number of parameters. Managed futures, however also did poorly in the out of sample period so the results could also be due to picking a time period where technical analysis did poorly. Individual indicators did occasionally show significant profits. Random forests and decision tree find variance as the most important input. Future research should consider alternative time periods, commodities, systems, and machine learning algorithms. If a scale neutral variable for variance could be developed, it should be used so that the models could be trained on data from multiple commodities to provide more training data.Agricultural Economic

Winsorize tree algorithm for handling outliers in classification problem

Classification and Regression Tree (CART) is designed to predict or classify the objects in the predetermined classes from a set of predictors. However, having outliers could affect the structures of CART, purity and predictive accuracy in classification. Some researchers opt to perform pre-pruning or post-pruning of the CART in handling the outliers. This study proposes a modified classification tree algorithm called Winsorize tree based on the distribution of classes in the training dataset. The Winsorize tree investigates all possible outliers from node to node before checking the potential splitting point to gain the node with the highest purity of the nodes. The upper fence and lower fence of a boxplot are used to detect potential outliers whose values exceeding the tail of Q ± (1.5×Interquartile range). The identified outliers are neutralized using the Winsorize method whilst the Winsorize Gini index is then used to compute the divergences among probability distributions of the target predictor’s values until stopping criteria are met. This study uses three stopping rules: node achieved the minimum 10% of total training set

Stock Market Random Forest-Text Mining (SMRF-TM) Approach to Analyse Critical Indicators of Stock Market Movements

The Stock Market is a significant sector of a country’s economy and has a crucial role in the growth of commerce and industry. Hence, discovering efficient ways to analyse and visualise stock market data is considered a significant issue in modern finance. The use of data mining techniques to predict stock market movements has been extensively studied using historical market prices but such approaches are constrained to make assessments within the scope of existing information, and thus they are not able to model any random behaviour of the stock market or identify the causes behind events. One area of limited success in stock market prediction comes from textual data, which is a rich source of information. Analysing textual data related to the Stock Market may provide better understanding of random behaviours of the market. Text Mining combined with the Random Forest algorithm offers a novel approach to the study of critical indicators, which contribute to the prediction of stock market abnormal movements. In this thesis, a Stock Market Random Forest-Text Mining system (SMRF-TM) is developed and is used to mine the critical indicators related to the 2009 Dubai stock market debt standstill. Random forest and expectation maximisation are applied to classify the extracted features into a set of meaningful and semantic classes, thus extending current approaches from three to eight classes: critical down, down, neutral, up, critical up, economic, social and political. The study demonstrates that Random Forest has outperformed other classifiers and has achieved the best accuracy in classifying the bigram features extracted from the corpus