Stock market prediction using machine learning classifiers and social media, news

Accurate stock market prediction is of great interest to investors; however, stock markets are driven by volatile factors such as microblogs and news that make it hard to predict stock market index based on merely the historical data. The enormous stock market volatility emphasizes the need to effectively assess the role of external factors in stock prediction. Stock markets can be predicted using machine learning algorithms on information contained in social media and financial news, as this data can change investors’ behavior. In this paper, we use algorithms on social media and financial news data to discover the impact of this data on stock market prediction accuracy for ten subsequent days. For improving performance and quality of predictions, feature selection and spam tweets reduction are performed on the data sets. Moreover, we perform experiments to find such stock markets that are difficult to predict and those that are more influenced by social media and financial news. We compare results of different algorithms to find a consistent classifier. Finally, for achieving maximum prediction accuracy, deep learning is used and some classifiers are ensembled. Our experimental results show that highest prediction accuracies of 80.53% and 75.16% are achieved using social media and financial news, respectively. We also show that New York and Red Hat stock markets are hard to predict, New York and IBM stocks are more influenced by social media, while London and Microsoft stocks by financial news. Random forest classifier is found to be consistent and highest accuracy of 83.22% is achieved by its ensemble

Bacterial-inspired feature selection algorithm and its application in fault diagnosis of complex structures

2016 IEEE Congress on Evolutionary Computation, CEC 2016, Vancouver, Canada, 24-29 July 2016Feature selection is an important preprocessing technique for data analysis and data mining. One of main challenge for feature selection is to overcome the curse of dimensionality. Bacterial algorithms, like Bacterial Foraging Optimization (BFO), have been well-exploited as the metaheuristics for addressing the optimization problems. In this paper, an extended bacterial algorithm named as Bacterial-Inspired Feature Selection Algorithm (BIFS) is proposed. In BIFS, the searching process of bacteria consists of two main mechanisms: interactive swimming (or running) strategy used in Bacterial Colony Optimization (BCO), and random tumbling strategy embedded in Bacterial Foraging Optimization (BFO). The rule controlled foraging mode in BCO has been used in BIFS to overcome the high computational cost problem in most BFOs. Meanwhile, the 'roulette wheel weighting' strategy is employed to weight the influence of features on the fitness functions and evaluate the distribution of the features within the large search space. Experiments on six benchmark datasets show that the proposed algorithm (i.e. BIFS) achieves higher classification accuracy rate in comparison to the four bacterial based algorithms and other three evolutionary algorithms. Furthermore, an additional real application of the proposed bacterial-inspired feature selection algorithm for fault diagnosis of complex structures in engineering has been developed. The results show that the proposed bacterial-inspired algorithm is capable of selecting the most sensitive sensors to detect and isolate the fault of complex structures.Department of Mechanical Engineerin