Stock market prediction using machine learning classifiers and social media, news

Accurate stock market prediction is of great interest to investors; however, stock markets are driven by volatile factors such as microblogs and news that make it hard to predict stock market index based on merely the historical data. The enormous stock market volatility emphasizes the need to effectively assess the role of external factors in stock prediction. Stock markets can be predicted using machine learning algorithms on information contained in social media and financial news, as this data can change investors’ behavior. In this paper, we use algorithms on social media and financial news data to discover the impact of this data on stock market prediction accuracy for ten subsequent days. For improving performance and quality of predictions, feature selection and spam tweets reduction are performed on the data sets. Moreover, we perform experiments to find such stock markets that are difficult to predict and those that are more influenced by social media and financial news. We compare results of different algorithms to find a consistent classifier. Finally, for achieving maximum prediction accuracy, deep learning is used and some classifiers are ensembled. Our experimental results show that highest prediction accuracies of 80.53% and 75.16% are achieved using social media and financial news, respectively. We also show that New York and Red Hat stock markets are hard to predict, New York and IBM stocks are more influenced by social media, while London and Microsoft stocks by financial news. Random forest classifier is found to be consistent and highest accuracy of 83.22% is achieved by its ensemble

Clustering in Recommendation Systems Using Swarm Intelligence

Ένα σύστημα συστάσεων είναι μία εφαρμογή που εκμεταλλεύεται πληροφορίες για να βοηθήσει τους χρήστες στη λήψη αποφάσεων προτείνοντας αντικείμενα που μπορεί να τους αρέσουν. Ένα σύστημα συστάσεων που βασίζεται στην τεχνική του συνεργατικού φιλτραρίσματος (collaborative filtering) δημιουργεί συστάσεις στους χρήστες με βάση τις προτιμήσεις παρόμοιων χρηστών. Ωστόσο, αυτός ο τύπος συστήματος συστάσεων δεν είναι τόσο αποτελεσματικός όταν τα δεδομένα αυξάνονται σε μεγάλο βαθμό (scalability) ή όταν δεν υπάρχει αρκετή πληροφορία (sparsity), καθώς δεν ομαδοποιούνται σωστά οι παρόμοιοι χρήστες. Αυτή η διπλωματική εργασία προτείνει τρείς υβριδικούς αλγορίθμους που ο καθένας συνδυάζει τον αλγόριθμο k-means με έναν αλγόριθμο ευφυΐας σμήνους για να βελτιώσει την ομαδοποίηση των χρηστών, και κατ’ επέκταση την ποιότητα των συστάσεων. Οι αλγόριθμοι ευφυΐας σμήνους που χρησιμοποιούνται είναι o αλγόριθμος τεχνητής κοινωνίας μελισσών (artificial bee colony), ο αλγόριθμος βελτιστοποίησης αναζήτησης κούκων (cuckoo search optimization) και ο αλγόριθμος βελτιστοποίησης γκρίζων λύκων (grey-wolf optimization). Οι προτεινόμενες μέθοδοι αξιολογήθηκαν χρησιμοποιώντας ένα σύνολο δεδομένων του MovieLens. Η αξιολόγηση δείχνει πως τα προτεινόμενα συστήματα συστάσεων αποδίδουν καλύτερα σε σύγκριση με τις ήδη υπάρχουσες τεχνικές όσον αφορά τις μετρικές του μέσου απόλυτου σφάλματος (mean absolute error - MAE), της ακρίβειας (precision), του αθροίσματος των τετραγωνικών σφαλμάτων (sum of squared errors - SSE) και της ανάκλησης (recall). Επιπλέον, τα αποτελέσματα της αξιολόγησης δείχνουν πως ο υβριδικός αλγόριθμος που χρησιμοποιεί την μέθοδο της τεχνητής κοινωνίας μελισσών αποδίδει ελαφρώς καλύτερα από τους άλλους δύο προτεινόμενους αλγορίθμους.A recommender system (RS) is an application that exploits information to help users in decision making by suggesting items they might like. A collaborative recommender system generates recommendations to users based on their similar neighbor’s preferences. However, this type of recommender system faces the data sparsity and scalability problems making the neighborhood selection a challenging task. This thesis proposes three hybrid collaborative recommender systems that each one combines the k-means algorithm with a different bio-inspired technique to enhance the clustering task, and therefore to improve the recommendation quality. The used bio-inspired techniques are artificial bee colony (ABC), cuckoo search optimization (CSO), and grey-wolf optimizer (GWO). The proposed approaches were evaluated over a MovieLens dataset. The evaluation shows that the proposed recommender systems perform better compared to already existing techniques in terms of mean absolute error (MAE), precision, sum of squared errors (SSE), and recall. Moreover, the experimental results indicate that the hybrid recommender system that uses the ABC method performs slightly better than the other two proposed hybrid algorithms

Applied (Meta)-Heuristic in Intelligent Systems

Engineering and business problems are becoming increasingly difficult to solve due to the new economics triggered by big data, artificial intelligence, and the internet of things. Exact algorithms and heuristics are insufficient for solving such large and unstructured problems; instead, metaheuristic algorithms have emerged as the prevailing methods. A generic metaheuristic framework guides the course of search trajectories beyond local optimality, thus overcoming the limitations of traditional computation methods. The application of modern metaheuristics ranges from unmanned aerial and ground surface vehicles, unmanned factories, resource-constrained production, and humanoids to green logistics, renewable energy, circular economy, agricultural technology, environmental protection, finance technology, and the entertainment industry. This Special Issue presents high-quality papers proposing modern metaheuristics in intelligent systems

Living analytics methods for the social web

[no abstract

Security and Privacy for Modern Wireless Communication Systems

The aim of this reprint focuses on the latest protocol research, software/hardware development and implementation, and system architecture design in addressing emerging security and privacy issues for modern wireless communication networks. Relevant topics include, but are not limited to, the following: deep-learning-based security and privacy design; covert communications; information-theoretical foundations for advanced security and privacy techniques; lightweight cryptography for power constrained networks; physical layer key generation; prototypes and testbeds for security and privacy solutions; encryption and decryption algorithm for low-latency constrained networks; security protocols for modern wireless communication networks; network intrusion detection; physical layer design with security consideration; anonymity in data transmission; vulnerabilities in security and privacy in modern wireless communication networks; challenges of security and privacy in node–edge–cloud computation; security and privacy design for low-power wide-area IoT networks; security and privacy design for vehicle networks; security and privacy design for underwater communications networks

A Multi-Transformation Evolutionary Framework for Influence Maximization in Social Networks

Influence maximization is a crucial issue for mining the deep information of social networks, which aims to select a seed set from the network to maximize the number of influenced nodes. To evaluate the influence spread of a seed set efficiently, existing studies have proposed transformations with lower computational costs to replace the expensive Monte Carlo simulation process. These alternate transformations, based on network prior knowledge, induce different search behaviors with similar characteristics to various perspectives. Specifically, it is difficult for users to determine a suitable transformation a priori. This article proposes a multi-transformation evolutionary framework for influence maximization (MTEFIM) with convergence guarantees to exploit the potential similarities and unique advantages of alternate transformations and to avoid users manually determining the most suitable one. In MTEFIM, multiple transformations are optimized simultaneously as multiple tasks. Each transformation is assigned an evolutionary solver. Three major components of MTEFIM are conducted via: 1) estimating the potential relationship across transformations based on the degree of overlap across individuals of different populations, 2) transferring individuals across populations adaptively according to the inter-transformation relationship, and 3) selecting the final output seed set containing all the transformation's knowledge. The effectiveness of MTEFIM is validated on both benchmarks and real-world social networks. The experimental results show that MTEFIM can efficiently utilize the potentially transferable knowledge across multiple transformations to achieve highly competitive performance compared to several popular IM-specific methods. The implementation of MTEFIM can be accessed at https://github.com/xiaofangxd/MTEFIM.Comment: This work has been submitted to the IEEE Computational Intelligence Magazine for publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

Advances in Meta-Heuristic Optimization Algorithms in Big Data Text Clustering

This paper presents a comprehensive survey of the meta-heuristic optimization algorithms on the text clustering applications and highlights its main procedures. These Artificial Intelligence (AI) algorithms are recognized as promising swarm intelligence methods due to their successful ability to solve machine learning problems, especially text clustering problems. This paper reviews all of the relevant literature on meta-heuristic-based text clustering applications, including many variants, such as basic, modified, hybridized, and multi-objective methods. As well, the main procedures of text clustering and critical discussions are given. Hence, this review reports its advantages and disadvantages and recommends potential future research paths. The main keywords that have been considered in this paper are text, clustering, meta-heuristic, optimization, and algorithm

An enhanced binary bat and Markov clustering algorithms to improve event detection for heterogeneous news text documents

Event Detection (ED) works on identifying events from various types of data. Building an ED model for news text documents greatly helps decision-makers in various disciplines in improving their strategies. However, identifying and summarizing events from such data is a non-trivial task due to the large volume of published heterogeneous news text documents. Such documents create a high-dimensional feature space that influences the overall performance of the baseline methods in ED model. To address such a problem, this research presents an enhanced ED model that includes improved methods for the crucial phases of the ED model such as Feature Selection (FS), ED, and summarization. This work focuses on the FS problem by automatically detecting events through a novel wrapper FS method based on Adapted Binary Bat Algorithm (ABBA) and Adapted Markov Clustering Algorithm (AMCL), termed ABBA-AMCL. These adaptive techniques were developed to overcome the premature convergence in BBA and fast convergence rate in MCL. Furthermore, this study proposes four summarizing methods to generate informative summaries. The enhanced ED model was tested on 10 benchmark datasets and 2 Facebook news datasets. The effectiveness of ABBA-AMCL was compared to 8 FS methods based on meta-heuristic algorithms and 6 graph-based ED methods. The empirical and statistical results proved that ABBAAMCL surpassed other methods on most datasets. The key representative features demonstrated that ABBA-AMCL method successfully detects real-world events from Facebook news datasets with 0.96 Precision and 1 Recall for dataset 11, while for dataset 12, the Precision is 1 and Recall is 0.76. To conclude, the novel ABBA-AMCL presented in this research has successfully bridged the research gap and resolved the curse of high dimensionality feature space for heterogeneous news text documents. Hence, the enhanced ED model can organize news documents into distinct events and provide policymakers with valuable information for decision making

Followee recommendation in twitter using fuzzy link prediction.

In social networking sites, it is useful to receive recommendations about whom to contact or follow. These recommendations not only allow to establish connections with people one might already know in real life, but also with people or users that have similar interests or are potentially interesting. We propose an approach that tackles contact (followee) recommendation in Twitter by means of fuzzy logic. This fuzzy approach handles recommendation as a link prediction problem and uses three types of similarity between a pair of users: tweet similarity, followee id similarity, and followee tweet similarity. These similarities are calculated by extracting user profiles. These profiles are, in turn, obtained by considering Twitter as a heterogeneous information network. To test our approach, we crawled a repository of 6,000 users and 2 million tweets, and we measured accuracy by comparing our results with the actual followee lists of the users. These results, which are also compared against the results given by state-of-the-art methods, show a high accuracy. Other advantages of the fuzzy system include a self-explanatory capability and the ability to produce a non-binary friendship value