Evolutionary lazy learning for Naive Bayes classification

© 2016 IEEE. Most improvements for Naive Bayes (NB) have a common yet important flaw - these algorithms split the modeling of the classifier into two separate stages - the stage of preprocessing (e.g., feature selection and data expansion) and the stage of building the NB classifier. The first stage does not take the NB's objective function into consideration, so the performance of the classification cannot be guaranteed. Motivated by these facts and aiming to improve NB with accurate classification, we present a new learning algorithm called Evolutionary Local Instance Weighted Naive Bayes or ELWNB, to extend NB for classification. ELWNB combines local NB, instance weighted dataset extension and evolutionary algorithms seamlessly. Experiments on 20 UCI benchmark datasets demonstrate that ELWNB significantly outperforms NB and several other improved NB algorithms

The Challenges in SDN/ML Based Network Security : A Survey

Machine Learning is gaining popularity in the network security domain as many more network-enabled devices get connected, as malicious activities become stealthier, and as new technologies like Software Defined Networking (SDN) emerge. Sitting at the application layer and communicating with the control layer, machine learning based SDN security models exercise a huge influence on the routing/switching of the entire SDN. Compromising the models is consequently a very desirable goal. Previous surveys have been done on either adversarial machine learning or the general vulnerabilities of SDNs but not both. Through examination of the latest ML-based SDN security applications and a good look at ML/SDN specific vulnerabilities accompanied by common attack methods on ML, this paper serves as a unique survey, making a case for more secure development processes of ML-based SDN security applications.Comment: 8 pages. arXiv admin note: substantial text overlap with arXiv:1705.0056

Self-adaptive attribute weighting for Naive Bayes classification

©2014 Elsevier Ltd. All rights reserved. Naive Bayes (NB) is a popular machine learning tool for classification, due to its simplicity, high computational efficiency, and good classification accuracy, especially for high dimensional data such as texts. In reality, the pronounced advantage of NB is often challenged by the strong conditional independence assumption between attributes, which may deteriorate the classification performance. Accordingly, numerous efforts have been made to improve NB, by using approaches such as structure extension, attribute selection, attribute weighting, instance weighting, local learning and so on. In this paper, we propose a new Artificial Immune System (AIS) based self-adaptive attribute weighting method for Naive Bayes classification. The proposed method, namely AISWNB, uses immunity theory in Artificial Immune Systems to search optimal attribute weight values, where self-adjusted weight values will alleviate the conditional independence assumption and help calculate the conditional probability in an accurate way. One noticeable advantage of AISWNB is that the unique immune system based evolutionary computation process, including initialization, clone, section, and mutation, ensures that AISWNB can adjust itself to the data without explicit specification of functional or distributional forms of the underlying model. As a result, AISWNB can obtain good attribute weight values during the learning process. Experiments and comparisons on 36 machine learning benchmark data sets and six image classification data sets demonstrate that AISWNB significantly outperforms its peers in classification accuracy, class probability estimation, and class ranking performance

A comparative study of selected classification accuracy in user profiling

In recent years the used of personalization in service provisioning applications has been very popular. However, effective personalization cannot be achieved without accurate user profiles. A number of classification algorithms have been used to classify user related information to create accurate user profiles. In this study four different classification algorithms which are; naive Bayesian (NB), Bayesian Networks (BN), lazy learning of Bayesian rules (LBR) and instance-based learner (IB1) are compared using a set of user profile data. According to our simulation results NB and IB1 classifiers have the highest classification accuracy with the lowest error rate

Classification accuracy performance of Naïve Bayesian (NB), Bayesian Networks (BN), Lazy Learning of Bayesian Rules(LBR) and Instance-Based Learner (IB1) - comparative study

In recent years the used of personalization in service provisioning applications has been very popular. However, effective personalization cannot be achieved without accurate user profiles. A number of classification algorithms have been used to classify user related information to create accurate user profiles. In this study four different classification algorithms which are; naive Bayesian (NB), Bayesian networks (BN), lazy learning of Bayesian rules (LBR) and instance-based learner (IB1) are compared using a set of user profile data. According to our simulation results NB and IB1 classifiers have the highest classification accuracy with the lowest error rate. The obtained simulation results have been evaluated against the existing works of support vector machines (SVMs), decision trees (DTs) and neural networks (NNs)

Evolutionary Algorithms for Hyperparameter Search in Machine Learning

Machine learning algorithms usually have a number of hyperparameters. The choice of values for these hyperparameters may have a significant impact on the performance of an algorithm. In practice, for most learning algorithms the hyperparameter values are determined empirically, typically by search. From the research that has been done in this area, approaches for automating the search of hyperparameters mainly fall into the following categories: manual search, greedy search, random search, Bayesian model-based optimization, and evolutionary algorithm-based search. However, all these approaches have drawbacks — for example, manual and random search methods are undirected, greedy search is very inefficient, Bayesian model-based optimization is complicated and performs poorly with large numbers of hyperparameters, and classic evolutionary algorithm-based search can be very slow and risks falling into local optima. In this thesis we introduce three improved evolutionary algorithms applied to search for high-performing hyperparameter values for different learning algorithms. The first, named EWLNB, combines Naive Bayes and lazy instance-weighted learning. The second, EMLNB, extends this approach to multiple label classification. Finally, we further develop similar methods in an algorithm, named SEODP, for optimizing hyperparameters of deep networks, and report its usefulness on a real-world application of machine learning for philanthropy. EWLNB is a differential evolutionary algorithm which can automatically adapt to different datasets without human intervention by searching for the best hyperparameters for the models based on the characteristics of the datasets to which it is applied. To validate the EWLNB algorithm, we first use it to optimize two key parameters for a locally-weighted Naive Bayes model. Experimental evaluation of this approach on 56 of the benchmark UCI machine learning datasets demonstrate that EWLNB significantly outperforms Naive Bayes as well as several other improved versions of the Naive Bayes algorithms both in terms of classification accuracy and class probability estimation. We then extend the EWLNB approach in the form of the Evolutionary Multi-label Lazy Naive Bayes (EMLNB) algorithm to enable hyperparameter search for multi-label classification problems. Lastly, we revise the above algorithms to propose a method, SEODP, for optimizing deep learning (DL) architecture and hyperparameters. SEODP uses a semi-evolutionary and semi-random approach to search for hyperparameter values, which is designed to evolve a solution automatically over different datasets. SEODP is much faster than other methods, and can adaptively determine different deep network architectures automatically. Experimental results show that compared with manual search, SEODP is much more effective, and compared with grid search, SEODP can achieve optimal performance using only approximately 2% of the running time of greedy search. We also use SEODP on a real-world social-behavioral dataset from a charity organization for a philanthropy application. This dataset contains comprehensive real-time attributes on potential indicators for candidates to be donors. The results show that SEODP is a promising approach for optimizing deep network (DN) architectures over different types of datasets, including a real-world dataset. In summary, the results in this thesis indicate that our methods address the main drawback of evolutionary algorithms, which is the convergence time, and show experimentally that evolutionary-based algorithms can achieve good results in optimizing the hyperparameters for a range of different machine learning algorithms