A brief history of learning classifier systems: from CS-1 to XCS and its variants

© 2015, Springer-Verlag Berlin Heidelberg. The direction set by Wilson’s XCS is that modern Learning Classifier Systems can be characterized by their use of rule accuracy as the utility metric for the search algorithm(s) discovering useful rules. Such searching typically takes place within the restricted space of co-active rules for efficiency. This paper gives an overview of the evolution of Learning Classifier Systems up to XCS, and then of some of the subsequent developments of Wilson’s algorithm to different types of learning

Feature selection by multi-objective optimization: application to network anomaly detection by hierarchical self-organizing maps.

Feature selection is an important and active issue in clustering and classification problems. By choosing an adequate feature subset, a dataset dimensionality reduction is allowed, thus contributing to decreasing the classification computational complexity, and to improving the classifier performance by avoiding redundant or irrelevant features. Although feature selection can be formally defined as an optimisation problem with only one objective, that is, the classification accuracy obtained by using the selected feature subset, in recent years, some multi-objective approaches to this problem have been proposed. These either select features that not only improve the classification accuracy, but also the generalisation capability in case of supervised classifiers, or counterbalance the bias toward lower or higher numbers of features that present some methods used to validate the clustering/classification in case of unsupervised classifiers. The main contribution of this paper is a multi-objective approach for feature selection and its application to an unsupervised clustering procedure based on Growing Hierarchical Self-Organizing Maps (GHSOM) that includes a new method for unit labelling and efficient determination of the winning unit. In the network anomaly detection problem here considered, this multi-objective approach makes it possible not only to differentiate between normal and anomalous traffic but also among different anomalies. The efficiency of our proposals has been evaluated by using the well-known DARPA/NSL-KDD datasets that contain extracted features and labeled attacks from around 2 million connections. The selected feature sets computed in our experiments provide detection rates up to 99.8% with normal traffic and up to 99.6% with anomalous traffic, as well as accuracy values up to 99.12%.This work has been funded by FEDER funds and the Ministerio de Ciencia e Innovación of the Spanish Government under Project No. TIN2012-32039

Niching in Particole Swarm Optimization

The Particle Swarm Optimization (PSO) algorithm, like many optimization algorithms, is designed to find a single optimal solution. When dealing with multimodal functions, it needs some modifications to be able to locate multiple optima. In a parallel with Evolutionary Computation algorithms, these modifications can be grouped in the framework of Niching. In this thesis, we present a new approach to niching in PSO that is based on clustering particles to identify niches. The neighborhood structure, on which particles rely for communication, is exploited together with the niche information to perform parallel searches to locate multiple optima. The clustering approach was implemented in the k-means based PSO (kPSO), which employs the standard k-means clustering algorithm. We follow the development of kPSO, starting from a first, simple implementation, and then introducing several improvements, such as a mechanism to adaptively identify the number of clusters. The final kPSO algorithm proves to be a competitive solution when compared with other existing algorithms, since it shows better performance on most multimodal functions in a commonly used benchmark set