Evolutionary Strategies for Data Mining

Learning classifier systems (LCS) have been successful in generating rules for solving classification problems in data mining. The rules are of the form IF condition THEN action. The condition encodes the features of the input space and the action encodes the class label. What is lacking in those systems is the ability to express each feature using a function that is appropriate for that feature. The genetic algorithm is capable of doing this but cannot because only one type of membership function is provided. Thus, the genetic algorithm learns only the shape and placement of the membership function, and in some cases, the number of partitions generated by this function. The research conducted in this study employs a learning classifier system to generate the rules for solving classification problems, but also incorporates multiple types of membership functions, allowing the genetic algorithm to choose an appropriate one for each feature of the input space and determine the number of partitions generated by each function. In addition, three membership functions were introduced. This paper describes the framework and implementation of this modified learning classifier system (M-LCS). Using the M-LCS model, classifiers were simulated for two benchmark classification problems and two additional real-world problems. The results of these four simulations indicate that the M-LCS model provides an alternative approach to designing a learning classifier system. The following contributions are made to the field of computing: 1) a framework for developing a learning classifier system that employs multiple types of membership functions, 2) a model, M-LCS, that was developed from the framework, and 3) the addition of three membership functions that have not been used in the design of learning classifier systems

MILCS: A mutual information learning classifier system

This paper introduces a new variety of learning classifier system (LCS), called MILCS, which utilizes mutual information as fitness feedback. Unlike most LCSs, MILCS is specifically designed for supervised learning. MILCS's design draws on an analogy to the structural learning approach of cascade correlation networks. We present preliminary results, and contrast them to results from XCS. We discuss the explanatory power of the resulting rule sets, and introduce a new technique for visualizing explanatory power. Final comments include future directions for this research, including investigations in neural networks and other systems. Copyright 2007 ACM

An architectural framework for self-configuration and self-improvement at runtime

[no abstract

学習戦略に基づく学習分類子システムの設計

On Learning Classifier Systems dubbed LCSs a leaning strategy which defines how LCSs cover a state-action space in a problem can be one of the most fundamental options in designing LCSs. There lacks an intensive study of the learning strategy to understand whether and how the learning strategy affects the performance of LCSs. This lack has resulted in the current design methodology of LCS which does not carefully consider the types of learning strategy. The thesis clarifies a need of a design methodology of LCS based on the learning strategy. That is, the thesis shows the learning strategy can be an option that determines the potential performance of LCSs and then claims that LCSs should be designed on the basis of the learning strategy in order to improve the performance of LCSs. First, the thesis empirically claims that the current design methodology of LCS, without the consideration of learning strategy, can be limited to design a proper LCS to solve a problem. This supports the need of design methodology based on the learning strategy. Next, the thesis presents an example of how LCS can be designed on the basis of the learning strategy. The thesis empirically show an adequate learning strategy improving the performance of LCS can be decided depending on a type of problem difficulties such as missing attributes. Then, the thesis draws an inclusive guideline that explains which learning strategy should be used to address which types of problem difficulties. Finally, the thesis further shows, on an application of LCS for a human daily activity recognition problem, the adequate learning strategy according to the guideline effectively improves the performance of the application. The thesis concludes that the learning strategy is the option of the LCS design which determines the potential performance of LCSs. Thus, before designing any type of LCSs including their applications, the learning strategy should be adequately selected at first, because their performance degrades when they employ an inadequate learning strategy to a problem they want to solve. In other words, LCSs should be designed on the basis of the adequate learning strategy.電気通信大学201

Hyper-heuristic decision tree induction

A hyper-heuristic is any algorithm that searches or operates in the space of heuristics as opposed to the space of solutions. Hyper-heuristics are increasingly used in function and combinatorial optimization. Rather than attempt to solve a problem using a fixed heuristic, a hyper-heuristic approach attempts to find a combination of heuristics that solve a problem (and in turn may be directly suitable for a class of problem instances). Hyper-heuristics have been little explored in data mining. This work presents novel hyper-heuristic approaches to data mining, by searching a space of attribute selection criteria for decision tree building algorithm. The search is conducted by a genetic algorithm. The result of the hyper-heuristic search in this case is a strategy for selecting attributes while building decision trees. Most hyper-heuristics work by trying to adapt the heuristic to the state of the problem being solved. Our hyper-heuristic is no different. It employs a strategy for adapting the heuristic used to build decision tree nodes according to some set of features of the training set it is working on. We introduce, explore and evaluate five different ways in which this problem state can be represented for a hyper-heuristic that operates within a decisiontree building algorithm. In each case, the hyper-heuristic is guided by a rule set that tries to map features of the data set to be split by the decision tree building algorithm to a heuristic to be used for splitting the same data set. We also explore and evaluate three different sets of low-level heuristics that could be employed by such a hyper-heuristic. This work also makes a distinction between specialist hyper-heuristics and generalist hyper-heuristics. The main difference between these two hyperheuristcs is the number of training sets used by the hyper-heuristic genetic algorithm. Specialist hyper-heuristics are created using a single data set from a particular domain for evolving the hyper-heurisic rule set. Such algorithms are expected to outperform standard algorithms on the kind of data set used by the hyper-heuristic genetic algorithm. Generalist hyper-heuristics are trained on multiple data sets from different domains and are expected to deliver a robust and competitive performance over these data sets when compared to standard algorithms. We evaluate both approaches for each kind of hyper-heuristic presented in this thesis. We use both real data sets as well as synthetic data sets. Our results suggest that none of the hyper-heuristics presented in this work are suited for specialization – in most cases, the hyper-heuristic’s performance on the data set it was specialized for was not significantly better than that of the best performing standard algorithm. On the other hand, the generalist hyper-heuristics delivered results that were very competitive to the best standard methods. In some cases we even achieved a significantly better overall performance than all of the standard methods

Learning classifier systems from first principles: A probabilistic reformulation of learning classifier systems from the perspective of machine learning

Learning Classifier Systems (LCS) are a family of rule-based machine learning methods. They aim at the autonomous production of potentially human readable results that are the most compact generalised representation whilst also maintaining high predictive accuracy, with a wide range of application areas, such as autonomous robotics, economics, and multi-agent systems. Their design is mainly approached heuristically and, even though their performance is competitive in regression and classification tasks, they do not meet their expected performance in sequential decision tasks despite being initially designed for such tasks. It is out contention that improvement is hindered by a lack of theoretical understanding of their underlying mechanisms and dynamics.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Autocatalytic closure and the evolution of cellular information processing networks

Cellular Information Processing Networks (CIPNs) are chemical networks of interacting molecules occurring in living cells. Through complex molecular interactions, CIPNs are able to coordinate critical cellular activities in response to internal and external stimuli. We hypothesise that CIPNs may be abstractly regarded as subsets of collectively autocatalytic (i.e., organisationally closed) reaction networks. These closure properties would subsequently interact with the evolution and adaptation of CIPNs capable of distinct information processing abilities. This hypothesis is motivated by the fact that CIPNs may require a mechanism enabling the self-maintenance of core components of the network when subjected to internal and external perturbations and during cellular divisions. Indeed, partially replicated or defective CIPNs may lead to the malfunctioning and premature death of the cell. In this thesis, we evaluate different existing computational approaches to model and evolve chemical reaction networks in silico. Following this literature review, we propose an evolutionary simulation platform capable of evolving artificial CIPNs from a bottom-up perspective. This system is a novel agent-based Artificial Chemistry (AC) which employs a term rewriting system called the Molecular Classifier System (MCS.bl). The latter is derived from the Holland broadcast language formalism. Our first series of experiments focuses on the emergence and evolution of selfmaintaining molecular organisations in the MCS.bl. Such experiments naturally relate to similar studies conducted in ACs such as Tierra, Alchemy and α-universes. Our results demonstrate some counter-intuitive outcomes, not indicated in previous literature. We examine each of these “unexpected” evolutionary dynamics (including an elongation catastrophe phenomenon) which presented various degenerate evolutionary trajectories. To address these robustness and evolvability issues, we evaluate several model variants of the MCS.bl. This investigation illuminates the key properties required to allow the self-maintenance and stable evolution of closed reaction networks in ACs. We demonstrate how the elongation catastrophe phenomenon can be prevented using a multi-level selectional model of the MCS.bl (which acts both at the molecular and cellular level). Using this multi-level selectional MCS.bl which was implemented as a parallel system, we successfully evolve an artificial CIPN to perform a simple pre-specified information processing task. We also demonstrate how signalling crosstalk may enable the cooperation of distinct closed CIPNs when mixed together in the same reaction space. We finally present the evolution of closed crosstalking and multitasking CIPNs exhibiting a higher level of complexity