Evolutionary Strategies for Data Mining

Learning classifier systems (LCS) have been successful in generating rules for solving classification problems in data mining. The rules are of the form IF condition THEN action. The condition encodes the features of the input space and the action encodes the class label. What is lacking in those systems is the ability to express each feature using a function that is appropriate for that feature. The genetic algorithm is capable of doing this but cannot because only one type of membership function is provided. Thus, the genetic algorithm learns only the shape and placement of the membership function, and in some cases, the number of partitions generated by this function. The research conducted in this study employs a learning classifier system to generate the rules for solving classification problems, but also incorporates multiple types of membership functions, allowing the genetic algorithm to choose an appropriate one for each feature of the input space and determine the number of partitions generated by each function. In addition, three membership functions were introduced. This paper describes the framework and implementation of this modified learning classifier system (M-LCS). Using the M-LCS model, classifiers were simulated for two benchmark classification problems and two additional real-world problems. The results of these four simulations indicate that the M-LCS model provides an alternative approach to designing a learning classifier system. The following contributions are made to the field of computing: 1) a framework for developing a learning classifier system that employs multiple types of membership functions, 2) a model, M-LCS, that was developed from the framework, and 3) the addition of three membership functions that have not been used in the design of learning classifier systems

Evolving neuro-fuzzy tools for system classification and prediction

"Classification and prediction algorithims have recently become very powerful tools to a wide array of real-world applications. Some real world applications include system condition monitoring, bioinformatics, robotics, predictive control, earthquake prediction, weather forecasting, stock market and traffic pattern prediction, just to name a few. Within this work, several novel approaches, as well as modifications to some existing approaches, are introduced in order to improve the performance of current classification and prediction paradigms. In the first section of this work, a novel weighted recurrent neuro-fuzzy inference system is introduced alongside two existing neural networks. It is found that the novel design outperforms both the existing neural networks in terms of equal-step and sequential-step inputs for time-series forecasting. The second contribution of this work is the development of a novel evolving clustering algorithim for classification and prediction. This particular algorithim starts without any priori knowledge of the distribution of the data set. The novel design is capable of revealing the true cluster configuration in a single pass of the data, estimating the location and variance of each cluster. After a rigorous performance evaluation, it is found that the novel design outperforms many existing clustering approaches including the well-known potential-based evolving Takagi-Sugeno (eTS) clustering scheme. The third and fourth contributions of this work are the development of a second novel clustering technique and a novel hybrid training technique. The clustering technique is a combination of the aforementioned scheme and the potential-based technique. The new training algorithm is a combination of the decoupled-extended Kalman filter (for the backward pass) and the recursive least-sequares estimate (for the forward pass). It is found that the novel clustering technique outperforms many available clustering techniques. Also, the novel training algorithm is proven to outperform most existing training techniques."--Abstrac

Doctor of Philosophy

dissertationThe goal of machine learning is to develop efficient algorithms that use training data to create models that generalize well to unseen data. Learning algorithms can use labeled data, unlabeled data or both. Supervised learning algorithms learn a model using labeled data only. Unsupervised learning methods learn the internal structure of a dataset using only unlabeled data. Lastly, semisupervised learning is the task of finding a model using both labeled and unlabeled data. In this research work, we contribute to both supervised and semisupervised learning. We contribute to supervised learning by proposing an efficient high-dimensional space coverage scheme which is based on the disjunctive normal form. We use conjunctions of a set of half-spaces to create a set of convex polytopes. Disjunction of these polytopes can provide desirable coverage of space. Unlike traditional methods based on neural networks, we do not initialize the model parameters randomly. As a result, our model minimizes the risk of poor local minima and higher learning rates can be used which leads to faster convergence. We contribute to semisupervised learning by proposing 2 unsupervised loss functions that form the basis of a novel semisupervised learning method. The first loss function is called Mutual-Exclusivity. The motivation of this method is the observation that an optimal decision boundary lies between the manifolds of different classes where there are no or very few samples. Decision boundaries can be pushed away from training samples by maximizing their margin and it is not necessary to know the class labels of the samples to maximize the margin. The second loss is named Transformation/Stability and is based on the fact that the prediction of a classifier for a data sample should not change with respect to transformations and perturbations applied to that data sample. In addition, internal variations of a learning system should have little to no effect on the output. The proposed loss minimizes the variation in the prediction of the network for a specific data sample. We also show that the same technique can be used to improve the robustness of a learning model with respect to adversarial examples

DESIGN AND IMPLEMENTATION OF INTELLIGENT MONITORING SYSTEMS FOR THERMAL POWER PLANT BOILER TRIPS

Steam boilers represent the main equipment in the power plant. Some boiler trips may lead to an entire shutdown of the plant, which is economically burdensome. An early detection and diagnosis of the boiler trips is crucial to maintain normal and safe operational conditions of the plant. Numbers of methodologies have been proposed in the literature for fault diagnosis of power plants. However, rapid deployment of these methodologies is difficult to be achieved due to certain inherent limitations such as system inability to learn or a dynamically improve the system performance and the brittleness of the system beyond its domain of expertise. As a potential solution to these problems, two artificial intelligent monitoring systems specialized in boiler trips have been proposed and coded within the MA TLAB environment in the present work. The training and validation of the two systems have been performed using real operational data which was captured from the plant integrated acquisition system of JANAMANJUNG coal-fired power plant. An integrated plant data preparation framework for seven boiler trips with related operational variables, has been proposed for the training and validation of the proposed artificial intelligent systems. The feedforward neural network methodology has been adopted as a major computational intelligent tool in both systems. The root mean square error has been widely used as a performance indicator of the proposed systems. The first intelligent monitoring system represents the use of the pure artificial neural network system for boiler trip detection. The final architecture for this system has been explored after investigation of various main neural network topology combinations which include one and two hidden layers, one to ten neurons for each hidden layer, three types of activation function, and four types of multidimensional minimization training algorithms. It has been found that there was no general neural network topology combination that can be applied for all boiler trips. All seven boiler trips under consideration had been detected by the proposed systems before or at the same time as the plant control system. The second intelligent monitoring system represents mergmg of genetic algorithms and artificial neural networks as a hybrid intelligent system. For this hybrid intelligent system, the selection of appropriate variables from hundreds of boiler operation variables with optimal neural network topology combinations to monitor boiler trips was a major concern. The encoding and optimization process using genetic algorithms has been applied successfully. A slightly lower root mean square error was observed in the second system which reveals that the hybrid intelligent system performed better than the pure neural network system. Also, the optimal selection of the most influencing variables was performed successfully by the hybrid intelligent system. The proposed artificial intelligent systems could be adopted on-line as a reliable controller of the thermal power plant boiler