A Hybrid of Ant Colony Optimization Algorithm and Simulated Annealing for Classification Rules

Ant colony optimization (ACO) is a metaheuristic approach inspired from the behaviour of natural ants and can be used to solve a variety of combinatorial optimization problems. Classification rule induction is one of the problems solved by the Ant-miner algorithm, a variant of ACO, which was initiated by Parpinelli in 2001. Previous studies have shown that ACO is a promising machine learning technique to generate classification rules. However, the Ant-miner is less class focused since the rule’s class is assigned after the rule was constructed. There is also the case where the Ant-miner cannot find any optimal solution for some data sets. Thus, this thesis proposed two variants of hybrid ACO with simulated annealing (SA) algorithm for solving problem of classification rule induction. In the first proposed algorithm, SA is used to optimize the rule's discovery activity by an ant. Benchmark data sets from various fields were used to test the proposed algorithms. Experimental results obtained from this proposed algorithm are comparable to the results of the Ant-miner and other well-known rule induction algorithms in terms of rule accuracy, but are better in terms of rule simplicity. The second proposed algorithm uses SA to optimize the terms selection while constructing a rule. The algorithm fixes the class before rule's construction. Since the algorithm fixed the class before each rule's construction, a much simpler heuristic and fitness function is proposed. Experimental results obtained from the proposed algorithm are much higher than other compared algorithms, in terms of predictive accuracy. The successful work on hybridization of ACO and SA algorithms has led to the improved learning ability of ACO for classification. Thus, a higher predictive power classification model for various fields could be generated

The behavior of simulated annealing in stochastic optimization

In this thesis we examine the performance of simulated annealing (SA) on various response surfaces. The main goals of the study are to evaluate the effectiveness of SA for stochastic optimization, develop modifications to SA in an attempt to improve its performance, and to evaluate whether artificially adding noise to a deterministic response surface might improve the performance of SA. SA is applied to several different response surfaces with different levels of complexity. We first experiment with two basic approaches of computing the performance measure for stochastic surfaces, constant sample size and variable sample size. We found that the constant sample size performed best. At the same time we also show that artificially adding noise may improve the performance of SA on more complex deterministic response surfaces. We develop a hybrid version of SA in which the genetic algorithm is embedded within SA. The effectiveness of the hybrid approach is not conclusive and needs further investigation. Finally, we conclude with a brief discussion on the strengths and weaknesses of the proposed method and an outline of future directions