Discovering Regression Rules with Ant Colony Optimization

The majority of Ant Colony Optimization (ACO) algorithms for data mining have dealt with classification or clustering problems. Regression remains an unexplored research area to the best of our knowledge. This paper proposes a new ACO algorithm that generates regression rules for data mining applications. The new algorithm combines components from an existing deterministic (greedy) separate and conquer algorithm—employing the same quality metrics and continuous attribute processing techniques—allowing a comparison of the two. The new algorithm has been shown to decrease the relative root mean square error when compared to the greedy algorithm. Additionally a different approach to handling continuous attributes was investigated showing further improvements were possible

Archive-Based Pheromone Model for Discovering Regression Rules with Ant Colony Optimization

In this paper we introduce a new algorithm, called Ant-Miner-Reg_MA, to tackle the regression problem using an archive-based pheromone model. Existing regression algorithms handle continuous attribute using a discretisation procedure, either in a preprocessing stage or during rule creation. Using an archive as a pheromone model, inspired by the ACO for Mixed-Variable (ACO_MV), we eliminate the need for a discretisation procedure. We compare the proposed Ant-Miner-Reg_MA against Ant-Miner-Reg, an ACO-based regression algorithm that uses a dynamic discretisation procedure, inspired on M5 algorithm, during rule construction process. Our results show that Ant-Miner-Reg_MA achieved a significant improvement in the relative root mean square error of the models created, overcoming the limitations of the dynamic discretisation procedure

A Comparative Study of Classification Rule Discovery with Ant Colony Optimization: AntMiner

Rule based classification is the fundamental and important task of data classification. To discover classification rules, ant colony optimization algorithms are successfully applied that follow a sequential covering approach to build a list of rules. AntMiner Rule Based Classification algorithms are inspired from self- organizing behaviour of ant colonies. In this paper, we presented a study on Ant Colony Optimization Algorithm, AntMiner, c_AntMiner, c_AntMiner2, c_AntMiner PB and  conducted experiments to find predictive accuracy against well-known rule induction algorithms JRIP and PART and results shows that AntMiner and its variants shows comparable as well as better performance in some datasets taken in the experimental study

New Archive-Based Ant Colony Optimization Algorithms for Learning Predictive Rules from Data

Data mining is the process of extracting knowledge and patterns from data. Classification and Regression are among the major data mining tasks, where the goal is to predict a value of an attribute of interest for each data instance, given the values of a set of predictive attributes. Most classification and regression problems involve continuous, ordinal and categorical attributes. Currently Ant Colony Optimization (ACO) algorithms have focused on directly handling categorical attributes only; continuous attributes are transformed using a discretisation procedure in either a preprocessing stage or dynamically during the rule creation. The use of a discretisation procedure has several limitations: (i) it increases the computational runtime, since several candidates values need to evaluated; (ii) requires access to the entire attribute domain, which in some applications all data is not available; (iii) the values used to create discrete intervals are not optimised in combination with the values of other attributes. This thesis investigates the use of solution archive pheromone model, based on Ant Colony Optimization for mixed-variable (ACOMV) algorithm, to directly cope with all attribute types. Firstly, an archive-based ACO classification algorithm is presented, followed by an automatic design framework to generate new configuration of ACO algorithms. Then, we addressed the challenging problem of mining data streams, presenting a new ACO algorithm in combination with a hybrid pheromone model. Finally, the archive-based approach is extended to cope with regression problems. All algorithms presented are compared against well-known algorithms from the literature using publicly available data sets. Our results have been shown to improve the computational time while maintaining a competitive predictive performance

Discovering Regression and Classification Rules with Monotonic Constraints Using Ant Colony Optimization

Data mining is a broad area that encompasses many different tasks from the supervised classification and regression tasks to unsupervised association rule mining and clustering. A first research thread in this thesis is the introduction of new Ant Colony Optimization (ACO)-based algorithms that tackle the regression task in data mining, exploring three different learning strategies: Iterative Rule Learning, Pittsburgh and Michigan strategies. The Iterative Rule Learning strategy constructs one rule at a time, where the best rule created by the ant colony is added to the rule list at each iteration, until a complete rule list is created. In the Michigan strategy, each ant constructs a single rule and from this collection of rules a niching algorithm combines the rules to create the final rule list. Finally, in the Pittsburgh strategy each ant constructs an entire rule list at each iteration, with the best list constructed by an ant in any iteration representing the final model. The most successful Pittsburgh-based Ant-Miner-Reg_PB algorithm, among the three variants, has been shown to be competitive against a well-known regression rule induction algorithm from the literature. The second research thread pursued involved incorporating existing domain knowledge to guide the construction of models as it is rare to find new domains that nothing is known about. One type of domain knowledge that occurs frequently in real world data-sets is monotonic constraints which capture increasing or decreasing trends within the data. In this thesis, monotonic constraints have been introduced into ACO-based rule induction algorithms for both classification and regression tasks. The enforcement of monotonic constraints has been implemented as a two step process. The first is a soft constraint preference in the model construction phase. This is followed by a hard constraint post-processing pruning suite to ensure the production of monotonic models. The new algorithms presented here have been shown to maintain and improve their predictive power when compared to non-monotonic rule induction algorithms