Investigating Evaluation Measures in Ant Colony Algorithms for Learning Decision Tree Classifiers

Ant-Tree-Miner is a decision tree induction algorithm that is based on the Ant Colony Optimization (ACO) meta- heuristic. Ant-Tree-Miner-M is a recently introduced extension of Ant-Tree-Miner that learns multi-tree classification models. A multi-tree model consists of multiple decision trees, one for each class value, where each class-based decision tree is responsible for discriminating between its class value and all other values present in the class domain (one vs. all). In this paper, we investigate the use of 10 different classification quality evaluation measures in Ant-Tree-Miner-M, which are used for both candidate model evaluation and model pruning. Our experimental results, using 40 popular benchmark datasets, identify several quality functions that substantially improve on the simple Accuracy quality function that was previously used in Ant-Tree-Miner-M

Optimal Sparse Decision Trees

Decision tree algorithms have been among the most popular algorithms for interpretable (transparent) machine learning since the early 1980's. The problem that has plagued decision tree algorithms since their inception is their lack of optimality, or lack of guarantees of closeness to optimality: decision tree algorithms are often greedy or myopic, and sometimes produce unquestionably suboptimal models. Hardness of decision tree optimization is both a theoretical and practical obstacle, and even careful mathematical programming approaches have not been able to solve these problems efficiently. This work introduces the first practical algorithm for optimal decision trees for binary variables. The algorithm is a co-design of analytical bounds that reduce the search space and modern systems techniques, including data structures and a custom bit-vector library. Our experiments highlight advantages in scalability, speed, and proof of optimality.Comment: 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canad

The decision tree approach to classification

A class of multistage decision tree classifiers is proposed and studied relative to the classification of multispectral remotely sensed data. The decision tree classifiers are shown to have the potential for improving both the classification accuracy and the computation efficiency. Dimensionality in pattern recognition is discussed and two theorems on the lower bound of logic computation for multiclass classification are derived. The automatic or optimization approach is emphasized. Experimental results on real data are reported, which clearly demonstrate the usefulness of decision tree classifiers

Learning Multi-Tree Classification Models with Ant Colony Optimization

Ant Colony Optimization (ACO) is a meta-heuristic for solving combinatorial optimization problems, inspired by the behaviour of biological ant colonies. One of the successful applications of ACO is learning classification models (classifiers). A classifier encodes the relationships between the input attribute values and the values of a class attribute in a given set of labelled cases and it can be used to predict the class value of new unlabelled cases. Decision trees have been widely used as a type of classification model that represent comprehensible knowledge to the user. In this paper, we propose the use of ACO-based algorithms for learning an extended multi-tree classification model, which consists of multiple decision trees, one for each class value. Each class-based decision trees is responsible for discriminating between its class value and all other values available in the class domain. Our proposed algorithms are empirically evaluated against well-known decision trees induction algorithms, as well as the ACO-based Ant-Tree-Miner algorithm. The results show an overall improvement in predictive accuracy over 32 benchmark datasets. We also discuss how the new multi-tree models can provide the user with more understanding and knowledge-interpretability in a given domain

Differential Evolution for Many-Particle Adaptive Quantum Metrology

We devise powerful algorithms based on differential evolution for adaptive many-particle quantum metrology. Our new approach delivers adaptive quantum metrology policies for feedback control that are orders-of-magnitude more efficient and surpass the few-dozen-particle limitation arising in methods based on particle-swarm optimization. We apply our method to the binary-decision-tree model for quantum-enhanced phase estimation as well as to a new problem: a decision tree for adaptive estimation of the unknown bias of a quantum coin in a quantum walk and show how this latter case can be realized experimentally.Comment: Fig. 2(a) is the cover of Physical Review Letters Vol. 110 Issue 2

Implementasi Algoritma Ant Tree Miner Untuk Klasifikasi Jenis Fauna

Classification is a field of data mining that has many methods, one of them is decision tree. Decision tree is proven to be able to classify many kinds of data such as image data and time series data. However, there are several obstacles that are often encountered in the decision tree method. Running time required for the execution of this algorithm is quite long, so this study proposed to use the ant tree miner algorithm which is a development algorithm from the C4.5 decision tree. Ant tree miner works by utilizing ant colony optimization in the process of building its tree structure. Use ant colony optimization expected can optimize the tree that will be formed. From the testing that have been carried out, an accuracy of about 95% is obtained in the process of classifying Zoo dataset with the number of ants between 60 - 90

Robust optimization with incremental recourse

In this paper, we consider an adaptive approach to address optimization problems with uncertain cost parameters. Here, the decision maker selects an initial decision, observes the realization of the uncertain cost parameters, and then is permitted to modify the initial decision. We treat the uncertainty using the framework of robust optimization in which uncertain parameters lie within a given set. The decision maker optimizes so as to develop the best cost guarantee in terms of the worst-case analysis. The recourse decision is ``incremental"; that is, the decision maker is permitted to change the initial solution by a small fixed amount. We refer to the resulting problem as the robust incremental problem. We study robust incremental variants of several optimization problems. We show that the robust incremental counterpart of a linear program is itself a linear program if the uncertainty set is polyhedral. Hence, it is solvable in polynomial time. We establish the NP-hardness for robust incremental linear programming for the case of a discrete uncertainty set. We show that the robust incremental shortest path problem is NP-complete when costs are chosen from a polyhedral uncertainty set, even in the case that only one new arc may be added to the initial path. We also address the complexity of several special cases of the robust incremental shortest path problem and the robust incremental minimum spanning tree problem

Evolutionary Algorithms in Decision Tree Induction

One of the biggest problem that many data analysis techniques have to deal with nowadays is Combinatorial Optimization that, in the past, has led many methods to be taken apart. Actually, the (still not enough!) higher computing power available makes it possible to apply such techniques within certain bounds. Since other research fields like Artificial Intelligence have been (and still are) dealing with such problems, their contribute to statistics has been very significant. This chapter tries to cast the Combinatorial Optimization methods into the Artificial Intelligence framework, particularly with respect Decision Tree Induction, which is considered a powerful instrument for the knowledge extraction and the decision making support. When the exhaustive enumeration and evaluation of all the possible candidate solution to a Tree-based Induction problem is not computationally affordable, the use of Nature Inspired Optimization Algorithms, which have been proven to be powerful instruments for attacking many combinatorial optimization problems, can be of great help. In this respect, the attention is focused on three main problems involving Decision Tree Induction by mainly focusing the attention on the Classification and Regression Tree-CART (Breiman et al., 1984) algorithm. First, the problem of splitting complex predictors such a multi-attribute ones is faced through the use of Genetic Algorithms. In addition, the possibility of growing “optimal” exploratory trees is also investigated by making use of Ant Colony Optimization (ACO) algorithm. Finally, the derivation of a subset of decision trees for modelling multi-attribute response on the basis of a data-driven heuristic is also described. The proposed approaches might be useful for knowledge extraction from large databases as well as for data mining applications. The solution they offer for complicated data modelling and data analysis problems might be considered for a possible implementation in a Decision Support System (DSS). The remainder of the chapter is as follows. Section 2 describes the main features and the recent developments of Decision Tree Induction. An overview of Combinatorial Optimization with a particular focus on Genetic Algorithms and Ant Colony Optimization is presented in section 3. The use of these two algorithms within the Decision Tree Induction Framework is described in section 4, together with the description of the algorithm for modelling multi-attribute response. Section 5 summarizes the results of the proposed method on real and simulated datasets. Concluding remarks are presented in section 6. The chapter also includes an appendix that presents J-Fast, a Java-based software for Decision Tree that currently implements Genetic Algorithms and Ant Colony Optimization