Ensemble of Example-Dependent Cost-Sensitive Decision Trees

Several real-world classification problems are example-dependent cost-sensitive in nature, where the costs due to misclassification vary between examples and not only within classes. However, standard classification methods do not take these costs into account, and assume a constant cost of misclassification errors. In previous works, some methods that take into account the financial costs into the training of different algorithms have been proposed, with the example-dependent cost-sensitive decision tree algorithm being the one that gives the highest savings. In this paper we propose a new framework of ensembles of example-dependent cost-sensitive decision-trees. The framework consists in creating different example-dependent cost-sensitive decision trees on random subsamples of the training set, and then combining them using three different combination approaches. Moreover, we propose two new cost-sensitive combination approaches; cost-sensitive weighted voting and cost-sensitive stacking, the latter being based on the cost-sensitive logistic regression method. Finally, using five different databases, from four real-world applications: credit card fraud detection, churn modeling, credit scoring and direct marketing, we evaluate the proposed method against state-of-the-art example-dependent cost-sensitive techniques, namely, cost-proportionate sampling, Bayes minimum risk and cost-sensitive decision trees. The results show that the proposed algorithms have better results for all databases, in the sense of higher savings.Comment: 13 pages, 6 figures, Submitted for possible publicatio

A cost-sensitive decision tree learning algorithm based on a multi-armed bandit framework

This paper develops a new algorithm for inducing cost-sensitive decision trees that is inspired by the multi-armed bandit problem, in which a player in a casino has to decide which slot machine (bandit) from a selection of slot machines is likely to pay out the most. Game Theory proposes a solution to this multi-armed bandit problem by using a process of exploration and exploitation in which reward is maximized. This paper utilizes these concepts to develop a new algorithm by viewing the rewards as a reduction in costs, and utilizing the exploration and exploitation techniques so that a compromise between decisions based on accuracy and decisions based on costs can be found. The algorithm employs the notion of lever pulls in the multi-armed bandit game to select the attributes during decision tree induction, using a look-ahead methodology to explore potential attributes and exploit the attributes which maximizes the reward. The new algorithm is evaluated on fifteen datasets and compared to six well-known algorithms J48, EG2, MetaCost, AdaCostM1, ICET and ACT. The results obtained show that the new multi-armed based algorithm can produce more cost-effective trees without compromising accuracy. The paper also includes a critical appraisal of the limitations of the new algorithm and proposes avenues for further research

Predicting employee absenteeism for cost effective interventions

This paper describes a decision support system designed for a Belgian Human Resource (HR) and Well-Being Service Provider. Their goal is to improve health and well-being in the workplace, and to this end, the task is to identify groups of employees at risk of sickness absence who can then be targeted with interventions aiming to reduce or prevent absences. To facilitate deployment, we apply a range of existing machine-learning methods to obtain predictions at monthly intervals using real HR and payroll data that contains no health-related predictors. We model employee absence as a binary classification problem with loss asymmetry and conceptualise a misclassification cost matrix of employee sickness absence. Model performance is evaluated using cost-based metrics, which have intuitive interpretation. We also demonstrate how this problem can be approached when costs are unknown. The proposed flexible evaluation procedure is not restricted to a specific model or domain and can be applied to address other HR analytics questions when deployed. Our approach of considering a wider range of methods and cost-based performance evaluation is novel in the domain of absenteeism prediction.publishedVersio