1 research outputs found
Test cost and misclassification cost trade-off using reframing
Many solutions to cost-sensitive classification (and regression) rely on some
or all of the following assumptions: we have complete knowledge about the cost
context at training time, we can easily re-train whenever the cost context
changes, and we have technique-specific methods (such as cost-sensitive
decision trees) that can take advantage of that information. In this paper we
address the problem of selecting models and minimising joint cost (integrating
both misclassification cost and test costs) without any of the above
assumptions. We introduce methods and plots (such as the so-called JROC plots)
that can work with any off-the-shelf predictive technique, including ensembles,
such that we reframe the model to use the appropriate subset of attributes (the
feature configuration) during deployment time. In other words, models are
trained with the available attributes (once and for all) and then deployed by
setting missing values on the attributes that are deemed ineffective for
reducing the joint cost. As the number of feature configuration combinations
grows exponentially with the number of features we introduce quadratic methods
that are able to approximate the optimal configuration and model choices, as
shown by the experimental results.Comment: Keywords: test cost, misclassification cost, missing values,
reframing, ROC analysis, operating context, feature configuration, feature
selectio