2 research outputs found
Approximation Algorithms for Clustering via Weighted Impurity Measures
An impurity measures maps a -dimensional vector to a non-negative value so that the more homogeneous , the larger its impurity. We study clustering based on impurity measures:
given a collection of many -dimensional vectors and an impurity
measure , the goal is to find a partition of into groups
that minimizes the total impurities of the groups in , i.e.,
Impurity minimization is widely used as quality assessment measure in
probability distribution clustering and in categorical clustering where it is
not possible to rely on geometric properties of the data set. However, in
contrast to the case of metric based clustering, the current knowledge of
impurity measure based clustering in terms of approximation and
inapproximability results is very limited.
Our research contributes to fill this gap. We first present a simple linear
time algorithm that simultaneously achieves -approximation for the Gini
impurity measure and -approximation for the Entropy impurity measure. Then, for the Entropy
impurity measure---where we also show that finding the optimal clustering is
strongly NP-hard---we are able to design a polynomial time
-approximation algorithm. Our algorithm relies on a
nontrivial characterization of a class of clusterings that necessarily includes
a partition achieving --approximation of the impurity
of the optimal partition. Remarkably, this is the first polynomial time
algorithm with approximation guarantee independent of the number of
points/vector and not relying on any restriction on the components of the
vectors for producing clusterings with minimum entropy
Minimization of Gini impurity via connections with the k-means problem
The Gini impurity is one of the measures used to select attribute in Decision
Trees/Random Forest construction. In this note we discuss connections between
the problem of computing the partition with minimum Weighted Gini impurity and
the -means clustering problem. Based on these connections we show that the
computation of the partition with minimum Weighted Gini is a NP-Complete
problem and we also discuss how to obtain new algorithms with provable
approximation for the Gini Minimization problem