A current assumption of most clustering methods is that the training data and
future data are taken from the same distribution. However, this assumption may
not hold in most real-world scenarios. In this paper, we propose an information
theoretical importance sampling based approach for clustering problems (ITISC)
which minimizes the worst case of expected distortions under the constraint of
distribution deviation. The distribution deviation constraint can be converted
to the constraint over a set of weight distributions centered on the uniform
distribution derived from importance sampling. The objective of the proposed
approach is to minimize the loss under maximum degradation hence the resulting
problem is a constrained minimax optimization problem which can be reformulated
to an unconstrained problem using the Lagrange method. The optimization problem
can be solved by both an alternative optimization algorithm or a general
optimization routine by commercially available software. Experiment results on
synthetic datasets and a real-world load forecasting problem validate the
effectiveness of the proposed model. Furthermore, we show that fuzzy c-means is
a special case of ITISC with the logarithmic distortion, and this observation
provides an interesting physical interpretation for fuzzy exponent m.Comment: 15 pages, 9 figure