1,966 research outputs found
Centroid-Based Clustering with ab-Divergences
Centroid-based clustering is a widely used technique within unsupervised learning
algorithms in many research fields. The success of any centroid-based clustering relies on the
choice of the similarity measure under use. In recent years, most studies focused on including several
divergence measures in the traditional hard k-means algorithm. In this article, we consider the
problem of centroid-based clustering using the family of ab-divergences, which is governed by two
parameters, a and b. We propose a new iterative algorithm, ab-k-means, giving closed-form solutions
for the computation of the sided centroids. The algorithm can be fine-tuned by means of this pair of
values, yielding a wide range of the most frequently used divergences. Moreover, it is guaranteed to
converge to local minima for a wide range of values of the pair (a, b). Our theoretical contribution
has been validated by several experiments performed with synthetic and real data and exploring the
(a, b) plane. The numerical results obtained confirm the quality of the algorithm and its suitability to
be used in several practical applications.MINECO TEC2017-82807-
On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract means
The Jensen-Shannon divergence is a renown bounded symmetrization of the
unbounded Kullback-Leibler divergence which measures the total Kullback-Leibler
divergence to the average mixture distribution. However the Jensen-Shannon
divergence between Gaussian distributions is not available in closed-form. To
bypass this problem, we present a generalization of the Jensen-Shannon (JS)
divergence using abstract means which yields closed-form expressions when the
mean is chosen according to the parametric family of distributions. More
generally, we define the JS-symmetrizations of any distance using generalized
statistical mixtures derived from abstract means. In particular, we first show
that the geometric mean is well-suited for exponential families, and report two
closed-form formula for (i) the geometric Jensen-Shannon divergence between
probability densities of the same exponential family, and (ii) the geometric
JS-symmetrization of the reverse Kullback-Leibler divergence. As a second
illustrating example, we show that the harmonic mean is well-suited for the
scale Cauchy distributions, and report a closed-form formula for the harmonic
Jensen-Shannon divergence between scale Cauchy distributions. We also define
generalized Jensen-Shannon divergences between matrices (e.g., quantum
Jensen-Shannon divergences) and consider clustering with respect to these novel
Jensen-Shannon divergences.Comment: 30 page
Non-flat clustering whith alpha-divergences
International audienceThe scope of the well-known -means algorithm has been broadly extended with some recent results: first, the k-means++ initialization method gives some approximation guarantees; second, the Bregman k-means algorithm generalizes the classical algorithm to the large family of Bregman divergences. The Bregman seeding framework combines approximation guarantees with Bregman divergences. We present here an extension of the k-means algorithm using the family of alpha-divergences. With the framework for representational Bregman divergences, we show that an alpha-divergence based k-means algorithm can be designed. We present preliminary experiments for clustering and image segmentation applications. Since alpha-divergences are the natural divergences for constant curvature spaces, these experiments are expected to give information on the structure of the data
The Burbea-Rao and Bhattacharyya centroids
We study the centroid with respect to the class of information-theoretic
Burbea-Rao divergences that generalize the celebrated Jensen-Shannon divergence
by measuring the non-negative Jensen difference induced by a strictly convex
and differentiable function. Although those Burbea-Rao divergences are
symmetric by construction, they are not metric since they fail to satisfy the
triangle inequality. We first explain how a particular symmetrization of
Bregman divergences called Jensen-Bregman distances yields exactly those
Burbea-Rao divergences. We then proceed by defining skew Burbea-Rao
divergences, and show that skew Burbea-Rao divergences amount in limit cases to
compute Bregman divergences. We then prove that Burbea-Rao centroids are
unique, and can be arbitrarily finely approximated by a generic iterative
concave-convex optimization algorithm with guaranteed convergence property. In
the second part of the paper, we consider the Bhattacharyya distance that is
commonly used to measure overlapping degree of probability distributions. We
show that Bhattacharyya distances on members of the same statistical
exponential family amount to calculate a Burbea-Rao divergence in disguise.
Thus we get an efficient algorithm for computing the Bhattacharyya centroid of
a set of parametric distributions belonging to the same exponential families,
improving over former specialized methods found in the literature that were
limited to univariate or "diagonal" multivariate Gaussians. To illustrate the
performance of our Bhattacharyya/Burbea-Rao centroid algorithm, we present
experimental performance results for -means and hierarchical clustering
methods of Gaussian mixture models.Comment: 13 page
Cramer-Rao Lower Bound and Information Geometry
This article focuses on an important piece of work of the world renowned
Indian statistician, Calyampudi Radhakrishna Rao. In 1945, C. R. Rao (25 years
old then) published a pathbreaking paper, which had a profound impact on
subsequent statistical research.Comment: To appear in Connected at Infinity II: On the work of Indian
mathematicians (R. Bhatia and C.S. Rajan, Eds.), special volume of Texts and
Readings In Mathematics (TRIM), Hindustan Book Agency, 201
Centroid-Based Clustering with αβ-Divergences
Article number 196Centroid-based clustering is a widely used technique within unsupervised learning
algorithms in many research fields. The success of any centroid-based clustering relies on the
choice of the similarity measure under use. In recent years, most studies focused on including several
divergence measures in the traditional hard k-means algorithm. In this article, we consider the
problem of centroid-based clustering using the family of αβ-divergences, which is governed by two
parameters, α and β. We propose a new iterative algorithm, αβ-k-means, giving closed-form solutions
for the computation of the sided centroids. The algorithm can be fine-tuned by means of this pair of
values, yielding a wide range of the most frequently used divergences. Moreover, it is guaranteed to
converge to local minima for a wide range of values of the pair (α, β). Our theoretical contribution
has been validated by several experiments performed with synthetic and real data and exploring the
(α, β) plane. The numerical results obtained confirm the quality of the algorithm and its suitability to
be used in several practical applicationsMinisterio de Economía y Competitividad de España (MINECO) TEC2017-82807-
- …