kk-MLE: A fast algorithm for learning statistical mixture models

We describe $k$-MLE, a fast and efficient local search algorithm for learning finite statistical mixtures of exponential families such as Gaussian mixture models. Mixture models are traditionally learned using the expectation-maximization (EM) soft clustering technique that monotonically increases the incomplete (expected complete) likelihood. Given prescribed mixture weights, the hard clustering $k$-MLE algorithm iteratively assigns data to the most likely weighted component and update the component models using Maximum Likelihood Estimators (MLEs). Using the duality between exponential families and Bregman divergences, we prove that the local convergence of the complete likelihood of $k$-MLE follows directly from the convergence of a dual additively weighted Bregman hard clustering. The inner loop of $k$-MLE can be implemented using any $k$-means heuristic like the celebrated Lloyd's batched or Hartigan's greedy swap updates. We then show how to update the mixture weights by minimizing a cross-entropy criterion that implies to update weights by taking the relative proportion of cluster points, and reiterate the mixture parameter update and mixture weight update processes until convergence. Hard EM is interpreted as a special case of $k$-MLE when both the component update and the weight update are performed successively in the inner loop. To initialize $k$-MLE, we propose $k$-MLE++, a careful initialization of $k$-MLE guaranteeing probabilistically a global bound on the best possible complete likelihood.Comment: 31 pages, Extend preliminary paper presented at IEEE ICASSP 201

Bregman Voronoi Diagrams: Properties, Algorithms and Applications

The Voronoi diagram of a finite set of objects is a fundamental geometric structure that subdivides the embedding space into regions, each region consisting of the points that are closer to a given object than to the others. We may define many variants of Voronoi diagrams depending on the class of objects, the distance functions and the embedding space. In this paper, we investigate a framework for defining and building Voronoi diagrams for a broad class of distance functions called Bregman divergences. Bregman divergences include not only the traditional (squared) Euclidean distance but also various divergence measures based on entropic functions. Accordingly, Bregman Voronoi diagrams allow to define information-theoretic Voronoi diagrams in statistical parametric spaces based on the relative entropy of distributions. We define several types of Bregman diagrams, establish correspondences between those diagrams (using the Legendre transformation), and show how to compute them efficiently. We also introduce extensions of these diagrams, e.g. k-order and k-bag Bregman Voronoi diagrams, and introduce Bregman triangulations of a set of points and their connexion with Bregman Voronoi diagrams. We show that these triangulations capture many of the properties of the celebrated Delaunay triangulation. Finally, we give some applications of Bregman Voronoi diagrams which are of interest in the context of computational geometry and machine learning.Comment: Extend the proceedings abstract of SODA 2007 (46 pages, 15 figures

Bregman Voronoi diagrams

A preliminary version appeared in the 18th ACM-SIAM Symposium on Discrete Algorithms, pp. 746- 755, 2007International audienceThe Voronoi diagram of a finite set of objects is a fundamental geometric structure that subdivides the embedding space into regions, each region consisting of the points that are closer to a given object than to the others. We may define various variants of Voronoi diagrams depending on the class of objects, the distance function and the embedding space. In this paper, we investigate a framework for defining and building Voronoi diagrams for a broad class of distance functions called Bregman divergences. Bregman divergences include not only the traditional (squared) Euclidean distance but also various divergence measures based on entropic functions. Accordingly, Bregman Voronoi diagrams allow one to define information-theoretic Voronoi diagrams in sta- tistical parametric spaces based on the relative entropy of distributions. We define several types of Bregman diagrams, establish correspondences between those diagrams (using the Legendre transformation), and show how to compute them efficiently. We also introduce extensions of these diagrams, e.g. k-order and k-bag Bregman Voronoi diagrams, and introduce Bregman triangulations of a set of points and their connection with Bregman Voronoi diagrams. We show that these triangulations capture many of the properties of the celebrated Delaunay triangulation

The Hidden Geometry of Particle Collisions

We establish that many fundamental concepts and techniques in quantum field theory and collider physics can be naturally understood and unified through a simple new geometric language. The idea is to equip the space of collider events with a metric, from which other geometric objects can be rigorously defined. Our analysis is based on the energy mover's distance, which quantifies the "work" required to rearrange one event into another. This metric, which operates purely at the level of observable energy flow information, allows for a clarified definition of infrared and collinear safety and related concepts. A number of well-known collider observables can be exactly cast as the minimum distance between an event and various manifolds in this space. Jet definitions, such as exclusive cone and sequential recombination algorithms, can be directly derived by finding the closest few-particle approximation to the event. Several area- and constituent-based pileup mitigation strategies are naturally expressed in this formalism as well. Finally, we lift our reasoning to develop a precise distance between theories, which are treated as collections of events weighted by cross sections. In all of these various cases, a better understanding of existing methods in our geometric language suggests interesting new ideas and generalizations.Comment: 56 pages, 11 figures, 5 tables; v2: minor changes and updated references; v3: updated to match JHEP versio

Algorithmic Superactivation of Asymptotic Quantum Capacity of Zero-Capacity Quantum Channels

The superactivation of zero-capacity quantum channels makes it possible to use two zero-capacity quantum channels with a positive joint capacity for their output. Currently, we have no theoretical background to describe all possible combinations of superactive zero-capacity channels; hence, there may be many other possible combinations. In practice, to discover such superactive zero-capacity channel-pairs, we must analyze an extremely large set of possible quantum states, channel models, and channel probabilities. There is still no extremely efficient algorithmic tool for this purpose. This paper shows an efficient algorithmical method of finding such combinations. Our method can be a very valuable tool for improving the results of fault-tolerant quantum computation and possible communication techniques over very noisy quantum channels.Comment: 35 pages, 17 figures, Journal-ref: Information Sciences (Elsevier, 2012), presented in part at Quantum Information Processing 2012 (QIP2012), v2: minor changes, v3: published version; Information Sciences, Elsevier, ISSN: 0020-0255; 201

Une fonction distance à k points pour l'inférence géométrique robuste

Analyzing the sub-level sets of the distance to a compact sub-manifold of R d is a common method in topological data analysis, to understand its topology. Therefore, topological inference procedures usually rely on a distance estimate based on n sample points [41]. In the case where sample points are corrupted by noise, the distance-to-measure function (DTM, [16]) is a surrogate for the distance-to-compact-set function. In practice, computing the homology of its sub-level sets requires to compute the homology of unions of n balls ([28, 14]), that might become intractable whenever n is large. To simultaneously face the two problems of a large number of points and noise, we introduce the k-power-distance-to-measure function (k-PDTM). This new approximation of the distance-to-measure may be thought of as a k-pointbased approximation of the DTM. Its sublevel sets consist in unions of k balls, and this distance is also proved robust to noise. We assess the quality of this approximation for k possibly drastically smaller than n, and provide an algorithm to compute this k-PDTM from a sample. Numerical experiments illustrate the good behavior of this k-points approximation in a noisy topological inference framework.Afin de comprendre la topologie d'une sous-variété compacte de R^d, il est courant en analyse topologique des données d'analyser les sous-niveaux de la fonction distance à cette sous-variété. C'est pourquoi, les procédures d'inférence topologique reposent souvent sur des estimées de la fonction distance, construites sur n points. Lorsque l'échantillon de points est corrompu par des données aberrantes, la fonction distance à la mesure (DTM) est une alternative à la distance au compact. En pratique, le calcul de l'homologie de ses sous-niveaux revient à calculer l'homologie d'unions de n boules, ce qui devient impossible lorsque n est grand. Afin de pallier simultanément le problème du grand nombre de points et du bruit, nous introduisons la fonction k-puissance distance à la mesure (k-PDTM). Il s'agit d'une nouvelle approximation de la fonction distance à la mesure qui peut être vue comme une approximation de la DTM basée sur k points. Ses sous-niveaux sont des unions de k boules, et cette distance est robuste au bruit. Nous étudions la qualité de cette approximation pour k possiblement beaucoup plus petit que n, et fournissons un algorithme permettant de calculer cette k-PDTM à partir d'un échantillon de points. Des expériences numériques illustrent le bon comportement de cette approximation construite sur k points, dans le cadre de l'inférence topologique avec bruit

A unified approach to energy conservation and potential vorticity dynamics for arbitrarily-structured C-grids

Copyright © 2010 Elsevier. NOTICE: this is the author’s version of a work that was accepted for publication in Journal of Computational Physics. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Journal of Computational Physics, Vol. 229, Issue 9 (2010), DOI: 10.1016/j.jcp.2009.12.007A numerical scheme applicable to arbitrarily-structured C-grids is presented for the nonlinear shallow-water equations. By discretizing the vector-invariant form of the momentum equation, the relationship between the nonlinear Coriolis force and the potential vorticity flux can be used to guarantee that mass, velocity and potential vorticity evolve in a consistent and compatible manner. Underpinning the consistency and compatibility of the discrete system is the construction of an auxiliary thickness equation that is staggered from the primary thickness equation and collocated with the vorticity field. The numerical scheme also exhibits conservation of total energy to within time-truncation error. Simulations of the standard shallow-water test cases confirm the analysis and show convergence rates between 1st1st- and 2nd2nd-order accuracy when discretizing the system with quasi-uniform spherical Voronoi diagrams. The numerical method is applicable to a wide class of meshes, including latitude–longitude grids, Voronoi diagrams, Delaunay triangulations and conformally-mapped cubed-sphere meshes