Fast Projection onto the Simplex and the l1 Ball

International audienceA new algorithm is proposed to project, exactly and in finite time, a vector of arbitrary size onto a simplex or an l1-norm ball. It can be viewed as a Gauss-Seidel-like variant of Michelot’s variable fixing algorithm; that is, the threshold used to fix the variables is updated after each element is read, instead of waiting for a full reading pass over the list of non-fixed elements. This algorithm is empirically demonstrated to be faster than existing methods

Projection Onto A Simplex

This mini-paper presents a fast and simple algorithm to compute the projection onto the canonical simplex $\triangle^n$. Utilizing the Moreau's identity, we show that the problem is essentially a univariate minimization and the objective function is strictly convex and continuously differentiable. Moreover, it is shown that there are at most n candidates which can be computed explicitly, and the minimizer is the only one that falls into the correct interval

High-Dimensional Menger-Type Curvatures - Part I: Geometric Multipoles and Multiscale Inequalities

We define a discrete Menger-type curvature of d+2 points in a real separable Hilbert space H by an appropriate scaling of the squared volume of the corresponding (d+1)-simplex. We then form a continuous curvature of an Ahlfors d-regular measure on H by integrating the discrete curvature according to the product measure. The aim of this work, continued in a subsequent paper, is to estimate multiscale least squares approximations of such measures by the Menger-type curvature. More formally, we show that the continuous d-dimensional Menger-type curvature is comparable to the ``Jones-type flatness''. The latter quantity adds up scaled errors of approximations of a measure by d-planes at different scales and locations, and is commonly used to characterize uniform rectifiability. We thus obtain a characterization of uniform rectifiability by using the Menger-type curvature. In the current paper (part I) we control the continuous Menger-type curvature of an Ahlfors d-regular measure by its Jones-type flatness.Comment: 47 pages, 13 figures. Minor revisions and the inclusion of figure

A Novel Rate Control Algorithm for Onboard Predictive Coding of Multispectral and Hyperspectral Images

Predictive coding is attractive for compression onboard of spacecrafts thanks to its low computational complexity, modest memory requirements and the ability to accurately control quality on a pixel-by-pixel basis. Traditionally, predictive compression focused on the lossless and near-lossless modes of operation where the maximum error can be bounded but the rate of the compressed image is variable. Rate control is considered a challenging problem for predictive encoders due to the dependencies between quantization and prediction in the feedback loop, and the lack of a signal representation that packs the signal's energy into few coefficients. In this paper, we show that it is possible to design a rate control scheme intended for onboard implementation. In particular, we propose a general framework to select quantizers in each spatial and spectral region of an image so as to achieve the desired target rate while minimizing distortion. The rate control algorithm allows to achieve lossy, near-lossless compression, and any in-between type of compression, e.g., lossy compression with a near-lossless constraint. While this framework is independent of the specific predictor used, in order to show its performance, in this paper we tailor it to the predictor adopted by the CCSDS-123 lossless compression standard, obtaining an extension that allows to perform lossless, near-lossless and lossy compression in a single package. We show that the rate controller has excellent performance in terms of accuracy in the output rate, rate-distortion characteristics and is extremely competitive with respect to state-of-the-art transform coding

A Practical Algorithm for Topic Modeling with Provable Guarantees

Topic models provide a useful method for dimensionality reduction and exploratory data analysis in large text corpora. Most approaches to topic model inference have been based on a maximum likelihood objective. Efficient algorithms exist that approximate this objective, but they have no provable guarantees. Recently, algorithms have been introduced that provide provable bounds, but these algorithms are not practical because they are inefficient and not robust to violations of model assumptions. In this paper we present an algorithm for topic model inference that is both provable and practical. The algorithm produces results comparable to the best MCMC implementations while running orders of magnitude faster.Comment: 26 page

Regularized Optimal Transport and the Rot Mover's Distance

This paper presents a unified framework for smooth convex regularization of discrete optimal transport problems. In this context, the regularized optimal transport turns out to be equivalent to a matrix nearness problem with respect to Bregman divergences. Our framework thus naturally generalizes a previously proposed regularization based on the Boltzmann-Shannon entropy related to the Kullback-Leibler divergence, and solved with the Sinkhorn-Knopp algorithm. We call the regularized optimal transport distance the rot mover's distance in reference to the classical earth mover's distance. We develop two generic schemes that we respectively call the alternate scaling algorithm and the non-negative alternate scaling algorithm, to compute efficiently the regularized optimal plans depending on whether the domain of the regularizer lies within the non-negative orthant or not. These schemes are based on Dykstra's algorithm with alternate Bregman projections, and further exploit the Newton-Raphson method when applied to separable divergences. We enhance the separable case with a sparse extension to deal with high data dimensions. We also instantiate our proposed framework and discuss the inherent specificities for well-known regularizers and statistical divergences in the machine learning and information geometry communities. Finally, we demonstrate the merits of our methods with experiments using synthetic data to illustrate the effect of different regularizers and penalties on the solutions, as well as real-world data for a pattern recognition application to audio scene classification