Ensemble estimation of multivariate f-divergence

f-divergence estimation is an important problem in the fields of information theory, machine learning, and statistics. While several divergence estimators exist, relatively few of their convergence rates are known. We derive the MSE convergence rate for a density plug-in estimator of f-divergence. Then by applying the theory of optimally weighted ensemble estimation, we derive a divergence estimator with a convergence rate of O(1/T) that is simple to implement and performs well in high dimensions. We validate our theoretical results with experiments.Comment: 14 pages, 6 figures, a condensed version of this paper was accepted to ISIT 2014, Version 2: Moved the proofs of the theorems from the main body to appendices at the en

Renormalization and quantum field theory

The aim of this paper is to describe how to use regularization and renormalization to construct a perturbative quantum field theory from a Lagrangian. We first define renormalizations and Feynman measures, and show that although there need not exist a canonical Feynman measure, there is a canonical orbit of Feynman measures under renormalization. We then construct a perturbative quantum field theory from a Lagrangian and a Feynman measure, and show that it satisfies perturbative analogues of the Wightman axioms, extended to allow time-ordered composite operators over curved spacetimes.Comment: 30 pages Revised version fixes a gap in the definition of Feynman measure, and has other minor change

Gaussian limits for generalized spacings

Nearest neighbor cells in $R^d,d\in\mathbb{N}$, are used to define coefficients of divergence ($\phi$-divergences) between continuous multivariate samples. For large sample sizes, such distances are shown to be asymptotically normal with a variance depending on the underlying point density. In $d=1$, this extends classical central limit theory for sum functions of spacings. The general results yield central limit theorems for logarithmic $k$-spacings, information gain, log-likelihood ratios and the number of pairs of sample points within a fixed distance of each other.Comment: Published in at http://dx.doi.org/10.1214/08-AAP537 the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org

On the Minimization of Convex Functionals of Probability Distributions Under Band Constraints

The problem of minimizing convex functionals of probability distributions is solved under the assumption that the density of every distribution is bounded from above and below. A system of sufficient and necessary first-order optimality conditions as well as a bound on the optimality gap of feasible candidate solutions are derived. Based on these results, two numerical algorithms are proposed that iteratively solve the system of optimality conditions on a grid of discrete points. Both algorithms use a block coordinate descent strategy and terminate once the optimality gap falls below the desired tolerance. While the first algorithm is conceptually simpler and more efficient, it is not guaranteed to converge for objective functions that are not strictly convex. This shortcoming is overcome in the second algorithm, which uses an additional outer proximal iteration, and, which is proven to converge under mild assumptions. Two examples are given to demonstrate the theoretical usefulness of the optimality conditions as well as the high efficiency and accuracy of the proposed numerical algorithms.Comment: 13 pages, 5 figures, 2 tables, published in the IEEE Transactions on Signal Processing. In previous versions, the example in Section VI.B contained some mistakes and inaccuracies, which have been fixed in this versio

Small oscillations of a chiral Gross-Neveu system

We study the small oscillations regime (RPA approximation) of the time-dependent mean-field equations, obtained in a previous work, which describe the time evolution of one-body dynamical variables of a uniform Chiral Gross-Neveu system. In this approximation we obtain an analytical solution for the time evolution of the one-body dynamical variables. The two-fermion physics can be explored through this solution. The condition for the existence of bound states is examined.Comment: 21pages, Latex, 1postscript figur