9,605 research outputs found
Approximate Profile Maximum Likelihood
We propose an efficient algorithm for approximate computation of the profile
maximum likelihood (PML), a variant of maximum likelihood maximizing the
probability of observing a sufficient statistic rather than the empirical
sample. The PML has appealing theoretical properties, but is difficult to
compute exactly. Inspired by observations gleaned from exactly solvable cases,
we look for an approximate PML solution, which, intuitively, clumps comparably
frequent symbols into one symbol. This amounts to lower-bounding a certain
matrix permanent by summing over a subgroup of the symmetric group rather than
the whole group during the computation. We extensively experiment with the
approximate solution, and find the empirical performance of our approach is
competitive and sometimes significantly better than state-of-the-art
performance for various estimation problems
Direct Ensemble Estimation of Density Functionals
Estimating density functionals of analog sources is an important problem in
statistical signal processing and information theory. Traditionally, estimating
these quantities requires either making parametric assumptions about the
underlying distributions or using non-parametric density estimation followed by
integration. In this paper we introduce a direct nonparametric approach which
bypasses the need for density estimation by using the error rates of k-NN
classifiers asdata-driven basis functions that can be combined to estimate a
range of density functionals. However, this method is subject to a non-trivial
bias that dramatically slows the rate of convergence in higher dimensions. To
overcome this limitation, we develop an ensemble method for estimating the
value of the basis function which, under some minor constraints on the
smoothness of the underlying distributions, achieves the parametric rate of
convergence regardless of data dimension.Comment: 5 page
On the Minimization of Convex Functionals of Probability Distributions Under Band Constraints
The problem of minimizing convex functionals of probability distributions is
solved under the assumption that the density of every distribution is bounded
from above and below. A system of sufficient and necessary first-order
optimality conditions as well as a bound on the optimality gap of feasible
candidate solutions are derived. Based on these results, two numerical
algorithms are proposed that iteratively solve the system of optimality
conditions on a grid of discrete points. Both algorithms use a block coordinate
descent strategy and terminate once the optimality gap falls below the desired
tolerance. While the first algorithm is conceptually simpler and more
efficient, it is not guaranteed to converge for objective functions that are
not strictly convex. This shortcoming is overcome in the second algorithm,
which uses an additional outer proximal iteration, and, which is proven to
converge under mild assumptions. Two examples are given to demonstrate the
theoretical usefulness of the optimality conditions as well as the high
efficiency and accuracy of the proposed numerical algorithms.Comment: 13 pages, 5 figures, 2 tables, published in the IEEE Transactions on
Signal Processing. In previous versions, the example in Section VI.B
contained some mistakes and inaccuracies, which have been fixed in this
versio
On deconvolution of distribution functions
The subject of this paper is the problem of nonparametric estimation of a
continuous distribution function from observations with measurement errors. We
study minimax complexity of this problem when unknown distribution has a
density belonging to the Sobolev class, and the error density is ordinary
smooth. We develop rate optimal estimators based on direct inversion of
empirical characteristic function. We also derive minimax affine estimators of
the distribution function which are given by an explicit convex optimization
problem. Adaptive versions of these estimators are proposed, and some numerical
results demonstrating good practical behavior of the developed procedures are
presented.Comment: Published in at http://dx.doi.org/10.1214/11-AOS907 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
- …