Hashing-Based-Estimators for Kernel Density in High Dimensions

Given a set of points $P\subset \mathbb{R}^{d}$ and a kernel $k$, the Kernel Density Estimate at a point $x\in\mathbb{R}^{d}$ is defined as $\mathrm{KDE}_{P}(x)=\frac{1}{|P|}\sum_{y\in P} k(x,y)$. We study the problem of designing a data structure that given a data set $P$ and a kernel function, returns *approximations to the kernel density* of a query point in *sublinear time*. We introduce a class of unbiased estimators for kernel density implemented through locality-sensitive hashing, and give general theorems bounding the variance of such estimators. These estimators give rise to efficient data structures for estimating the kernel density in high dimensions for a variety of commonly used kernels. Our work is the first to provide data-structures with theoretical guarantees that improve upon simple random sampling in high dimensions.Comment: A preliminary version of this paper appeared in FOCS 201

ROBUST KERNEL ESTIMATOR FOR DENSITIES OF UNKNOWN

Results on nonparametric kernel estimators of density differ according to the assumed degree of density smoothness; it is often assumed that the density function is at least twice differentiable. However, there are cases where non-smooth density functions may be of interest. We provide asymptotic results for kernel estimation of a continuous density for an arbitrary bandwidth/kernel pair. We also derive the limit joint distribution of kernel density estimators coresponding to different bandwidths and kernel functions. Using these reults, we construct an estimator that combines several estimators for different bandwidth/kernel pairs to protect against the negative consequences of errors in assumptions about order of smoothness. The results of a Monte Carlo experiment confirm the usefulness of the combined estimator. We demonstrate that while in the standard normal case the combined estimator has a relatively higher mean squared error than the standard kernel estimator, both estimators are highly accurate. On the other hand, for a non-smooth density where the MSE gets very large, the combined estimator provides uniformly better results than the standard estimator.

Kernel density estimation via diffusion

We present a new adaptive kernel density estimator based on linear diffusion processes. The proposed estimator builds on existing ideas for adaptive smoothing by incorporating information from a pilot density estimate. In addition, we propose a new plug-in bandwidth selection method that is free from the arbitrary normal reference rules used by existing methods. We present simulation examples in which the proposed approach outperforms existing methods in terms of accuracy and reliability.Comment: Published in at http://dx.doi.org/10.1214/10-AOS799 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Nonparametric Density Estimation for Linear Processes with Infinite Variance

We consider nonparametric estimation of marginal density functions of linear processes by using kernel density estimators. We assume that the innovation processes are i.i.d. and have infinite-variance. We present the asymptotic distributions of the kernel density estimators with the order of bandwidths fixed as h=cn-1/5, where n is the sample size. The asymptotic distributions depend on both the coefficients of linear processes and the tail behavior of the innovations. In some cases, the kernel estimators have the same asymptotic distributions as for i.i.d. observations. In other cases, the normalized kernel density estimators converge in distribution to stable distributions. A simulation study is also carried out to examine small sample properties.linear processes, kernel density estimator, domain of attraction, stable distribution, noncentral limit theorem, martingale central limit theorem

Sparse kernel density estimation technique based on zero-norm constraint

A sparse kernel density estimator is derived based on the zero-norm constraint, in which the zero-norm of the kernel weights is incorporated to enhance model sparsity. The classical Parzen window estimate is adopted as the desired response for density estimation, and an approximate function of the zero-norm is used for achieving mathemtical tractability and algorithmic efficiency. Under the mild condition of the positive definite design matrix, the kernel weights of the proposed density estimator based on the zero-norm approximation can be obtained using the multiplicative nonnegative quadratic programming algorithm. Using the -optimality based selection algorithm as the preprocessing to select a small significant subset design matrix, the proposed zero-norm based approach offers an effective means for constructing very sparse kernel density estimates with excellent generalisation performance

From target to projectile and back again: selfduality of high energy evolution

We prove that the complete kernel for the high energy evolution in QCD must be selfdual. The relevant duality transformation is formulated in precise mathematical terms and is shown to transform the charge density into the functional derivative with respect to the single-gluon scattering matrix. This transformation interchanges the high and the low density regimes. We demostrate that the original JIMWLK kernel, valid at large density is indeed dual to the low denisity limit of the complete kernel derived recently in hep-ph/0501198.Comment: 4 pages. References and comments added. To appear in PR