A Note on the Estimation of the Hölder Constant

In this article, we develop a nonparametric estimator for the Hölder constant of a density function. We consider a simulation study to evaluate the performance of the proposal and construct smooth bootstrap confidence intervals. Also, we give a brief review over the impossibility to decide whether a density function is Hölder.Fil: Henry, Guillermo Sebastian. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Rodriguez, Daniela Andrea. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Sued, Raquel Mariela. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentin

Scalable Hash-Based Estimation of Divergence Measures

We propose a scalable divergence estimation method based on hashing. Consider two continuous random variables $X$ and $Y$ whose densities have bounded support. We consider a particular locality sensitive random hashing, and consider the ratio of samples in each hash bin having non-zero numbers of Y samples. We prove that the weighted average of these ratios over all of the hash bins converges to f-divergences between the two samples sets. We show that the proposed estimator is optimal in terms of both MSE rate and computational complexity. We derive the MSE rates for two families of smooth functions; the H\"{o}lder smoothness class and differentiable functions. In particular, it is proved that if the density functions have bounded derivatives up to the order $d/2$, where $d$ is the dimension of samples, the optimal parametric MSE rate of $O(1/N)$ can be achieved. The computational complexity is shown to be $O(N)$, which is optimal. To the best of our knowledge, this is the first empirical divergence estimator that has optimal computational complexity and achieves the optimal parametric MSE estimation rate.Comment: 11 pages, Proceedings of the 21st International Conference on Artificial Intelligence and Statistics (AISTATS) 2018, Lanzarote, Spai

Direct Estimation of Information Divergence Using Nearest Neighbor Ratios

We propose a direct estimation method for R\'{e}nyi and f-divergence measures based on a new graph theoretical interpretation. Suppose that we are given two sample sets $X$ and $Y$, respectively with $N$ and $M$ samples, where $\eta:=M/N$ is a constant value. Considering the $k$-nearest neighbor ($k$-NN) graph of $Y$ in the joint data set $(X,Y)$, we show that the average powered ratio of the number of $X$ points to the number of $Y$ points among all $k$-NN points is proportional to R\'{e}nyi divergence of $X$ and $Y$ densities. A similar method can also be used to estimate f-divergence measures. We derive bias and variance rates, and show that for the class of $\gamma$-H\"{o}lder smooth functions, the estimator achieves the MSE rate of $O(N^{-2\gamma/(\gamma+d)})$. Furthermore, by using a weighted ensemble estimation technique, for density functions with continuous and bounded derivatives of up to the order $d$, and some extra conditions at the support set boundary, we derive an ensemble estimator that achieves the parametric MSE rate of $O(1/N)$. Our estimators are more computationally tractable than other competing estimators, which makes them appealing in many practical applications.Comment: 2017 IEEE International Symposium on Information Theory (ISIT

Empirical Bayes conditional density estimation

The problem of nonparametric estimation of the conditional density of a response, given a vector of explanatory variables, is classical and of prominent importance in many prediction problems since the conditional density provides a more comprehensive description of the association between the response and the predictor than, for instance, does the regression function. The problem has applications across different fields like economy, actuarial sciences and medicine. We investigate empirical Bayes estimation of conditional densities establishing that an automatic data-driven selection of the prior hyper-parameters in infinite mixtures of Gaussian kernels, with predictor-dependent mixing weights, can lead to estimators whose performance is on par with that of frequentist estimators in being minimax-optimal (up to logarithmic factors) rate adaptive over classes of locally H\"older smooth conditional densities and in performing an adaptive dimension reduction if the response is independent of (some of) the explanatory variables which, containing no information about the response, are irrelevant to the purpose of estimating its conditional density

Oscillating Gaussian Processes

In this article we introduce and study oscillating Gaussian processes defined by $X_t = \alpha_+ Y_t {\bf 1}_{Y_t >0} + \alpha_- Y_t{\bf 1}_{Y_t<0}$, where $\alpha_+,\alpha_->0$ are free parameters and $Y$ is either stationary or self-similar Gaussian process. We study the basic properties of $X$ and we consider estimation of the model parameters. In particular, we show that the moment estimators converge in $L^p$ and are, when suitably normalised, asymptotically normal

Minimal H\"older regularity implying finiteness of integral Menger curvature

We study two families of integral functionals indexed by a real number $p > 0$. One family is defined for 1-dimensional curves in $\R^3$ and the other one is defined for $m$-dimensional manifolds in $\R^n$. These functionals are described as integrals of appropriate integrands (strongly related to the Menger curvature) raised to power $p$. Given $p > m(m+1)$ we prove that $C^{1,\alpha}$ regularity of the set (a curve or a manifold), with $\alpha > \alpha_0 = 1 - \frac{m(m+1)}p$ implies finiteness of both curvature functionals ($m=1$ in the case of curves). We also show that $\alpha_0$ is optimal by constructing examples of $C^{1,\alpha_0}$ functions with graphs of infinite integral curvature