12,491 research outputs found
Ensemble Estimation of Information Divergence
Recent work has focused on the problem of nonparametric estimation of information divergence functionals between two continuous random variables. Many existing approaches require either restrictive assumptions about the density support set or difficult calculations at the support set boundary which must be known a priori. The mean squared error (MSE) convergence rate of a leave-one-out kernel density plug-in divergence functional estimator for general bounded density support sets is derived where knowledge of the support boundary, and therefore, the boundary correction is not required. The theory of optimally weighted ensemble estimation is generalized to derive a divergence estimator that achieves the parametric rate when the densities are sufficiently smooth. Guidelines for the tuning parameter selection and the asymptotic distribution of this estimator are provided. Based on the theory, an empirical estimator of Rényi-α divergence is proposed that greatly outperforms the standard kernel density plug-in estimator in terms of mean squared error, especially in high dimensions. The estimator is shown to be robust to the choice of tuning parameters. We show extensive simulation results that verify the theoretical results of our paper. Finally, we apply the proposed estimator to estimate the bounds on the Bayes error rate of a cell classification problem
Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector
For the important classical problem of inference on a sparse high-dimensional
normal mean vector, we propose a novel empirical Bayes model that admits a
posterior distribution with desirable properties under mild conditions. In
particular, our empirical Bayes posterior distribution concentrates on balls,
centered at the true mean vector, with squared radius proportional to the
minimax rate, and its posterior mean is an asymptotically minimax estimator. We
also show that, asymptotically, the support of our empirical Bayes posterior
has roughly the same effective dimension as the true sparse mean vector.
Simulation from our empirical Bayes posterior is straightforward, and our
numerical results demonstrate the quality of our method compared to others
having similar large-sample properties.Comment: 18 pages, 3 figures, 3 table
The Horseshoe Estimator: Posterior Concentration around Nearly Black Vectors
We consider the horseshoe estimator due to Carvalho, Polson and Scott (2010)
for the multivariate normal mean model in the situation that the mean vector is
sparse in the nearly black sense. We assume the frequentist framework where the
data is generated according to a fixed mean vector. We show that if the number
of nonzero parameters of the mean vector is known, the horseshoe estimator
attains the minimax risk, possibly up to a multiplicative constant. We
provide conditions under which the horseshoe estimator combined with an
empirical Bayes estimate of the number of nonzero means still yields the
minimax risk. We furthermore prove an upper bound on the rate of contraction of
the posterior distribution around the horseshoe estimator, and a lower bound on
the posterior variance. These bounds indicate that the posterior distribution
of the horseshoe prior may be more informative than that of other one-component
priors, including the Lasso.Comment: This version differs from the final published version in pagination
and typographical detail; Available at
http://projecteuclid.org/euclid.ejs/141813426
Estimation of Stress-Strength model in the Generalized Linear Failure Rate Distribution
In this paper, we study the estimation of , also so-called the
stress-strength model, when both and are two independent random
variables with the generalized linear failure rate distributions, under
different assumptions about their parameters. We address the maximum likelihood
estimator (MLE) of and the associated asymptotic confidence interval. In
addition, we compute the MLE and the corresponding Bootstrap confidence
interval when the sample sizes are small. The Bayes estimates of and the
associated credible intervals are also investigated. An extensive computer
simulation is implemented to compare the performances of the proposed
estimators. Eventually, we briefly study the estimation of this model when the
data obtained from both distributions are progressively type-II censored. We
present the MLE and the corresponding confidence interval under three different
progressive censoring schemes. We also analysis a set of real data for
illustrative purpose.Comment: 31 pages, 2 figures, preprin
Asymptotic Properties of Bayes Risk of a General Class of Shrinkage Priors in Multiple Hypothesis Testing Under Sparsity
Consider the problem of simultaneous testing for the means of independent
normal observations. In this paper, we study some asymptotic optimality
properties of certain multiple testing rules induced by a general class of
one-group shrinkage priors in a Bayesian decision theoretic framework, where
the overall loss is taken as the number of misclassified hypotheses. We assume
a two-groups normal mixture model for the data and consider the asymptotic
framework adopted in Bogdan et al. (2011) who introduced the notion of
asymptotic Bayes optimality under sparsity in the context of multiple testing.
The general class of one-group priors under study is rich enough to include,
among others, the families of three parameter beta, generalized double Pareto
priors, and in particular the horseshoe, the normal-exponential-gamma and the
Strawderman-Berger priors. We establish that within our chosen asymptotic
framework, the multiple testing rules under study asymptotically attain the
risk of the Bayes Oracle up to a multiplicative factor, with the constant in
the risk close to the constant in the Oracle risk. This is similar to a result
obtained in Datta and Ghosh (2013) for the multiple testing rule based on the
horseshoe estimator introduced in Carvalho et al. (2009, 2010). We further show
that under very mild assumption on the underlying sparsity parameter, the
induced decision rules based on an empirical Bayes estimate of the
corresponding global shrinkage parameter proposed by van der Pas et al. (2014),
attain the optimal Bayes risk up to the same multiplicative factor
asymptotically. We provide a unifying argument applicable for the general class
of priors under study. In the process, we settle a conjecture regarding
optimality property of the generalized double Pareto priors made in Datta and
Ghosh (2013). Our work also shows that the result in Datta and Ghosh (2013) can
be improved further
Meta learning of bounds on the Bayes classifier error
Meta learning uses information from base learners (e.g. classifiers or
estimators) as well as information about the learning problem to improve upon
the performance of a single base learner. For example, the Bayes error rate of
a given feature space, if known, can be used to aid in choosing a classifier,
as well as in feature selection and model selection for the base classifiers
and the meta classifier. Recent work in the field of f-divergence functional
estimation has led to the development of simple and rapidly converging
estimators that can be used to estimate various bounds on the Bayes error. We
estimate multiple bounds on the Bayes error using an estimator that applies
meta learning to slowly converging plug-in estimators to obtain the parametric
convergence rate. We compare the estimated bounds empirically on simulated data
and then estimate the tighter bounds on features extracted from an image patch
analysis of sunspot continuum and magnetogram images.Comment: 6 pages, 3 figures, to appear in proceedings of 2015 IEEE Signal
Processing and SP Education Worksho
- …