Optimal testing for properties of distributions

Given samples from an unknown discrete distribution p, is it possible to distinguish whether p belongs to some class of distributions C versus p being far from every distribution in C? This fundamental question has received tremendous attention in statistics, focusing primarily on asymptotic analysis, as well as in information theory and theoretical computer science, where the emphasis has been on small sample size and computational complexity. Nevertheless, even for basic properties of discrete distributions such as monotonicity, independence, logconcavity, unimodality, and monotone-hazard rate, the optimal sample complexity is unknown. We provide a general approach via which we obtain sample-optimal and computationally efficient testers for all these distribution families. At the core of our approach is an algorithm which solves the following problem: Given samples from an unknown distribution p, and a known distribution q, are p and q close in x[superscript 2]-distance, or far in total variation distance? The optimality of our testers is established by providing matching lower bounds, up to constant factors. Finally, a necessary building block for our testers and an important byproduct of our work are the first known computationally efficient proper learners for discrete log-concave, monotone hazard rate distributions

Testing uniformity on high-dimensional spheres against monotone rotationally symmetric alternatives

We consider the problem of testing uniformity on high-dimensional unit spheres. We are primarily interested in non-null issues. We show that rotationally symmetric alternatives lead to two Local Asymptotic Normality (LAN) structures. The first one is for fixed modal location $\theta$ and allows to derive locally asymptotically most powerful tests under specified $\theta$. The second one, that addresses the Fisher-von Mises-Langevin (FvML) case, relates to the unspecified-$\theta$ problem and shows that the high-dimensional Rayleigh test is locally asymptotically most powerful invariant. Under mild assumptions, we derive the asymptotic non-null distribution of this test, which allows to extend away from the FvML case the asymptotic powers obtained there from Le Cam's third lemma. Throughout, we allow the dimension $p$ to go to infinity in an arbitrary way as a function of the sample size $n$. Some of our results also strengthen the local optimality properties of the Rayleigh test in low dimensions. We perform a Monte Carlo study to illustrate our asymptotic results. Finally, we treat an application related to testing for sphericity in high dimensions

Some goodness-of-fit tests and efficient estimation in longitudinal surveys under missing data

In statistical analysis, distribution assumptions are often subject to be tested. In this dissertation, the problem of testing two important distribution assumptions, the normal distribution and uniform distribution, is considered. Specifically, a new characterization of multivariate normality based on univariate projections is developed. On the other hand, a powerful affine invariant test of multivariate normality is proposed. Moreover, this dissertation also presents an asymptotic distribution free test of multivariate uniformity based on $m$-nearest neighbors. Both tests have demonstrated good power performance by numerical studies. Incomplete data is another commonly encountered issue in practice. This dissertation also reports an efficient estimation method for population mean in longitudinal surveys under monotone missing pattern. The proposed method is developed using the generalized method of moments technique by incorporating all the available information at each time point. Efficiency of the method over the direct propensity score type estimator is also demonstrated by limited numerical studies

Inference Based on Conditional Moment Inequalities

In this paper, we propose an instrumental variable approach to constructing confidence sets (CS's) for the true parameter in models defined by conditional moment inequalities/equalities. We show that by properly choosing instrument functions, one can transform conditional moment inequalities/equalities into unconditional ones without losing identification power. Based on the unconditional moment inequalities/equalities, we construct CS's by inverting Cramer-von Mises-type or Kolmogorov-Smirnov-type tests. Critical values are obtained using generalized moment selection (GMS) procedures. We show that the proposed CS's have correct uniform asymptotic coverage probabilities. New methods are required to establish these results because an infinite-dimensional nuisance parameter affects the asymptotic distributions. We show that the tests considered are consistent against all fixed alternatives and have power against some n^{-1/2}-local alternatives, though not all such alternatives. Monte Carlo simulations for three different models show that the methods perform well in finite samples.Asymptotic size, asymptotic power, conditional moment inequalities, confidence set, Cramer-von Mises, generalized moment selection, Kolmogorov-Smirnov, moment inequalities

Tests based on characterizations, and their efficiencies: a survey

A survey of goodness-of-fit and symmetry tests based on the characterization properties of distributions is presented. This approach became popular in recent years. In most cases the test statistics are functionals of $U$-empirical processes. The limiting distributions and large deviations of new statistics under the null hypothesis are described. Their local Bahadur efficiency for various parametric alternatives is calculated and compared with each other as well as with diverse previously known tests. We also describe new directions of possible research in this domain.Comment: Open access in Acta et Commentationes Universitatis Tartuensis de Mathematic

Theory of Stochastic Optimal Economic Growth

This paper is a survey of the theory of stochastic optimal economic growth.International Development,