178,489 research outputs found
Higher criticism for detecting sparse heterogeneous mixtures
Higher criticism, or second-level significance testing, is a
multiple-comparisons concept mentioned in passing by Tukey. It concerns a
situation where there are many independent tests of significance and one is
interested in rejecting the joint null hypothesis. Tukey suggested comparing
the fraction of observed significances at a given \alpha-level to the expected
fraction under the joint null. In fact, he suggested standardizing the
difference of the two quantities and forming a z-score; the resulting z-score
tests the significance of the body of significance tests. We consider a
generalization, where we maximize this z-score over a range of significance
levels 0<\alpha\leq\alpha_0.
We are able to show that the resulting higher criticism statistic is
effective at resolving a very subtle testing problem: testing whether n normal
means are all zero versus the alternative that a small fraction is nonzero. The
subtlety of this ``sparse normal means'' testing problem can be seen from work
of Ingster and Jin, who studied such problems in great detail. In their
studies, they identified an interesting range of cases where the small fraction
of nonzero means is so small that the alternative hypothesis exhibits little
noticeable effect on the distribution of the p-values either for the bulk of
the tests or for the few most highly significant tests.
In this range, when the amplitude of nonzero means is calibrated with the
fraction of nonzero means, the likelihood ratio test for a precisely specified
alternative would still succeed in separating the two hypotheses.Comment: Published by the Institute of Mathematical Statistics
(http://www.imstat.org) in the Annals of Statistics
(http://www.imstat.org/aos/) at http://dx.doi.org/10.1214/00905360400000026
- …