The Interval Property in Multiple Testing of Pairwise Differences

The usual step-down and step-up multiple testing procedures most often lack an important intuitive, practical, and theoretical property called the interval property. In short, the interval property is simply that for an individual hypothesis, among the several to be tested, the acceptance sections of relevant statistics are intervals. Lack of the interval property is a serious shortcoming. This shortcoming is demonstrated for testing various pairwise comparisons in multinomial models, multivariate normal models and in nonparametric models. Residual based stepwise multiple testing procedures that do have the interval property are offered in all these cases.Comment: Published in at http://dx.doi.org/10.1214/11-STS372 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Characterization of Bayes procedures for multiple endpoint problems and inadmissibility of the step-up procedure

The problem of multiple endpoint testing for k endpoints is treated as a 2^k finite action problem. The loss function chosen is a vector loss function consisting of two components. The two components lead to a vector risk. One component of the vector risk is the false rejection rate (FRR), that is, the expected number of false rejections. The other component is the false acceptance rate (FAR), that is, the expected number of acceptances for which the corresponding null hypothesis is false. This loss function is more stringent than the positive linear combination loss function of Lehmann [Ann. Math. Statist. 28 (1957) 1-25] and Cohen and Sackrowitz [Ann. Statist. (2005) 33 126-144] in the sense that the class of admissible rules is larger for this vector risk formulation than for the linear combination risk function. In other words, fewer procedures are inadmissible for the vector risk formulation. The statistical model assumed is that the vector of variables Z is multivariate normal with mean vector \mu and known intraclass covariance matrix \Sigma. The endpoint hypotheses are H_i:\mu_i=0 vs K_i:\mu_i>0, i=1,...,k. A characterization of all symmetric Bayes procedures and their limits is obtained. The characterization leads to a complete class theorem. The complete class theorem is used to provide a useful necessary condition for admissibility of a procedure. The main result is that the step-up multiple endpoint procedure is shown to be inadmissible.Comment: Published at http://dx.doi.org/10.1214/009053604000000986 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Decision theory results for one-sided multiple comparison procedures

A resurgence of interest in multiple hypothesis testing has occurred in the last decade. Motivated by studies in genomics, microarrays, DNA sequencing, drug screening, clinical trials, bioassays, education and psychology, statisticians have been devoting considerable research energy in an effort to properly analyze multiple endpoint data. In response to new applications, new criteria and new methodology, many ad hoc procedures have emerged. The classical requirement has been to use procedures which control the strong familywise error rate (FWE) at some predetermined level \alpha. That is, the probability of any false rejection of a true null hypothesis should be less than or equal to \alpha. Finding desirable and powerful multiple test procedures is difficult under this requirement. One of the more recent ideas is concerned with controlling the false discovery rate (FDR), that is, the expected proportion of rejected hypotheses which are, in fact, true. Many multiple test procedures do control the FDR. A much earlier approach to multiple testing was formulated by Lehmann [Ann. Math. Statist. 23 (1952) 541-552 and 28 (1957) 1-25]. Lehmann's approach is decision theoretic and he treats the multiple endpoints problem as a 2^k finite action problem when there are k endpoints. This approach is appealing since unlike the FWE and FDR criteria, the finite action approach pays attention to false acceptances as well as false rejections.Comment: Published at http://dx.doi.org/10.1214/009053604000000968 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

An Alternative to Student\u27s t-Test for Problems With Indifference Zones

Consider a sample from a normal population with mean, μ, and variance unknown. Suppose it is desired to test H0:μ ≤ μ0 versus H1:μ ≥ μ1, with the region HI1:μ0 \u3c μ \u3c μ1 being a (nonempty) indifference zone. It is shown that the usual Student\u27s t-test is inadmissible for this problem. An alternative test is proposed. The two sided problem with indifference region is also discussed. By contrast with the above result, the usual Student\u27s t-test is admissible here. However the two sided version of the alternative test mentioned above does offer some practical advantages relative to the two sided t-test. A 3-decision version of the two sided problem is also discussed. Here the t-test is inadmissible, and is dominated by the appropriate version of the alternative test. The results concerning tests are also reformulated as results about confidence procedures

A new multiple testing method in the dependent case

The most popular multiple testing procedures are stepwise procedures based on $P$-values for individual test statistics. Included among these are the false discovery rate (FDR) controlling procedures of Benjamini--Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289--300] and their offsprings. Even for models that entail dependent data, $P$-values based on marginal distributions are used. Unlike such methods, the new method takes dependency into account at all stages. Furthermore, the $P$-value procedures often lack an intuitive convexity property, which is needed for admissibility. Still further, the new methodology is computationally feasible. If the number of tests is large and the proportion of true alternatives is less than say 25 percent, simulations demonstrate a clear preference for the new methodology. Applications are detailed for models such as testing treatments against control (or any intraclass correlation model), testing for change points and testing means when correlation is successive.Comment: Published in at http://dx.doi.org/10.1214/08-AOS616 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

The influence of the nonrecent past in prediction for stochastic processes

Consider the stochastic processes X1, X2,... and [Lambda]1, [Lambda]2,... where the X process can be thought of as observations on the [Lambda] process. We investigate the asymptotic behavior of the conditional distributions of Xt+v given X1,..., Xt and [Lambda]t+v given X1,..., Xt with regard to their dependency on the "early" part of the X process. These distributions arise in various time series and sequential decision theory problems. The results support the intuitively reasonable and often used (as a basic tenet of model building) assumption that only the more recent past is needed for near optimal prediction.Stochastic process prediction martingale Markov process stationary process

Admissibility of estimators of the probability of unobserved outcomes

Probability of new species, Good's estimator, Robbins' estimator, admissibility, uniformly minimum variance unbiased, uniformly minimum mean squared error unbiased, sufficiency, completeness,