Recent developments towards optimality in multiple hypothesis testing

There are many different notions of optimality even in testing a single hypothesis. In the multiple testing area, the number of possibilities is very much greater. The paper first will describe multiplicity issues that arise in tests involving a single parameter, and will describe a new optimality result in that context. Although the example given is of minimal practical importance, it illustrates the crucial dependence of optimality on the precise specification of the testing problem. The paper then will discuss the types of expanded optimality criteria that are being considered when hypotheses involve multiple parameters, will note a few new optimality results, and will give selected theoretical references relevant to optimality considerations under these expanded criteria.Comment: Published at http://dx.doi.org/10.1214/074921706000000374 in the IMS Lecture Notes--Monograph Series (http://www.imstat.org/publications/lecnotes.htm) by the Institute of Mathematical Statistics (http://www.imstat.org

Confidence Intervals On Subsets May Be Misleading

A combination of hypothesis testing and confidence interval construction is often used in social and behavioral science studies. Sometimes confidence intervals are computed or reported only if a null hypothesis is rejected, perhaps to see whether the range of values is of practical importance. Sometimes they are constructed or reported only if a null hypothesis is accepted, in order to assess the range of plausible nonnull values due to inadequate power to detect them. Even if always computed, they are interpreted differently, depending on whether the null value is or is not included. Furthermore, many studies in which the null hypothesis is not rejected are never published (the “file drawer” problem). This article discusses the coverage probability of nominal 1− α confidence intervals when examining intervals that do or do not cover some specified null value, usually zero. A briefer treatment considers interval coverage when undesirable results are suppressed. The coverage probability of such conditional confidence intervals may be very far from the nominal value. The magnitude of the effect of selection on interval coverage probability and possible resultant biases in inference are illustrated, and discussed in relation to effect sizes of importance in social and behavioral science research and to estimation of effect sizes

On Optimality of Stepdown and Stepup Multiple Test Procedures

Consider the multiple testing problem of testing k null hypotheses, where the unknown family of distributions is assumed to satisfy a certain monotonicity assumption. Attention is restricted to procedures that control the familywise error rate in the strong sense and which satisfy a monotonicity condition. Under these assumptions, we prove certain maximin optimality results for some well-known stepdown and stepup procedures.Comment: Published at http://dx.doi.org/10.1214/009053605000000066 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Multiple Hypothesis Testing in Microarray Experiments

DNA microarrays are a new and promising biotechnology which allows the monitoring of expression levels in cells for thousands of genes simultaneously. An important and common question in microarray experiments is the identification of differentially expressed genes, i.e., genes whose expression levels are associated with a response or covariate of interest. The biological question of differential expression can be restated as a problem in multiple hypothesis testing: the simultaneous test for each gene of the null hypothesis of no association between the expression levels and the responses or covariates. As a typical microarray experiment measures expression levels for thousands of genes simultaneously, large multiplicity problems are generated. This article discusses different approaches to multiple hypothesis testing in the context of microarray experiments and compares the procedures on microarray and simulated datasets

Sentiment analysis with genetically evolved Gaussian kernels

Sentiment analysis consists of evaluating opinions or statements based on text analysis. Among the methods used to estimate the degree to which a text expresses a certain sentiment are those based on Gaussian Processes. However, traditional Gaussian Processes methods use a prede- fined kernels with hyperparameters that can be tuned but whose structure can not be adapted. In this paper, we propose the application of Genetic Programming for the evolution of Gaussian Process kernels that are more precise for sentiment analysis. We use use a very flexible representation of kernels combined with a multi-objective approach that considers si- multaneously two quality metrics and the computational time required to evaluate those kernels. Our results show that the algorithm can outper- form Gaussian Processes with traditional kernels for some of the sentiment analysis tasks considered