1,478 research outputs found
Multiple Testing for Exploratory Research
Motivated by the practice of exploratory research, we formulate an approach
to multiple testing that reverses the conventional roles of the user and the
multiple testing procedure. Traditionally, the user chooses the error
criterion, and the procedure the resulting rejected set. Instead, we propose to
let the user choose the rejected set freely, and to let the multiple testing
procedure return a confidence statement on the number of false rejections
incurred. In our approach, such confidence statements are simultaneous for all
choices of the rejected set, so that post hoc selection of the rejected set
does not compromise their validity. The proposed reversal of roles requires
nothing more than a review of the familiar closed testing procedure, but with a
focus on the non-consonant rejections that this procedure makes. We suggest
several shortcuts to avoid the computational problems associated with closed
testing.Comment: Published in at http://dx.doi.org/10.1214/11-STS356 the Statistical
Science (http://www.imstat.org/sts/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Rejoinder to "Multiple Testing for Exploratory Research"
Rejoinder to "Multiple Testing for Exploratory Research" by J. J. Goeman, A.
Solari [arXiv:1208.2841].Comment: Published in at http://dx.doi.org/10.1214/11-STS356REJ the
Statistical Science (http://www.imstat.org/sts/) by the Institute of
Mathematical Statistics (http://www.imstat.org
- …