Multiple Testing for Exploratory Research

Motivated by the practice of exploratory research, we formulate an approach to multiple testing that reverses the conventional roles of the user and the multiple testing procedure. Traditionally, the user chooses the error criterion, and the procedure the resulting rejected set. Instead, we propose to let the user choose the rejected set freely, and to let the multiple testing procedure return a confidence statement on the number of false rejections incurred. In our approach, such confidence statements are simultaneous for all choices of the rejected set, so that post hoc selection of the rejected set does not compromise their validity. The proposed reversal of roles requires nothing more than a review of the familiar closed testing procedure, but with a focus on the non-consonant rejections that this procedure makes. We suggest several shortcuts to avoid the computational problems associated with closed testing.Comment: Published in at http://dx.doi.org/10.1214/11-STS356 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Multi Split Conformal Prediction

Split conformal prediction is a computationally efficient method for performing distribution-free predictive inference in regression. It involves, however, a one-time random split of the data, and the result depends on the particular split. To address this problem, we propose multi split conformal prediction, a simple method based on Markov's inequality to aggregate single split conformal prediction intervals across multiple splits.Comment: 12 pages, 1 figure, 2 tabl

Rejoinder to "Multiple Testing for Exploratory Research"

Rejoinder to "Multiple Testing for Exploratory Research" by J. J. Goeman, A. Solari [arXiv:1208.2841].Comment: Published in at http://dx.doi.org/10.1214/11-STS356REJ the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Only Closed Testing Procedures are Admissible for Controlling False Discovery Proportions

We consider the class of all multiple testing methods controlling tail probabilities of the false discovery proportion, either for one random set or simultaneously for many such sets. This class encompasses methods controlling familywise error rate, generalized familywise error rate, false discovery exceedance, joint error rate, simultaneous control of all false discovery proportions, and others, as well as seemingly unrelated methods such as gene set testing in genomics and cluster inference methods in neuroimaging. We show that all such methods are either equivalent to a closed testing method, or are uniformly improved by one. Moreover, we show that a closed testing method is admissible as a method controlling tail probabilities of false discovery proportions if and only if all its local tests are admissible. This implies that, when designing such methods, it is sufficient to restrict attention to closed testing methods only. We demonstrate the practical usefulness of this design principle by constructing a uniform improvement of a recently proposed method

Trypanosomatid infections among vertebrates of Chile: a systematic review

We present a review on the natural infection by trypanosomatids of nonhuman vertebrates in Chile, aiming to synthesize and update the knowledge on the diversity of trypanosomatids infecting native and alien vertebrate species. To this end, we conducted a systematic review of literature records published from 1900 to April 2020 on four databases, focusing on the 21 genera of trypanosomatids and Chile. The methods and findings of our review have been based on the preferred reporting items for systematic reviews and meta-analysis (prisma) checklist. We found 29,756 records but only 71 presented relevant information for this review. Overall, there are only two reported trypanosomatid genera infecting vertebrate species in Chile, the genera Trypanosoma and Leishmania. The former is mostly represented by Trypanosoma cruzi (90% of the total records) and to a much lesser extent by Trypanosoma avium, Trypanosoma humboldti, Trypanosoma lewisi, and a couple of unidentified trypanosomatids. A total of 25 mammals have been reported as being infected by T. cruzi, including 14 native and 11 alien species from Orders Artiodactyla, Carnivora, Chiroptera, Didelphimorphia, Lagomorpha, Perissodactyla, and Rodentia. Extensive screening studies using new analytical tools are necessary to grasp the whole potential diversity of trypanosomatid species infecting vertebrates in Chile