Statistical reproducibility for pairwise t-tests in pharmaceutical research

This paper investigates statistical reproducibility of the t-test. We formulate reproducibility as a predictive inference problem and apply the nonparametric predictive inference (NPI) method. Within our research framework, statistical reproducibility provides inference on the probability that the same test outcome would be reached, if the test were repeated under identical conditions. We present an NPI algorithm to calculate the reproducibility of the t-test and then use simulations to explore the reproducibility both under the null and alternative hypotheses. We then apply NPI reproducibility to a real life scenario of a preclinical experiment, which involves multiple pairwise comparisons of test groups, where different groups are given a different concentration of a drug. The aim of the experiment is to decide the concentration of the drug which is most effective. In both simulations and the application scenario, we study the relationship between reproducibility and two test statistics, the Cohen’s d and the p-value. We also compare the reproducibility of the t-test with the reproducibility of the Wilcoxon Mann-Whitney test. Finally, we examine reproducibility for the final decision of choosing a particular dose in the multiple pairwise comparisons scenario. This paper presents advances on the topic of test reproducibility with relevance for tests used in pharmaceutical research

Impact of temporal variation on design and analysis of mouse knockout phenotyping studies.

A significant challenge facing high-throughput phenotyping of in-vivo knockout mice is ensuring phenotype calls are robust and reliable. Central to this problem is selecting an appropriate statistical analysis that models both the experimental design (the workflow and the way control mice are selected for comparison with knockout animals) and the sources of variation. Recently we proposed a mixed model suitable for small batch-oriented studies, where controls are not phenotyped concurrently with mutants. Here we evaluate this method both for its sensitivity to detect phenotypic effects and to control false positives, across a range of workflows used at mouse phenotyping centers. We found the sensitivity and control of false positives depend on the workflow. We show that the phenotypes in control mice fluctuate unexpectedly between batches and this can cause the false positive rate of phenotype calls to be inflated when only a small number of batches are tested, when the effect of knockout becomes confounded with temporal fluctuations in control mice. This effect was observed in both behavioural and physiological assays. Based on this analysis, we recommend two approaches (workflow and accompanying control strategy) and associated analyses, which would be robust, for use in high-throughput phenotyping pipelines. Our results show the importance in modelling all sources of variability in high-throughput phenotyping studies

Genetic determinants of micronucleus formation in vivo

Genomic instability arising from defective responses to DNA damage or mitotic chromosomal imbalances can lead to the sequestration of DNA in aberrant extranuclear structures called micronuclei (MN). Although MN are a hallmark of ageing and diseases associated with genomic instability, the catalogue of genetic players that regulate the generation of MN remains to be determined. Here we analyse 997 mouse mutant lines, revealing 145 genes whose loss significantly increases (n = 71) or decreases (n = 74) MN formation, including many genes whose orthologues are linked to human disease. We found that mice null for Dscc1, which showed the most significant increase in MN, also displayed a range of phenotypes characteristic of patients with cohesinopathy disorders. After validating the DSCC1-associated MN instability phenotype in human cells, we used genome-wide CRISPR-Cas9 screening to define synthetic lethal and synthetic rescue interactors. We found that the loss of SIRT1 can rescue phenotypes associated with DSCC1 loss in a manner paralleling restoration of protein acetylation of SMC3. Our study reveals factors involved in maintaining genomic stability and shows how this information can be used to identify mechanisms that are relevant to human disease biology