Reproducibility of preclinical animal research improves with heterogeneity of study samples

Single-laboratory studies conducted under highly standardized conditions are the gold standard in preclinical animal research. Using simulations based on 440 preclinical studies across 13 different interventions in animal models of stroke, myocardial infarction, and breast cancer, we compared the accuracy of effect size estimates between single-laboratory and multi-laboratory study designs. Single-laboratory studies generally failed to predict effect size accurately, and larger sample sizes rendered effect size estimates even less accurate. By contrast, multi-laboratory designs including as few as 2 to 4 laboratories increased coverage probability by up to 42 percentage points without a need for larger sample sizes. These findings demonstrate that within-study standardization is a major cause of poor reproducibility. More representative study samples are required to improve the external validity and reproducibility of preclinical animal research and to prevent wasting animals and resources for inconclusive research

IL-12Rβ1 Deficiency in Two of Fifty Children with Severe Tuberculosis from Iran, Morocco, and Turkey

BACKGROUND AND OBJECTIVES: In the last decade, autosomal recessive IL-12Rβ1 deficiency has been diagnosed in four children with severe tuberculosis from three unrelated families from Morocco, Spain, and Turkey, providing proof-of-principle that tuberculosis in otherwise healthy children may result from single-gene inborn errors of immunity. We aimed to estimate the fraction of children developing severe tuberculosis due to IL-12Rβ1 deficiency in areas endemic for tuberculosis and where parental consanguinity is common. METHODS AND PRINCIPAL FINDINGS: We searched for IL12RB1 mutations in a series of 50 children from Iran, Morocco, and Turkey. All children had established severe pulmonary and/or disseminated tuberculosis requiring hospitalization and were otherwise normally resistant to weakly virulent BCG vaccines and environmental mycobacteria. In one child from Iran and another from Morocco, homozygosity for loss-of-function IL12RB1 alleles was documented, resulting in complete IL-12Rβ1 deficiency. Despite the small sample studied, our findings suggest that IL-12Rβ1 deficiency is not a very rare cause of pediatric tuberculosis in these countries, where it should be considered in selected children with severe disease. SIGNIFICANCE: This finding may have important medical implications, as recombinant IFN-γ is an effective treatment for mycobacterial infections in IL-12Rβ1-deficient patients. It also provides additional support for the view that severe tuberculosis in childhood may result from a collection of single-gene inborn errors of immunity

The Researchers' View of Scientific Rigor Survey on the Conduct and Reporting of In Vivo Research

Reproducibility in animal research is alarmingly low, and a lack of scientific rigor has been proposed as a major cause. Systematic reviews found low reporting rates of measures against risks of bias (e.g., randomization, blinding), and a correlation between low reporting rates and overstated treatment effects. Reporting rates of measures against bias are thus used as a proxy measure for scientific rigor, and reporting guidelines (e.g., ARRIVE) have become a major weapon in the fight against risks of bias in animal research. Surprisingly, animal scientists have never been asked about their use of measures against risks of bias and how they report these in publications. Whether poor reporting reflects poor use of such measures, and whether reporting guidelines may effectively reduce risks of bias has therefore remained elusive. To address these questions, we asked in vivo researchers about their use and reporting of measures against risks of bias and examined how self-reports relate to reporting rates obtained through systematic reviews. An online survey was sent out to all registered in vivo researchers in Switzerland (N = 1891) and was complemented by personal interviews with five representative in vivo researchers to facilitate interpretation of the survey results. Return rate was 28% (N = 530), of which 302 participants (16%) returned fully completed questionnaires that were used for further analysis. According to the researchers' self-report, they use measures against risks of bias to a much greater extent than suggested by reporting rates obtained through systematic reviews. However, the researchers' self-reports are likely biased to some extent. Thus, although they claimed to be reporting measures against risks of bias at much lower rates than they claimed to be using these measures, the self-reported reporting rates were considerably higher than reporting rates found by systematic reviews. Furthermore, participants performed rather poorly when asked to choose effective over ineffective measures against six different biases. Our results further indicate that knowledge of the ARRIVE guidelines had a positive effect on scientific rigor. However, the ARRIVE guidelines were known by less than half of the participants (43.7%); and among those whose latest paper was published in a journal that had endorsed the ARRIVE guidelines, more than half (51%) had never heard of these guidelines. Our results suggest that whereas reporting rates may underestimate the true use of measures against risks of bias, self-reports may overestimate it. To a large extent, this discrepancy can be explained by the researchers' ignorance and lack of knowledge of risks of bias and measures to prevent them. Our analysis thus adds significant new evidence to the assessment of research integrity in animal research. Our findings further question the confidence that the authorities have in scientific rigor, which is taken for granted in the harm-benefit analyses on which approval of animal experiments is based. Furthermore, they suggest that better education on scientific integrity and good research practice is needed. However, they also question reliance on reporting rates as indicators of scientific rigor and highlight a need for more reliable predictors

Authorization of Animal Experiments Is Based on Confidence Rather than Evidence of Scientific Rigor.

Accumulating evidence indicates high risk of bias in preclinical animal research, questioning the scientific validity and reproducibility of published research findings. Systematic reviews found low rates of reporting of measures against risks of bias in the published literature (e.g., randomization, blinding, sample size calculation) and a correlation between low reporting rates and inflated treatment effects. That most animal research undergoes peer review or ethical review would offer the possibility to detect risks of bias at an earlier stage, before the research has been conducted. For example, in Switzerland, animal experiments are licensed based on a detailed description of the study protocol and a harm-benefit analysis. We therefore screened applications for animal experiments submitted to Swiss authorities (n = 1,277) for the rates at which the use of seven basic measures against bias (allocation concealment, blinding, randomization, sample size calculation, inclusion/exclusion criteria, primary outcome variable, and statistical analysis plan) were described and compared them with the reporting rates of the same measures in a representative sub-sample of publications (n = 50) resulting from studies described in these applications. Measures against bias were described at very low rates, ranging on average from 2.4% for statistical analysis plan to 19% for primary outcome variable in applications for animal experiments, and from 0.0% for sample size calculation to 34% for statistical analysis plan in publications from these experiments. Calculating an internal validity score (IVS) based on the proportion of the seven measures against bias, we found a weak positive correlation between the IVS of applications and that of publications (Spearman's rho = 0.34, p = 0.014), indicating that the rates of description of these measures in applications partly predict their rates of reporting in publications. These results indicate that the authorities licensing animal experiments are lacking important information about experimental conduct that determines the scientific validity of the findings, which may be critical for the weight attributed to the benefit of the research in the harm-benefit analysis. Similar to manuscripts getting accepted for publication despite poor reporting of measures against bias, applications for animal experiments may often be approved based on implicit confidence rather than explicit evidence of scientific rigor. Our findings shed serious doubt on the current authorization procedure for animal experiments, as well as the peer-review process for scientific publications, which in the long run may undermine the credibility of research. Developing existing authorization procedures that are already in place in many countries towards a preregistration system for animal research is one promising way to reform the system. This would not only benefit the scientific validity of findings from animal experiments but also help to avoid unnecessary harm to animals for inconclusive research

Maternal vitamin C deficiency during pregnancy persistently impairs hippocampal neurogenesis in offspring of guinea pigs

While having the highest vitamin C (VitC) concentrations in the body, specific functions of VitC in the brain have only recently been acknowledged. We have shown that postnatal VitC deficiency in guinea pigs causes impairment of hippocampal memory function and leads to 30% less neurons. This study investigates how prenatal VitC deficiency affects postnatal hippocampal development and if any such effect can be reversed by postnatal VitC repletion. Eighty pregnant Dunkin Hartley guinea pig dams were randomized into weight stratified groups receiving High (900 mg) or Low (100 mg) VitC per kg diet. Newborn pups (n = 157) were randomized into a total of four postnatal feeding regimens: High/High (Control); High/Low (Depleted), Low/Low (Deficient); and Low/High (Repleted). Proliferation and migration of newborn cells in the dentate gyrus was assessed by BrdU labeling and hippocampal volumes were determined by stereology. Prenatal VitC deficiency resulted in a significant reduction in postnatal hippocampal volume (P<0.001) which was not reversed by postnatal repletion. There was no difference in postnatal cellular proliferation and survival rates in the hippocampus between dietary groups, however, migration of newborn cells into the granular layer of the hippocampus dentate gyrus was significantly reduced in prenatally deficient animals (P<0.01). We conclude that a prenatal VitC deficiency in guinea pigs leads to persistent impairment of postnatal hippocampal development which is not alleviated by postnatal repletion. Our findings place attention on a yet unrecognized consequence of marginal VitC deficiency during pregnancy

Original Data of Reichlin et al PLoS One 2016

Reproducibility in animal research is alarmingly low, and a lack of scientific rigor has been proposed as a major cause. Systematic reviews found low reporting rates of measures against risks of bias (e.g., randomization, blinding), and a correlation between low reporting rates and overstated treatment effects. Reporting rates of measures against bias are thus used as a proxy measure for scientific rigor, and reporting guidelines (e.g., ARRIVE) have become a major weapon in the fight against risks of bias in animal research. Surprisingly, animal scientists have never been asked about their use of measures against risks of bias and how they report these in publications. Whether poor reporting reflects poor use of such measures, and whether reporting guidelines may effectively reduce risks of bias has therefore remained elusive. To address these questions, we asked in vivo researchers about their use and reporting of measures against risks of bias and examined how self-reports relate to reporting rates obtained through systematic reviews. An online survey was sent out to all registered in vivo researchers in Switzerland (N=1891) and was complemented by personal interviews with five representative in vivo researchers to facilitate interpretation of the survey results

List of measures against risks of bias included in this study.

<p>List of measures against risks of bias included in this study.</p

Boxplot of IVS versus certification of institution.

<p>Comparison of IVS for (A) experimental conduct, and (B) for the reporting in publications with respect to working institutions being certified. Mean IVS are slightly but non-significantly higher for participants working for certified institutions.</p

Boxplot of IVS versus descriptors of model selection process.

<p>Descriptors are selected for the models with lowest BIC, thus best explaining the variation in IVS for (A) experimental conduct and (B) for publications. For (A) one value is missing as no IVS could be calculated, and for (B) 41 values are missing, because participants ticked ‘have not yet published’, gave ‘no answer’ or declared that these questions ‘do not apply to last manuscript’. For experimental conduct (A), the model including ARRIVE knowledge and research field best explained the IVS_Exp, whereas for publications (B), the model including only ARRIVE knowledge best explained the IVS_Pub. Red squares indicate the mean IVS; black circle the mean of IVS_Exp of participants with ARRIVE knowledge; grey triangle the mean IVS_Exp of participants without ARRIVE knowledge. Whiskers are 1.5*interquartile range.</p

Prevalence of the measures used and reported to avoid risks of bias by the participants to online survey.

<p>(A) Prevalence of use of bias avoidance measures during experimental conduct and reporting in the participants’ latest publication (percentages are corrected for ‘no answer’, ‘does not apply to last manuscript’ and ‘have not published so far’. (B) Internal validity scores (IVS) for experimental conduct and reporting in publications.</p