Genotypic variability enhances the reproducibility of an ecological study

Many scientific disciplines are currently experiencing a “reproducibility crisis” because numerous scientific findings cannot be repeated consistently. A novel but controversial hypothesis postulates that stringent levels of environmental and biotic standardization in experimental studies reduces reproducibility by amplifying impacts of lab-specific environmental factors not accounted for in study designs. A corollary to this hypothesis is that a deliberate introduction of controlled systematic variability (CSV) in experimental designs may lead to increased reproducibility. We tested this hypothesis using a multi-laboratory microcosm study in which the same ecological experiment was repeated in 14 laboratories across Europe. Each laboratory introduced environmental and genotypic CSV within and among replicated microcosms established in either growth chambers (with stringent control of environmental conditions) or glasshouses (with more variable environmental conditions). The introduction of genotypic CSV led to lower among-laboratory variability in growth chambers, indicating increased reproducibility, but had no significant effect in glasshouses where reproducibility was generally lower. Environmental CSV had little effect on reproducibility. Although there are multiple causes for the “reproducibility crisis”, deliberately including genetic variation may be a simple solution for increasing the reproducibility of ecological studies performed in controlled environments

Data from 14 labortories testing the impact of introduced variability on the reproducibility of a microcosm ecological experiment

Although microcosm experiments are a frequent tool used to address fundamental ecological questions, there has been no quantitative assessment of the reproducibility of any microcosm experiment. This dataset contains the response variables measured in a multi-laboratory microcosm study in which the same microcosm experiment was repeated in 14 laboratories across Europe. All laboratories simultaneously run a simple microcosm experiment using grass (Brachypodium distachyon L.) monocultures and grass and legume (Medicago truncatula Gaertn.) mixtures. All twelve variables were then used to calculate the effect of the presence of nitrogen-fixing legume on the grass-legume mixtures (i.e. the net legume effect). The project tested a controversial hypotheses postulating that stringent levels of environmental and biotic standardization in experimental studies reduces reproducibility by amplifying impacts of lab-specific environmental factors not accounted for in the experimental design. This implies that the deliberate introduction of controlled systematic variability (CSV) in experimental designs can increase reproducibility. To test this hypothesis, each laboratory followed the same experimental protocol and introduced environmental and genotypic controlled systematic variability (CSV) within and among replicated microcosms established in either growth chambers (with stringent control of environmental conditions) or glasshouses (with more variable environmental conditions). Data were used to test the extent to which the effect size of the net legume effect varied with the CSV treatment and to estimate the number of laboratories that produced results that can be considered reproducible