Skip to main content
Article thumbnail
Location of Repository

Pseudoreplication invalidates the results of many neuroscientific studies

By Stanley E. Lazic

Abstract

Background: Pseudoreplication occurs when observations are not statistically independent, but treated as if they are. This can occur when there are multiple observations on the same subjects, when samples are nested or hierarchically organised, or when measurements are correlated in time or space. Analysis of such data without taking these dependencies into account can lead to meaningless results, and examples can easily be found in the neuroscience literature.\ud \ud Results: A single issue of Nature Neuroscience provided a number of examples and is used as a case study to highlight how pseudoreplication arises in neuroscientific studies, why the analyses in these papers are incorrect, and appropriate analytical methods are provided. 12% of papers had pseudoreplication and a further 36% were suspected of having pseudoreplication, but it was not possible to determine for certain because insufficient information about the analysis was provided.\ud \ud Conclusions: Pseudoreplication undermines the conclusions from statistical analysis of data, and it would be easier to detect if the sample size, degrees of freedom, the test statistic, and precise p-values are reported. This information should be a requirement for all publications

Topics: General
Year: 2009
OAI identifier: oai:generic.eprints.org:945/core69

Suggested articles

Citations

  1. (2004). Alcaraz C: Incongruence between test statistics and P values in medical papers.
  2. (2008). Alonso JM: Task difficulty modulates the activity of specific neuronal populations in primary visual cortex. Nat Neurosci
  3. (1995). Altman DG: Calculating correlation coefficients with repeated observations: Part 1–Correlation within subjects. BMJ
  4. (1995). Altman DG: Calculating correlation coefficients with repeated observations: Part 2–Correlation between subjects. BMJ
  5. (1994). Altman DG: Correlation, regression, and repeated data. BMJ
  6. (1990). Analysis of serial measurements in medical research. BMJ
  7. (2008). and interpretation of studies in the standard murine model of ALS. Amyotroph Lateral Scler
  8. (2000). Bates DM: Mixed-Effects Models in S and S-Plus.
  9. (2007). Benos DJ: Guidelines for reporting statistics in journals published by the American Physiological Society: the sequel. Adv Physiol Educ
  10. (2004). Benos DJ: Guidelines for reporting statistics in journals published by the American Physiological Society. Adv Physiol Educ
  11. (2007). CC: Pharmacotherapy for cognitive impairment in a mouse model of Down syndrome.
  12. (2008). Chaouloff F: The stress hormone corticosterone conditions AMPAR surface trafficking and synaptic potentiation. Nat Neurosci
  13. (2009). CI: Circular analysis in systems neuroscience: the dangers of double dipping.
  14. (2001). CONSORT GROUP (Consolidated Standards of Reporting Trials): The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med
  15. (2001). CONSORT: The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials.
  16. (2008). Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing,
  17. (1988). Corrections: The effects of violations of independence assumptions in the one-way ANOVA. American Statistician
  18. (2006). Design and statistical methods in studies using animal models of development.
  19. (2007). DL: Error bars in experimental biology.
  20. (2009). Docosahexaenoic acid pretreatment confers neuroprotection in a rat model of perinatal cerebral hypoxia-ischemia.
  21. (2002). Does animal experimentation inform human healthcare? Observations from a systematic review of international animal experiments on fluid resuscitation. BMJ
  22. (2007). EJ: Molecular adaptations underlying susceptibility and resistance to social defeat in brain reward regions. Cell
  23. (2006). Error in statistical tests of error in statistical tests.
  24. (2006). Extending the Linear Model with R: Generalized Linear, Mixed Effects and Nonparametric Regression Models.
  25. (2007). FH: Synapse formation on neurons born in the adult hippocampus. Nat Neurosci
  26. (2008). Fritschy JM: Reporting statistical methods and statistical results in EJN.
  27. (2002). G: Statistical Rules of Thumb.
  28. (1982). Henning SJ: Diurnal rhythm of total and free concentrations of serum corticosterone in the rat. Acta Endocrinol (Copenh)
  29. (2008). Hobert O: Oxygen levels affect axon guidance and neuronal migration in Caenorhabditis elegans. Nat Neurosci
  30. (1975). Hogan MD: Selection of the experimental unit in teratology studies. Teratology
  31. (2008). Holtzman DM: Amyloid-beta dynamics correlate with neurological status in the injured human brain. Science
  32. (1997). I: Different physiological traits underlying increased body fat of fatty (fa/fa) and heterozygous (+/fa) rats.
  33. (2008). Increased dopamine after mating impairs olfaction and prevents odor interference with pregnancy. Nat Neurosci
  34. (1995). JA: Codominant effects of the fatty (fa) gene during early development of obesity.
  35. (2004). JH: Move over ANOVA: progress in analyzing repeated-measures data and its reflection in papers published in the Archives of General Psychiatry. Arch Gen Psychiatry
  36. (1987). JM: The effects of violations of independence assumptions in the one-way ANOVA. American Statistician
  37. (2008). JS: (Errors in statistical tests)3. Emerg Themes Epidemiol
  38. (2007). Khan KS: Comparison of treatment effects between animal experiments and clinical trials: systematic review. BMJ
  39. (2000). Lack of effect of an early stressful life event on sensorimotor gating in adult rats. Schizophr Res
  40. (1997). Leibel RL: The leptin receptor mediates apparent autocrine regulation of leptin gene expression. Biochem Biophys Res Commun
  41. (2004). MJ: Remedies for pseudoreplication. Fish Res
  42. (2008). Neuroscience: standard model.
  43. (1997). of Medical Journal Editors: Uniform requirements for manuscripts submitted to biomedical journals. Ann Intern Med
  44. (2004). On multilevel modeling of data from repeated measures designs: a tutorial. Speech Comm
  45. (2003). Principles: the need for better experimental design. Trends Pharmacol Sci
  46. (1984). Pseudoreplication and the design of ecological field experiments. Ecol Monogr
  47. (2003). Pseudoreplication in ecological research: the problem overlooked by Russian scientists.
  48. (1996). R: a language for data analysis and graphics.
  49. (1996). Reilly CK: Pseudoreplication Revisited. Ecology
  50. (2001). Remodeling of synaptic membranes after induction of long-term potentiation.
  51. (2002). Roberts I: Systematic reviews of animal experiments. Lancet
  52. (2008). Schinder AF: Neurons born in the adult dentate gyrus form functional synapses with target cells. Nat Neurosci
  53. (2003). Standardization and statistical approaches to therapeutic trials in the R6/2 mouse. Brain Res Bull
  54. (2007). Starcevic V: Inconsistencies between reported test statistics and p-values in two psychiatry journals.
  55. (2004). Statistical analyses of repeated measures in physiological research: a tutorial. Adv Physiol Educ
  56. Statistical Computing: An Introduction to Data Analysis using S-Plus.
  57. (2009). Statistical evaluation of methods for quantifying gene expression by autoradiography in histological sections.
  58. (1998). Statistical reviewing for medical journals. Stat Med
  59. (2007). Studies in animals should be more like those in humans. BMJ
  60. (2003). T: A mouse model of Rubinstein-Taybi syndrome: defective long-term memory is ameliorated by inhibitors of phosphodiesterase 4.
  61. (2008). T: Pikachurin, a dystroglycan ligand, is essential for photoreceptor ribbon synapse formation. Nat Neurosci
  62. (2007). Talman WT: A broader view of animal research.
  63. (2005). The Horizon homeopathic dilution experiment. Significance
  64. (2007). The R Book.
  65. (1994). The scandal of poor medical research. BMJ
  66. (2008). The temporal precision of reward prediction in dopamine neurons. Nat Neurosci
  67. (2007). Translating animal research into clinical benefit. BMJ
  68. (1993). White MD: Experiments with freshwater invertebrate zooplanktivores: quality of statistical analysis.
  69. (2005). Why most published research findings are false. PLoS Med
  70. (2008). Why we should use simpler models if the data allow this: relevance for ANOVA designs in experimental biology.

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.