An Open, Large-Scale, Collaborative Effort to Estimate the Reproducibility of Psychological Science

Reproducibility is a defining feature of science. However, because of strong incentives for innovation and weak incentives for confirmation, direct replication is rarely practiced or published. The Reproducibility Project is an open, large-scale, collaborative effort to systematically examine the rate and predictors of reproducibility in psychological science. So far, 72 volunteer researchers from 41 institutions have organized to openly and transparently replicate studies published in three prominent psychological journals in 2008. Multiple methods will be used to evaluate the findings, calculate an empirical rate of replication, and investigate factors that predict reproducibility. Whatever the result, a better understanding of reproducibility will ultimately improve confidence in scientific methodology and findings

An Open, Large-Scale, Collaborative Effort to Estimate the Reproducibility of Psychological Science

Reproducibility is a defining feature of science. However, because of strong incentives for innovation and weak incentives for confirmation, direct replication is rarely practiced or published. The Reproducibility Project is an open, large-scale, collaborative effort to systematically examine the rate and predictors of reproducibility in psychological science. So far, 72 volunteer researchers from 41 institutions have organized to openly and transparently replicate studies published in three prominent psychological journals in 2008. Multiple methods will be used to evaluate the findings, calculate an empirical rate of replication, and investigate factors that predict reproducibility. Whatever the result, a better understanding of reproducibility will ultimately improve confidence in scientific methodology and findings

Community-Based Cluster-Randomized Trial to Reduce Opioid Overdose Deaths

BACKGROUNDEvidence-based practices for reducing opioid-related overdose deaths include overdose education and naloxone distribution, the use of medications for the treatment of opioid use disorder, and prescription opioid safety. Data are needed on the effectiveness of a community-engaged intervention to reduce opioid-related overdose deaths through enhanced uptake of these practices.METHODSIn this community-level, cluster-randomized trial, we randomly assigned 67 communities in Kentucky, Massachusetts, New York, and Ohio to receive the intervention (34 communities) or a wait-list control (33 communities), stratified according to state. The trial was conducted within the context of both the coronavirus disease 2019 (Covid-19) pandemic and a national surge in the number of fentanyl-related overdose deaths. The trial groups were balanced within states according to urban or rural classification, previous overdose rate, and community population. The primary outcome was the number of opioid-related overdose deaths among community adults.RESULTSDuring the comparison period from July 2021 through June 2022, the population-averaged rates of opioid-related overdose deaths were similar in the intervention group and the control group (47.2 deaths per 100,000 population vs. 51.7 per 100,000 population), for an adjusted rate ratio of 0.91 (95% confidence interval, 0.76 to 1.09; P = 0.30). The effect of the intervention on the rate of opioid-related overdose deaths did not differ appreciably according to state, urban or rural category, age, sex, or race or ethnic group. Intervention communities implemented 615 evidence-based practice strategies from the 806 strategies selected by communities (254 involving overdose education and naloxone distribution, 256 involving the use of medications for opioid use disorder, and 105 involving prescription opioid safety). Of these evidence-based practice strategies, only 235 (38%) had been initiated by the start of the comparison year.CONCLUSIONSIn this 12-month multimodal intervention trial involving community coalitions in the deployment of evidence-based practices to reduce opioid overdose deaths, death rates were similar in the intervention group and the control group in the context of the Covid-19 pandemic and the fentanyl-related overdose epidemic. (Funded by the National Institutes of Health; HCS ClinicalTrials.gov number, NCT04111939.)

Reproducibility Project: Psychology

Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. We conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available

Data from: Estimating the reproducibility of psychological science

This record contains the underlying research data for the publication "Estimating the reproducibility of psychological science" and the full-text is available from: https://ink.library.smu.edu.sg/lkcsb_research/5257Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. We conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available. Replication effects were half the magnitude of original effects, representing a substantial decline. Ninety-seven percent of original studies had statistically significant results. Thirty-six percent of replications had statistically significant results; 47% of original effect sizes were in the 95% confidence interval of the replication effect size; 39% of effects were subjectively rated to have replicated the original result; and if no bias in original results is assumed, combining original and replication results left 68% with statistically significant effects. Correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams