Justify your alpha

Benjamin et al. proposed changing the conventional “statistical significance” threshold (i.e.,the alpha level) from p ≤ .05 to p ≤ .005 for all novel claims with relatively low prior odds. They provided two arguments for why lowering the significance threshold would “immediately improve the reproducibility of scientific research.” First, a p-value near .05provides weak evidence for the alternative hypothesis. Second, under certain assumptions, an alpha of .05 leads to high false positive report probabilities (FPRP2 ; the probability that a significant finding is a false positive

Justify your alpha

In response to recommendations to redefine statistical significance to p ≤ .005, we propose that researchers should transparently report and justify all choices they make when designing a study, including the alpha level

MP.2018.1479.Maraun

Submission to Meta-Psychology. Contains the full history, including all correspondence, decisions, reviews, revisions etc. Anyone can participate in open peer review directly by using the hypothes.is commenting function on the preprint

Peer Review Report

Peer Review Report of the submission MP.2018.87