Justify your alpha

Benjamin et al. proposed changing the conventional “statistical significance” threshold (i.e.,the alpha level) from p ≤ .05 to p ≤ .005 for all novel claims with relatively low prior odds. They provided two arguments for why lowering the significance threshold would “immediately improve the reproducibility of scientific research.” First, a p-value near .05provides weak evidence for the alternative hypothesis. Second, under certain assumptions, an alpha of .05 leads to high false positive report probabilities (FPRP2 ; the probability that a significant finding is a false positive

The Practical Alternative to the p Value Is the Correctly Used p Value

Because of the strong overreliance on p values in the scientific literature, some researchers have argued that we need to move beyond p values and embrace practical alternatives. When proposing alternatives to p values statisticians often commit the “statistician’s fallacy,” whereby they declare which statistic researchers really “want to know.” Instead of telling researchers what they want to know, statisticians should teach researchers which questions they can ask. In some situations, the answer to the question they are most interested in will be the p value. As long as null-hypothesis tests have been criticized, researchers have suggested including minimum-effect tests and equivalence tests in our statistical toolbox, and these tests have the potential to greatly improve the questions researchers ask. If anyone believes p values affect the quality of scientific research, preventing the misinterpretation of p values by developing better evidence-based education and user-centered statistical software should be a top priority. Polarized discussions about which statistic scientists should use has distracted us from examining more important questions, such as asking researchers what they want to know when they conduct scientific research. Before we can improve our statistical inferences, we need to improve our statistical questions

Improving Your Statistical Questions by Daniel Lakens Assignment Workstation

This is a workstation that can be used to perform all R computations for the Coursera course "Improving Your Statistical Questions" by Daniel Lakens from Eindhoven University of Technology. It contains all the R scripts for the assignments, and all requires R packages are preloaded

Index file, cascading style sheet, errata and copyright statements accompanying the book "The Perceptual Structure of Sound"

Index file (index.html), cascading style sheet (PSS.css), errata (PSS-errata.html), and copyright statements (copyrightDJH.html andcopyrightPRDJH.html) for the website : "https://dhermes.ieis.tue.nl/PSS" with links to the sound files (in the subdirectory wavFiles), used as auditory demonstrations for the book "The Perceptual Structure of Sound", and the html files (in the subdirectory htmlFiles) showing the Matlab scripts with which these sounds are generate

For whom the gamer trolls: A study of trolling interactions in the online gaming context

Original, unedited trolling case reports used in hybrid traditional/computational machine learning paper accepted for publication by JCMC in June 2019

Crud (Re)Defined

The idea that in behavioral research everything correlates with everything else was a niche area of the scientific literature for more than half a century. With the increasing availability of large data sets in psychology, the “crud” factor has, however, become more relevant than ever before. When referenced in empirical work, it is often used by researchers to discount minute—but statistically significant—effects that are deemed too small to be considered meaningful. This review tracks the history of the crud factor and examines how its use in the psychological- and behavioral-science literature has developed to this day. We highlight a common and deep-seated lack of understanding about what the crud factor is and discuss whether it can be proven to exist or estimated and how it should be interpreted. This lack of understanding makes the crud factor a convenient tool for psychologists to use to disregard unwanted results, even though the presence of a crud factor should be a large inconvenience for the discipline. To inspire a concerted effort to take the crud factor more seriously, we clarify the definitions of important concepts, highlight current pitfalls, and pose questions that need to be addressed to ultimately improve understanding of the crud factor. Such work will be necessary to develop the crud factor into a useful concept encouraging improved psychological research

Equivalence Testing for Psychological Research: A Tutorial

Psychologists must be able to test both for the presence of an effect and for the absence of an effect. In addition to testing against zero, researchers can use the two one-sided tests (TOST) procedure to test for equivalence and reject the presence of a smallest effect size of interest (SESOI). The TOST procedure can be used to determine if an observed effect is surprisingly small, given that a true effect at least as extreme as the SESOI exists. We explain a range of approaches to determine the SESOI in psychological science and provide detailed examples of how equivalence tests should be performed and reported. Equivalence tests are an important extension of the statistical tools psychologists currently use and enable researchers to falsify predictions about the presence, and declare the absence, of meaningful effects

Public Benchmark Dataset for Testing rPPG Algorithm Performance

The current needs in remote photoplethysmography (rPPG) research were investigated and it was concluded there is a lack of reproducibility and comparability in the current rPPG research. Three main challenges for rPPG algorithms were defined and incorporated in the dataset; lighting & skin tone, motion robustness and high heart rates & pulse-rate change robustness. Videos accompanied by ECG measurements were recorded to create test material for these challenges

Reviewers’ Decision to Sign Reviews is Related to Their Recommendation

Surveys indicate that researchers generally have a positive attitude towards open peer review when this consists of making reviews available alongside published articles. Researchers are more negative about revealing the identity of reviewers. They worry reviewers will be less likely to express criticism if their identity is known to authors. Experiments suggest that reviewers are somewhat less likely to recommend rejection when they are told their identity will be communicated to authors, than when they will remain anonymous. One recent study revealed reviewers in five journals who voluntarily signed their reviews gave more positive recommendations than those who did not sign their reviews. We replicate and extend this finding by analyzing 12010 open reviews in PeerJ and 4188 reviews in the Royal Society Open Science where authors can voluntarily sign their reviews. These results based on behavioral data from real peer reviews across a wide range of scientific disciplines demonstrate convincingly that reviewers' decision to sign is related to their recommendation. The proportion of signed reviews was higher for more positive recommendations, than for more negative recommendations. We also share all 23649 text mined reviews as raw data underlying our results that can be re-used by researchers interested in peer review

An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports

Selectively publishing results that support the tested hypotheses (“positive” results) distorts the available evidence for scientific claims. For the past decade, psychological scientists have been increasingly concerned about the degree of such distortion in their literature. A new publication format has been developed to prevent selective reporting: In Registered Reports (RRs), peer review and the decision to publish take place before results are known. We compared the results in published RRs (N = 71 as of November 2018) with a random sample of hypothesis-testing studies from the standard literature (N = 152) in psychology. Analyzing the first hypothesis of each article, we found 96% positive results in standard reports but only 44% positive results in RRs. We discuss possible explanations for this large difference and suggest that a plausible factor is the reduction of publication bias and/or Type I error inflation in the RR literature