The logic of p-values

Wagenmakers et al. addressed the illogic use of p-values in 'Psychological Science under Scrutiny'. While historical criticisms mostly deal with the illogical nature of null hypothesis significance testing (NHST), Wagenmakers et al. generalize such argumentation to the p-value itself. Unfortunately, Wagenmakers et al. misinterpret the formal logic basis of tests of significance (and, by extension, of tests of acceptance). This article highlights three instances where such logical interpretation fails and provides plausible corrections and further clarification

Bayesian Statistics the Fun Way: Understanding Statistics and Probability With Star Wars, Lego, and Rubber Ducks

Publishe

Parafoveal preview effects from word N+1 and word N+2 during reading: A critical review and Bayesian meta-analysis

The use of gaze-contingent display techniques to study reading has shown that readers attend not only to the currently fixated word, but also to the word to the right of the current fixation. However, a critical look at the literature shows that there are a number of questions that cannot be readily answered from the available literature reviews on the topic. First, there is no consensus on whether readers also attend to the second word to the right of fixation. Second, it is not clear whether parafoveal processing is more efficient in languages such as Chinese. Third, it is not well understood whether the measured effects are confounded by the properties of the parafoveal mask. The present study addressed these issues by performing a Bayesian meta-analysis of 93 experiments that used the boundary paradigm (Rayner, 1975). There were three main findings: 1) the advantage of previewing the second word to the right is modest in size and likely not centred on zero; 2) Chinese readers seem to make a more efficient use of parafoveal processing, but this is mostly evident in gaze duration; 3) there are interference effects associated with using different parafoveal masks that roughly increase when the mask is less word-like

Statistical Sensitiveness for the Behavioral Sciences

Research often necessitates of samples, yet obtaining large enough samples is not always possible. When it is, the researcher may use one of two methods for deciding upon the required sample size: rules-of-thumb, quick yet uncertain, and estimations for power, mathematically precise yet with the potential to overestimate or underestimate sample sizes when effect sizes are unknown. Misestimated sample sizes have negative repercussions in the form of increased costs, abandoned projects or abandoned publication of non-significant results. Here I describe a procedure for estimating sample sizes adequate for the testing approach which is most common in the behavioural, social, and biomedical sciences, that of Fisher’s tests of significance. The procedure focuses on a desired minimum effect size for the research at hand and finds the minimum sample size required for capturing such effect size as a statistically significant result. In a similar fashion than power analyses, sensitiveness analyses can also be extended to finding the minimum effect for a given sample size a priori as well as to calculating sensitiveness a posteriori. The article provides a full tutorial for carrying out a sensitiveness analysis, as well as empirical support via simulation.Published onlin

The fallacy of placing confidence in confidence intervals – A commentary

‘The fallacy of placing confidence in confidence intervals’ (Morey et al., 2016, Psychonomic Bulletin & Review, doi: 10.3758/s13423-015-0947-8) delved into a much needed technical and philosophical dissertation regarding the differences between typical (mis)interpretations of frequentist confidence intervals and the typical correct interpretation of Bayesian credible intervals. My contribution here partly strengthens the authors’ argument, partly closes some gaps they left open, and concludes with a note of attention to the possibility that there may be distinctions without real practical differences in the ultimate use of estimation by intervals, namely when assuming a common ground of uninformative priors and intervals as ranges of values instead of as posterior distributions per se.Published onlin

Commentary: Continuously cumulating meta-analysis and replicability

Publishe

P-values as percentiles. Commentary on: "Null hypothesis significance tests. A mix-up of two different theories: the basis for widespread confusion and numerous misinterpretations"

Publishe

Sorry to say, but pilots’ decisions were not irrational

Fradera’s Digest (2016) makes for interesting reading both for aviators and cognitive psychologists alike. Fradera reports on a research article by Walmsley and Gilbey (2016) and the Digest seems pretty accurate to the contents commented upon (in a way, thus, whatever praises or criticisms are raised apply equally to the latter article). The Digest is interesting because what it says is quite relevant in principle but rather misleading in practice. That is, the actual results reported by Walmsley and Gilbey, do not seem to support the portrayal of pilots as biased and irrational, a portrayal which originates in the interpretation of those results based on a flawed statistical technique—null hypothesis significance testing, or NHST. In a nutshell, Fradera opted to summarize the interpretation of (some) outputs made by Walmsley and Gilbey instead of re-interpreting those outputs anew within the context of the methodology and the results described in the original article, as I shall argue.Published onlin

Failings in COPE's guidelines to editors, and recommendations for improvement.

Letter highlighting failings in COPE's Guidelines to editors and proposing recommendations for improvement. The main recommendation is to create appropriate guidelines for dealing with fully disclosed (potential) conflicts of interest. COPE sought the topic as relevant and included a session on the topic as part of COPE's Forum (Feb 3, 2017; http://publicationethics.org/forum-discussion-topic-comments-please-7).Published onlin

Confidence intervals and tests are two sides of the same research question

Publishe