49 research outputs found

    A Tutorial on Hunting Statistical Significance by Chasing N.

    Get PDF
    There is increasing concern about the replicability of studies in psychology and cognitive neuroscience. Hidden data dredging (also called p-hacking) is a major contributor to this crisis because it substantially increases Type I error resulting in a much larger proportion of false positive findings than the usually expected 5%. In order to build better intuition to avoid, detect and criticize some typical problems, here I systematically illustrate the large impact of some easy to implement and so, perhaps frequent data dredging techniques on boosting false positive findings. I illustrate several forms of two special cases of data dredging. First, researchers may violate the data collection stopping rules of null hypothesis significance testing by repeatedly checking for statistical significance with various numbers of participants. Second, researchers may group participants post hoc along potential but unplanned independent grouping variables. The first approach 'hacks' the number of participants in studies, the second approach 'hacks' the number of variables in the analysis. I demonstrate the high amount of false positive findings generated by these techniques with data from true null distributions. I also illustrate that it is extremely easy to introduce strong bias into data by very mild selection and re-testing. Similar, usually undocumented data dredging steps can easily lead to having 20-50%, or more false positives.DS is supported by the James S. McDonnell Foundation

    Statistical reform and the replication crisis

    Get PDF
    The replication crisis has prompted many to call for statistical reform within the psychological sciences. Here we examine issues within Frequentist statistics that may have led to the replication crisis, and we examine the alternative---Bayesian statistics---that many have suggested as a replacement. The Frequentist approach and the Bayesian approach offer radically different perspectives on evidence and inference with the Frequentist approach prioritising error control and the Bayesian approach offering a formal method for quantifying the relative strength of evidence for hypotheses. We suggest that rather than mere statistical reform, what is needed is a better understanding of the different modes of statistical inference and a better understanding of how statistical inference relates to scientific inference

    When Null Hypothesis Significance Testing Is Unsuitable for Research: A Reassessment.

    Get PDF
    Null hypothesis significance testing (NHST) has several shortcomings that are likely contributing factors behind the widely debated replication crisis of (cognitive) neuroscience, psychology, and biomedical science in general. We review these shortcomings and suggest that, after sustained negative experience, NHST should no longer be the default, dominant statistical practice of all biomedical and psychological research. If theoretical predictions are weak we should not rely on all or nothing hypothesis tests. Different inferential methods may be most suitable for different types of research questions. Whenever researchers use NHST they should justify its use, and publish pre-study power calculations and effect sizes, including negative findings. Hypothesis-testing studies should be pre-registered and optimally raw data published. The current statistics lite educational approach for students that has sustained the widespread, spurious use of NHST should be phased out

    Understanding Mathematics Anxiety: Investigating the experiences of UK primary and secondary school students

    Get PDF
    The project investigated individuals’ attitudes towards mathematics because of what could be referred to as a “mathematics crisis” in the UK. Evidence suggests that functional literacy skills amongst working-age adults are steadily increasing but the proportion of adults with functional maths skills equivalent to a GCSE grade C has dropped from 26% in 2003 to only 22% in 2011 (National Numeracy, 2014). This number is strikingly low compared with the 57% who achieved the equivalent in functional literacy skills (National Numeracy, 2014)

    Stress, Time Pressure, Strategy Selection and Math Anxiety in Mathematics: A Review of the Literature.

    Get PDF
    We review how stress induction, time pressure manipulations and math anxiety can interfere with or modulate selection of problem-solving strategies (henceforth "strategy selection") in arithmetical tasks. Nineteen relevant articles were identified, which contain references to strategy selection and time limit (or time manipulations), with some also discussing emotional aspects in mathematical outcomes. Few of these take cognitive processes such as working memory or executive functions into consideration. We conclude that due to the sparsity of available literature our questions can only be partially answered and currently there is not much evidence of clear associations. We identify major gaps in knowledge and raise a series of open questions to guide further research

    Motor conflict in Stroop tasks: Direct evidence from single-trial electro-myography and electro-encephalography

    Get PDF
    Several brain imaging studies have assumed that response conflict is present in Stroop tasks. However, this has not been demonstrated directly. We examined the time-course of stimulus and response conflict resolution in a numerical Stroop task by combining single-trial electro-myography (EMG) and event-related brain potentials (ERP). EMG enabled the direct tracking of response conflict and the peak latency of the P300 ERP wave was used to index stimulus conflict. In correctly responded trials of the incongruent condition EMG detected robust incorrect response hand activation which appeared consistently in single trials. In 50–80% of the trials correct and incorrect response hand activation coincided temporally, while in 20–50% of the trials incorrect hand activation preceded correct hand activation. EMG data provides robust direct evidence for response conflict. However, congruency effects also appeared in the peak latency of the P300 wave which suggests that stimulus conflict also played a role in the Stroop paradigm. Findings are explained by the continuous flow model of information processing: Partially processed task-irrelevant stimulus information can result in stimulus conflict and can prepare incorrect response activity. A robust congruency effect appeared in the amplitude of incongruent vs. congruent ERPs between 330–400 ms, this effect may be related to the activity of the anterior cingulate cortex

    An event-related brain potential study of arithmetic syntax

    No full text
    Number cognition is important both for the society and for the individual. The disorder of number cognition (developmental dyscalculia) can have major impact on individual life quality. Several children and adults experience problems with one aspect of ‘arithmetic syntax’, the representation of place value structure. Here we provide the first cognitive neuroscience investigation of place value representation. Our study systematically contrasted semantic (the numerical distance effect) and syntactic (place value structure) violations in a mental addition verification task. Participants verified additions with four-digit numbers. Semantic incongruencies elicited the N400 ERP effect. A centro-parietal (putative P600) effect to place-value violations was not related to arithmetic syntax. Rather, this effect was an enlarged P3b reflecting different surprise values of place value vs. non-place value violations. This potential confound should be considered in numerical cognition experiments. The latency of the N400 and P3a effects were differentially affected by place-value analysis. The amplitude of the P3a and that of a fronto-central positive effect (FP600) was sensitive to place value analysis and digit content. Results suggest that ERPs can index the syntactical analysis of multi-digit numbers. Both ERP and behavioral data confirmed that multi-digit numbers were decomposed into their constituent digits, rather than evaluated holistically. These findings enable the detailed investigation of place value representation in both normal and dyscalculic populations
    corecore