Making ERP research more transparent: Guidelines for preregistration

A combination of confirmation bias, hindsight bias, and pressure to publish may prompt the (unconscious) exploration of various methodological options and reporting only the ones that lead to a (statistically) significant outcome. This undisclosed analytic flexibility is particularly relevant in EEG research, where a myriad of preprocessing and analysis pipelines can be used to extract information from complex multidimensional data. One solution to limit confirmation and hindsight bias by disclosing analytic choices is preregistration: researchers write a time-stamped, publicly accessible research plan with hypotheses, data collection plan, and the intended preprocessing and statistical analyses before the start of a research project. In this manuscript, we present an overview of the problems associated with undisclosed analytic flexibility, discuss why and how EEG researchers would benefit from adopting preregistration, provide guidelines and examples on how to preregister data preprocessing and analysis steps in typical ERP studies, and conclude by discussing possibilities and limitations of this open science practice

The Use of Decision–Analytic Models in Atopic Eczema: A Systematic Review and Critical Appraisal

Objective: The objective of this systematic review was to identify and assess the quality of published economic decision–analytic models within atopic eczema against best practice guidelines, with the intention of informing future decision–analytic models within this condition. Methods: A systematic search of the following online databases was performed: MEDLINE, EMBASE, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Cochrane Central Register of Controlled Trials, Database of Abstracts of Reviews of Effects, Cochrane Database of Systematic Reviews, NHS Economic Evaluation Database, EconLit, Scopus, Health Technology Assessment, Cost-Effectiveness Analysis Registry and Web of Science. Papers were eligible for inclusion if they described a decision–analytic model evaluating both the costs and benefits associated with an intervention or prevention for atopic eczema. Data were extracted using a standardised form by two independent reviewers, whilst quality was assessed using the model-specific Philips criteria. Results: Twenty-four models were identified, evaluating either preventions (n = 12) or interventions (n = 12): 14 reported using a Markov modelling approach, four utilised decision trees and one a discrete event simulation, whilst five did not specify the approach. The majority, 22 studies, reported that the intervention was dominant or cost effective, given the assumptions and analytical perspective taken. Notably, the models tended to be short-term (16 used a time horizon of ≤1 year), often providing little justification for the limited time horizon chosen. The methodological and reporting quality of the studies was generally weak, with only seven studies fulfilling more than 50% of their applicable Philips criteria. Conclusions: This is the first systematic review of decision models in eczema. Whilst the majority of models reported favourable outcomes in terms of the cost effectiveness of the new intervention, the usefulness of these findings for decision-making is questionable. In particular, there is considerable scope for increasing the range of interventions evaluated, for improving modelling structures and reporting quality

Applying the Analytic Hierarchy Process in healthcare research: A systematic literature review and evaluation of reporting

Background: The Analytic Hierarchy Process (AHP), developed by Saaty in the late 1970s, is one of the methods for multi-criteria decision making. The AHP disaggregates a complex decision problem into different hierarchical levels. The weight for each criterion and alternative are judged in pairwise comparisons and priorities are calculated by the Eigenvector method. The slowly increasing application of the AHP was the motivation for this study to explore the current state of its methodology in the healthcare context. Methods: A systematic literature review was conducted by searching the Pubmed and Web of Science databases for articles with the following keywords in their titles or abstracts: "Analytic Hierarchy Process," "Analytical Hierarchy Process," "multi-criteria decision analysis," "multiple criteria decision," "stated preference," and "pairwise comparison." In addition, we developed reporting criteria to indicate whether the authors reported important aspects and evaluated the resulting studies' reporting. Results: The systematic review resulted in 121 articles. The number of studies applying AHP has increased since 2005. Most studies were from Asia (almost 30 %), followed by the US (25.6 %). On average, the studies used 19.64 criteria throughout their hierarchical levels. Furthermore, we restricted a detailed analysis to those articles published within the last 5 years (n = 69). The mean of participants in these studies were 109, whereas we identified major differences in how the surveys were conducted. The evaluation of reporting showed that the mean of reported elements was about 6.75 out of 10. Thus, 12 out of 69 studies reported less than half of the criteria. Conclusion: The AHP has been applied inconsistently in healthcare research. A minority of studies described all the relevant aspects. Thus, the statements in this review may be biased, as they are restricted to the information available in the papers. Hence, further research is required to discover who should be interviewed and how, how inconsistent answers should be dealt with, and how the outcome and stability of the results should be presented. In addition, we need new insights to determine which target group can best handle the challenges of the AHP.CHER

Qualitative interviews in psychology: problems and possibilities

This paper distinguishes a series of contingent and necessary problems that arise in the design, conduct, analysis and reporting of open-ended or conversational qualitative interviews in psychological research. Contingent problems in the reporting of interviews include: (1) the deletion of the interviewer; (2) the conventions of representation of interaction; (3) the specificity of analytic observations; (4) the unavailability of the interview set-up; (5) the failure to consider interviews as interaction. Necessary problems include: (1) the flooding of the interview with social science agendas and categories; (2) the complex and varying footing positions of interviewer and interviewee; (3) the orientations to stake and interest on the part of the interviewer and interviewee; (4) the reproduction of cognitivism. The paper ends with two kinds of recommendation. First, we argue that interviews should be studied as an interactional object, and that study should feed back into the design, conduct and analysis of interviews so that they can be used more effectively in cases where they are the most appropriate data gathering tools. Second, these problems with open-ended interviews highlight a range of specific virtues of basing analysis on naturalistic materials. Reasons for moving away from the use of interviews for many research questions are described

Effectiveness of school-based preventive interventions on adolescent alcohol use: a meta-analysis of randomized controlled trials

BACKGROUND: Preventive interventions for adolescents are an important priority within school systems. Several interventions have been developed, but the effectiveness of such interventions varies considerably between studies. The purpose of this study was to assess the effectiveness of universal school-based prevention programs on alcohol use among adolescents by using meta-analytic techniques. METHOD: A systematic literature search in the databases, PubMed (Medline), PsycINFO (Ovid), EMBASE (Ovid) and WEB of Science (ISI) was conducted to search for empirical articles published in the period January 1990 to August 2014. RESULTS: In total, 28 randomized controlled studies with 39,289 participants at baseline were included. Of these 28 articles, 12 studies (N = 16279) reported continuous outcomes (frequency of alcohol use and quantity of alcohol use), and 16 studies (N = 23010) reported categorical data (proportion of students who drank alcohol). The results of the random effects analyses showed that the overall effect size among studies reporting continuous outcomes was small and demonstrated a favorable effect from the preventive interventions (Hedges’ [Image: see text] = 0.22, p < .01). The effect size among studies reporting categorical outcomes was not significant ([Image: see text] = 0.94, p = .25). The level of heterogeneity between studies was found to be significant in most analyses. Moderator analyses conducted to explore the heterogeneity showed neither significant difference between the different school levels (junior high schools and high schools), nor between the varied program intensities (low, medium and high intensity programs). The meta-regression analyses examining continuous moderators showed no significant effects for age or gender. CONCLUSIONS: The findings from this meta-analysis showed that, overall, the effects of school-based preventive alcohol interventions on adolescent alcohol use were small but positive among studies reporting the continuous measures, whereas no effect was found among studies reporting the categorical outcomes. Possible population health outcomes, with recommendations for policy and practice, are discussed further in this paper

A systematic review of pliance, tracking, and augmenting

A

The reliability, validity, and accuracy of self-reported absenteeism from work: a meta-analysis

Because of a variety of access limitations, self-reported absenteeism from work is often employed in research concerning health, organizational behavior, and economics, and it is ubiquitous in large scale population surveys in these domains. Several well established cognitive and social-motivational biases suggest that self-reports of absence will exhibit convergent validity with records-based measures but that people will tend to underreport the behavior. We used meta-analysis to summarize the reliability, validity, and accuracy of absence self-reports. The results suggested that self-reports of absenteeism offer adequate test–retest reliability and that they exhibit reasonably good rank order convergence with organizational records. However, people have a decided tendency to underreport their absenteeism, although such underreporting has decreased over time. Also, self-reports were more accurate when sickness absence rather than absence for any reason was probed. It is concluded that self-reported absenteeism might serve as a valid measure in some correlational research designs. However, when accurate knowledge of absolute absenteeism levels is essential, the tendency to underreport could result in flawed policy decisions

The bitter truth about sugar and willpower

Dual-process theories of higher order cognition (DPTs) have been enjoying much success, particularly since Kahneman’s 2002 Nobel prize address and recent book Thinking, Fast and Slow (2009). Historically, DPTs have attempted to provide a conceptual framework that helps classify and predict differences in patterns of behavior found under some circumstances and not others in a host of reasoning, judgment, and decision-making tasks. As evidence has changed and techniques for examining behavior have moved on, so too have DPTs. Killing two birds with one stone, Evans and Stanovich (2013, this issue) respond to five main criticisms of DPTs. Along with addressing each criticism in turn, they set out to clarify the essential defining characteristics that distinguish one form of higher order cognition from the other. The aim of this commentary is to consider the defining characteristics of Type 1 and Type 2 processing that have been proposed and to suggest that the evidence can be taken to support quantitative differences rather than qualitatively distinct processes