Four reasons to prefer Bayesian analyses over significance testing

Inference using significance testing and Bayes factors is compared and contrasted in five case studies based on real research. The first study illustrates that the methods will often agree, both in motivating researchers to conclude that H1 is supported better than H0, and the other way round, that H0 is better supported than H1. The next four, however, show that the methods will also often disagree. In these cases, the aim of the paper will be to motivate the sensible evidential conclusion, and then see which approach matches those intuitions. Specifically, it is shown that a high-powered non-significant result is consistent with no evidence for H0 over H1 worth mentioning, which a Bayes factor can show, and, conversely, that a low-powered non-significant result is consistent with substantial evidence for H0 over H1, again indicated by Bayesian analyses. The fourth study illustrates that a high-powered significant result may not amount to any evidence for H1 over H0, matching the Bayesian conclusion. Finally, the fifth study illustrates that different theories can be evidentially supported to different degrees by the same data; a fact that P-values cannot reflect but Bayes factors can. It is argued that appropriate conclusions match the Bayesian inferences, but not those based on significance testing, where they disagree

Social identity, social networks and recovery capital in emerging adulthood: a pilot study

Background It has been argued that recovery from substance dependence relies on a change in identity, with past research focused on ‘personal identity’. This study assessed support for a social identity model of recovery in emerging adults through examining associations between social identity, social networks, recovery capital, and quality of life. Methods Twenty participants aged 18–21 in residential treatment for substance misuse were recruited from four specialist youth drug treatment services - three detoxification facilities and one psychosocial rehabilitation facility in Victoria, Australia. Participants completed a detailed social network interview exploring the substance use of groups in their social networks and measures of quality of life, recovery capital, and social identity. Results Lower group substance use was associated with higher recovery capital, stronger identification with non-using groups, and greater importance of non-using groups in the social network. Additionally, greater identification with and importance of non-using groups were associated with better environmental quality of life, whereas greater importance conferred on using groups was associated with reduced environmental quality of life. Conclusions Support was found for the role of social identity processes in reported recovery capital and quality of life. Future research in larger, longitudinal samples is required to improve understanding of social identity processes during treatment and early recovery and its relationship to recovery stability. Keywords Social network Social identity Emerging adult Substance use Treatment Recovery Quality of lif

The dominance, prestige, and leadership account of social power motives

The power motive predicts influential social behaviour; however, its heterogeneous conceptualisations have produced inconsistent results. To overcome this problem, we developed and validated a unitary taxonomy of social power motives based on established delineations of social hierarchies: the dominance, prestige, and leadership account. While we could measure these motives both reliably and distinctively (study 1), we also showed they strongly related to a common power desire (study 2). Assessing their nomological networks (studies 3 and 4), we demonstrated distinct associations between the dominance motive (D: wanting to coerce others into adhering to one’s will) and anger and verbal aggression; the prestige motive (P: wanting to obtain admiration and respect) and the fear of losing reputation and claiming to have higher moral concerns; the leadership motive (L: wanting to take responsibility in and for one’s group) and emotional stability and helping behaviour. Furthermore, while D uniquely predicted agonistic/retaliatory behaviour in dictator games (study 5), L uniquely predicted the attainment of higher employment ranks in various professions (study 7). Finally, at least to some degree, P and L related positively, and D negatively to prosocial donating behaviour (study 6). This taxonomy represents a novel and powerful approach to predicting influential social behaviour

Many analysts, one data set: making transparent how variations in analytic choices affect results

Twenty-nine teams involving 61 analysts used the same data set to address the same research question: whether soccer referees are more likely to give red cards to dark-skin-toned players than to light-skin-toned players. Analytic approaches varied widely across the teams, and the estimated effect sizes ranged from 0.89 to 2.93 (Mdn = 1.31) in odds-ratio units. Twenty teams (69%) found a statistically significant positive effect, and 9 teams (31%) did not observe a significant relationship. Overall, the 29 different analyses used 21 unique combinations of covariates. Neither analysts’ prior beliefs about the effect of interest nor their level of expertise readily explained the variation in the outcomes of the analyses. Peer ratings of the quality of the analyses also did not account for the variability. These findings suggest that significant variation in the results of analyses of complex data may be difficult to avoid, even by experts with honest intentions. Crowdsourcing data analysis, a strategy in which numerous research teams are recruited to simultaneously investigate the same research question, makes transparent how defensible, yet subjective, analytic choices influence research results

When You Can, Scale Up: Large-Scale Study Shows No Effect of tDCS in an Ambiguous Risk-Taking Task

Background: A wide range of neuroimaging and neuromodulation studies have shown that the dorsolateral prefrontal cortex (DLPFC) plays a pivotal role in decision-making. Of particular interest is the question of its role in decision-making when conditions are uncertain and whether manipulating this neural substrate through neuromodulation changes subsequent risk-taking behaviour. Previous work using the Balloon Analogue Risk Task (BART) suggests that bilateral tDCS stimulation of the DLPFC reduces risk-taking behaviour but unilateral stimulation has no effect. However, participant numbers have been limited and may have biased the estimate of the size of the effect of the stimulation on task performance. Objectives/Hypothesis: We aimed to test the robustness and generalizability of these previous findings by using a very similar methodology but with a much larger sample. Methods: During both 20- and 30-minute tDCS stimulation at 2 mA, we administered the BART to about 200 participants assigned to bilateral DLPFC stimulation of either right anodal/left cathodal, left anodal/right cathodal or sham (Study 1 and Study 2); and to unilateral stimulation conditions (Study 2): right anodal, left anodal or sham with the referent electrode over the contralateral supraorbital region. Results: In the first bilateral study, we found that risk-taking was greater for participants in the right anodal/left cathodal stimulation group compared to those who received left anodal/right cathodal stimulation, but not compared to sham. The results obtained in the bilateral and unilateral stimulation protocols implemented in Study 2 yielded no evidence of any effect of stimulation. Combining the data from both studies, we found no statistically significant differences between mean performances of the nine stimulation groups. Indeed, all 95% confidence intervals for the nine means overlapped, suggesting that these randomly vary around a common population mean. Conclusions: This study showed that there was no detectable effect of tDCS stimulation on risky decision-making under ambiguity, compared to sham stimulation. Hence, using a much larger sample, we did not replicate previous work reporting a reduction in risky decisionmaking by bilateral stimulation of the DLPFC compared to sham. When the results of our bilateral and unilateral stimulation studies were combined, it emerged that the most likely explanation for the apparent significant results in our bilateral stimulation study was random variation in performance. This outcome is a further reminder of the need for appropriately sized samples to potentially achieve reliable outcomes in brain modulation studies

Mine or mother’s? Exploring the self-ownership effect across cultures

Peer reviewedPublisher PD

Many Labs 2: Investigating Variation in Replicability Across Samples and Settings

We conducted preregistered replications of 28 classic and contemporary published findings, with protocols that were peer reviewed in advance, to examine variation in effect magnitudes across samples and settings. Each protocol was administered to approximately half of 125 samples that comprised 15,305 participants from 36 countries and territories. Using the conventional criterion of statistical significance (p < .05), we found that 15 (54%) of the replications provided evidence of a statistically significant effect in the same direction as the original finding. With a strict significance criterion (p < .0001), 14 (50%) of the replications still provided such evidence, a reflection of the extremely highpowered design. Seven (25%) of the replications yielded effect sizes larger than the original ones, and 21 (75%) yielded effect sizes smaller than the original ones. The median comparable Cohen’s ds were 0.60 for the original findings and 0.15 for the replications. The effect sizes were small (< 0.20) in 16 of the replications (57%), and 9 effects (32%) were in the direction opposite the direction of the original effect. Across settings, the Q statistic indicated significant heterogeneity in 11 (39%) of the replication effects, and most of those were among the findings with the largest overall effect sizes; only 1 effect that was near zero in the aggregate showed significant heterogeneity according to this measure. Only 1 effect had a tau value greater than .20, an indication of moderate heterogeneity. Eight others had tau values near or slightly above .10, an indication of slight heterogeneity. Moderation tests indicated that very little heterogeneity was attributable to the order in which the tasks were performed or whether the tasks were administered in lab versus online. Exploratory comparisons revealed little heterogeneity between Western, educated, industrialized, rich, and democratic (WEIRD) cultures and less WEIRD cultures (i.e., cultures with relatively high and low WEIRDness scores, respectively). Cumulatively, variability in the observed effect sizes was attributable more to the effect being studied than to the sample or setting in which it was studied.UCR::Vicerrectoría de Investigación::Unidades de Investigación::Ciencias Sociales::Instituto de Investigaciones Psicológicas (IIP