Crowdsourcing hypothesis tests: Making transparent how design choices shape research results

To what extent are research results influenced by subjective decisions that scientists make as they design studies? Fifteen research teams independently designed studies to answer fiveoriginal research questions related to moral judgments, negotiations, and implicit cognition. Participants from two separate large samples (total N > 15,000) were then randomly assigned to complete one version of each study. Effect sizes varied dramatically across different sets of materials designed to test the same hypothesis: materials from different teams renderedstatistically significant effects in opposite directions for four out of five hypotheses, with the narrowest range in estimates being d = -0.37 to +0.26. Meta-analysis and a Bayesian perspective on the results revealed overall support for two hypotheses, and a lack of support for three hypotheses. Overall, practically none of the variability in effect sizes was attributable to the skill of the research team in designing materials, while considerable variability was attributable to the hypothesis being tested. In a forecasting survey, predictions of other scientists were significantly correlated with study results, both across and within hypotheses. Crowdsourced testing of research hypotheses helps reveal the true consistency of empirical support for a scientific claim.</div

Psychological outcomes of local heritage engagement: participation in community archaeological excavations increases wellbeing, self-efficacy, and perceived community support

There is evidence that engagement with tangible heritage is linked to improvements in wellbeing. However, experimental tests of this association, as well as theoretical accounts explaining this relationship, are lacking. The present study aims to compensate for this gap by developing a theoretical framework based on the social identity approach that explains the effect of community-based heritage engagement on wellbeing, and testing this effect in a quasi-experimental field study in the context of community test pit archaeological excavations. In line with the predictions, the results demonstrate that excavation participants (but not participants in the control condition) report improvements on a number of psychological outcomes after (as compared to before) participation in a two-day excavation programme (including wellbeing, self-efficacy, and perceived community support). The findings offer implications for community-based approaches to enhancing wellbeing, as well as the practice of conducting community-based archaeological excavations

Study two: Results of the moderated regression analysis.

<p>Notes. <i>N</i> = 54; NFC  =  need for closure.</p><p>Situational need for closure was coded as 0 =  low, 1 =  high.</p><p>Coefficients are unstandardized B.</p><p>*p<.05;</p><p>**p<.01.</p><p>Study two: Results of the moderated regression analysis.</p

Study one: Means, standard deviations and correlations among variables.

<p>Notes. <i>N</i> = 60.</p><p>*p<.05;</p><p>**p<.01.</p><p>Study one: Means, standard deviations and correlations among variables.</p

Selected paintings for the study.

<p>Selected paintings for the study.</p

Study two: Means, standard deviations and correlations among variables.

<p>Notes. <i>N</i> = 54</p><p>*p<.05;</p><p>**p<.01.</p><p>Study two: Means, standard deviations and correlations among variables.</p

Means of positive global evaluation of the political candidate as a function of message framing and regulatory focus.

<p>Means of positive global evaluation of the political candidate as a function of message framing and regulatory focus.</p

Means of prevision of a positive impact of the election of the candidate on the immigration as a function of message framing and regulatory focus.

<p>Means of prevision of a positive impact of the election of the candidate on the immigration as a function of message framing and regulatory focus.</p

Means of implicit attitudes toward nuclear energy as a function of message framing and regulatory focus.

<p>Means of implicit attitudes toward nuclear energy as a function of message framing and regulatory focus.</p

Study one: Results of the hierarchical regression analysis.

<p>Notes. <i>N</i> = 60. Coefficients are standardized beta.</p><p>*p<.05;</p><p>**p<.01.</p><p>Study one: Results of the hierarchical regression analysis.</p