The pipeline project:Pre-publication independent replications of a single laboratory's research pipeline

This crowdsourced project introduces a collaborative approach to improving the reproducibility of scientific research, in which findings are replicated in qualified independent laboratories before (rather than after) they are published. Our goal is to establish a non-adversarial replication process with highly informative final results. To illustrate the Pre-Publication Independent Replication (PPIR) approach, 25 research groups conducted replications of all ten moral judgment effects which the last author and his collaborators had "in the pipeline" as of August 2014. Six findings replicated according to all replication criteria, one finding replicated but with a significantly smaller effect size than the original, one finding replicated consistently in the original culture but not outside of it, and two findings failed to find support. In total, 40% of the original findings failed at least one major replication criterion. Potential ways to implement and incentivize pre-publication independent replication on a large scale are discussed. (C) 2015 The Authors. Published by Elsevier Inc.</p

Treatment Efficacy, Clinical Utility, and Cost-Effectiveness of Multidisciplinary Biopsychosocial Rehabilitation Treatments for Persistent Low Back Pain: A Systematic Review

Study Design: Systematic review. Objectives: To review the current literature on the treatment efficacy, clinical utility, and cost-effectiveness of multidisciplinary biopsychosocial rehabilitation (MBR) for patients suffering from persistent (nonspecific) lower back pain (LBP) in relation to pain intensity, disability, health-related quality of life, and work ability/sick leave. Methods: We carried out a systematic search of Web of Science, Cochrane Library, PubMed Central, EMBASE, and PsycINFO for English- and German-language literature published between January 2010 and July 2017. Study selection consisted of exclusion and inclusion phases. After screening for duplication, studies were excluded on the basis of criteria covering study design, number of participants, language of publication, and provision of information about the intervention. All the remaining articles dealing with the efficacy, utility, or cost-effectiveness of intensive (more than 25 hours per week) MBR encompassing at least 3 health domains and cognitive behavioral therapy–based psychological education were included. Results: The search retrieved 1199 publications of which 1116 were duplicates or met the exclusion criteria. Seventy of the remaining 83 articles did not meet the inclusion criteria; thus 13 studies were reviewed. All studies reporting changes in pain intensity or disability over 12 months after MBR reported moderate effect sizes and/or p-values for both outcomes. The effects on health-related quality of life were mixed, but MBR substantially reduced costs. Overall MBR produced an enduring improvement in work ability despite controversy and variable results. Conclusions: MBR is an effective treatment for nonspecific LBP, but there is room for improvement in cost-effectiveness and impact on sick leave, where the evidence was less compelling

A creative destruction approach to replication: implicit work and sex morality across cultures

How can we maximize what is learned from a replication study? In the creative destruction approach to replication, the original hypothesis is compared not only to the null hypothesis, but also to predictions derived from multiple alternative theoretical accounts of the phenomenon. To this end, new populations and measures are included in the design in addition to the original ones, to help determine which theory best accounts for the results across multiple key outcomes and contexts. The present pre-registered empirical project compared the Implicit Puritanism account of intuitive work and sex morality to theories positing regional, religious, and social class differences; explicit rather than implicit cultural differences in values; self-expression vs. survival values as a key cultural fault line; the general moralization of work; and false positive effects. Contradicting Implicit Puritanism's core theoretical claim of a distinct American work morality, a number of targeted findings replicated across multiple comparison cultures, whereas several failed to replicate in all samples and were identified as likely false positives. No support emerged for theories predicting regional variability and specific individual-differences moderators (religious affiliation, religiosity, and education level). Overall, the results provide evidence that work is intuitively moralized across cultures

Achtsamkeitstraining für die Firma? Gerne, aber bitte nachhaltig

Die weltweite Nachfrage nach Achtsamkeitstrainingskursen in Unternehmen hat in den letzten Jahren enorm zugenommen. Das große Interesse an Achtsamkeit liegt an der Hoffnung vieler Unternehmen weltweit, damit Stress im Firmenalltag abzubaün, mehr Gleichgewicht einzuleiten sowie in der Folge sogar Produktivitätssteigerungen realisieren zu können, aber auf sanfte, nachhaltige Art. Dennoch ist die Nachhaltigkeit von Achtsamkeitskursen in Unternehmen längst nicht so eingehend erforscht wie es die Popularität dieses Themas nahe legt. Das liegt hauptsächlich daran, dass bei vielen Kursen in Firmen Achtsamkeit mit Meditation gleichgestellt wird. Meditieren ist jedoch nur eine von vielen Methoden, um Achtsamkeit zu lernen und im Arbeitsalltag anzuwenden, und die Meditation ist nicht immer der beste Weg in die Achtsamkeit. In diesem Kapitel wird Schritt für Schritt erklärt, was Achtsamkeit ist und wie sie funktioniert, warum sie nicht dasselbe ist wie Achtsamkeitsmeditation und wie man Unternemen helfen kann, Achtsamkeit nachhaltig in ihre Firmenkultur einzubetten und somit langfristig zu erreichen, was sie sich durch Achtsamkeitskurse erhoffen

Data from a pre-publication independent replication initiative examining ten moral judgement effects

We present the data from a crowdsourced project seeking to replicate findings in independent laboratories before (rather than after) they are published. In this Pre-Publication Independent Replication (PPIR) initiative, 25 research groups attempted to replicate 10 moral judgment effects from a single laboratory's research pipeline of unpublished findings. The 10 effects were investigated using online/lab surveys containing psychological manipulations (vignettes) followed by questionnaires. Results revealed a mix of reliable, unreliable, and culturally moderated findings. Unlike any previous replication project, this dataset includes the data from not only the replications but also from the original studies, creating a unique corpus that researchers can use to better understand reproducibility and irreproducibility in science

The pipeline project: Pre-publication independent replications of a single laboratory's research pipeline

This crowdsourced project introduces a collaborative approach to improving the reproducibility of scientific research, in which findings are replicated in qualified independent laboratories before (rather than after) they are published. Our goal is to establish a non-adversarial replication process with highly informative final results. To illustrate the Pre-Publication Independent Replication (PPIR) approach, 25 research groups conducted replications of all ten moral judgment effects which the last author and his collaborators had “in the pipeline” as of August 2014. Six findings replicated according to all replication criteria, one finding replicated but with a significantly smaller effect size than the original, one finding replicated consistently in the original culture but not outside of it, and two findings failed to find support. In total, 40% of the original findings failed at least one major replication criterion. Potential ways to implement and incentivize pre-publication independent replication on a large scale are discussed

Examining the generalizability of research findings from archival data

This initiative examined systematically the extent to which a large set of archival research findings generalizes across contexts. We repeated the key analyses for 29 original strategic management effects in the same context (direct reproduction) as well as in 52 novel time periods and geographies; 45% of the reproductions returned results matching the original reports together with 55% of tests in different spans of years and 40% of tests in novel geographies. Some original findings were associated with multiple new tests. Reproducibility was the best predictor of generalizability—for the findings that proved directly reproducible, 84% emerged in other available time periods and 57% emerged in other geographies. Overall, only limited empirical evidence emerged for context sensitivity. In a forecasting survey, independent scientists were able to anticipate which effects would find support in tests in new samples

Examining the generalizability of research findings from archival data

This initiative examined systematically the extent to which a large set of archival research findings generalizes across contexts. We repeated the key analyses for 29 original strategic management effects in the same context (direct reproduction) as well as in 52 novel time periods and geographies; 45% of the reproductions returned results matching the original reports together with 55% of tests in different spans of years and 40% of tests in novel geographies. Some original findings were associated with multiple new tests. Reproducibility was the best predictor of generalizability-for the findings that proved directly reproducible, 84% emerged in other available time periods and 57% emerged in other geographies. Overall, only limited empirical evidence emerged for context sensitivity. In a forecasting survey, independent scientists were able to anticipate which effects would find support in tests in new samples

Crowdsourcing hypothesis tests: Making transparent how design choices shape research results

To what extent are research results influenced by subjective decisions that scientists make as they design studies? Fifteen research teams independently designed studies to answer fiveoriginal research questions related to moral judgments, negotiations, and implicit cognition. Participants from two separate large samples (total N > 15,000) were then randomly assigned to complete one version of each study. Effect sizes varied dramatically across different sets of materials designed to test the same hypothesis: materials from different teams renderedstatistically significant effects in opposite directions for four out of five hypotheses, with the narrowest range in estimates being d = -0.37 to +0.26. Meta-analysis and a Bayesian perspective on the results revealed overall support for two hypotheses, and a lack of support for three hypotheses. Overall, practically none of the variability in effect sizes was attributable to the skill of the research team in designing materials, while considerable variability was attributable to the hypothesis being tested. In a forecasting survey, predictions of other scientists were significantly correlated with study results, both across and within hypotheses. Crowdsourced testing of research hypotheses helps reveal the true consistency of empirical support for a scientific claim.</div