The Pipeline Project: Pre-publication Independent Replications of a Single Laboratory’s Research Pipeline

This crowdsourced project introduces a collaborative approach to improving the reproducibility of scientific research, in which findings are replicated in qualified independent laboratories before (rather than after) they are published. Our goal is to establish a non-adversarial replication process with highly informative final results. To illustrate the Pre-Publication Independent Replication (PPIR) approach, 25 research groups conducted replications of all ten moral judgment effects which the last author and his collaborators had “in the pipeline” as of August 2014. Six findings replicated according to all replication criteria, one finding replicated but with a significantly smaller effect size than the original, one finding replicated consistently in the original culture but not outside of it, and two findings failed to find support. In total, 40% of the original findings failed at least one major replication criterion. Potential ways to implement and incentivize pre-publication independent replication on a large scale are discussed

Data from a pre-publication independent replication initiative examining ten moral judgement effects

We present the data from a crowdsourced project seeking to replicate findings in independent laboratories before (rather than after) they are published. In this Pre-Publication Independent Replication (PPIR) initiative, 25 research groups attempted to replicate 10 moral judgment effects from a single laboratory’s research pipeline of unpublished findings. The 10 effects were investigated using online/lab surveys containing psychological manipulations (vignettes) followed by questionnaires. Results revealed a mix of reliable, unreliable, and culturally moderated findings. Unlike any previous replication project, this dataset includes the data from not only the replications but also from the original studies, creating a unique corpus that researchers can use to better understand reproducibility and irreproducibility in science

The Self-Contaminating Nature of Repeated Reports of Negative Emotions

Three experiments examined effects of measuring self-reported emotional intensity on sub- sequent self-reported emotional intensity. Across 3 experiments, we induced sadness, envy, and happiness and manipulated the number of emotional intensity measurements. In all experiments, repeated measurement led to weaker intensity of negative emotions than did a single measurement. Although the intensity of happiness was unaffected by repeated measurement, data suggest that measurements interfered with ongoing emotional experi- ence. We suggest that our findings have methodological, conceptual, and practical implica- tions, but perhaps foremost is the warning that social scientists may have greater cause for caution regarding repeated self-report measures than previously thought

Accuracy, Error, and Bias in Predictions for Real Versus Hypothetical Events

Participants made predictions about performance on tasks that they did or did not expect to complete. In three experiments, participants in task-unexpected conditions were unrealistically optimistic: They overestimated how well they would perform, often by a large margin, and their predictions were not correlated with their performance. By contrast, participants assigned to task-expected conditions made predictions that were not only less optimistic but strikingly accurate. Consistent with predictions from construal level theory, data from a fourth experiment suggest that it is the uncertainty associated with hypothetical tasks, and not a lack of cognitive processing, that frees people to make optimistic prediction errors. Unrealistic optimism, when it occurs, may be truly unrealistic; however, it may be less ubiquitous than has been previously suggested

If Only I had the Time! The Impact of Time Salience on Consumers’ Evaluations of Product Offers

We explore consumers\u27 consideration of their time budgets when evaluating product offers in a context in which we expect those budgets are most easily ignored—product giveaways. Across three studies, we manipulate the salience of time for participants considering free seminars (Study 1a) and free vacations (Studies 1b and 2) to be received in the near or distant future. Beginning with Study 1, we demonstrate that when time is made salient to them, consumers consider slack in their time budgets when evaluating near-future but not distant-future product giveaways. Otherwise, consumers appear to largely ignore time budget slack when evaluating free offers. In Study 2, we replicate these basic effects while providing evidence that consumers\u27 consider slack in their time budgets at the point they commit to a giveaway rather than at the point when they will receive the product. We discuss these findings in terms of both their theoretical and marketing implications

Prescribed Optimism: Is it Right to be Wrong About the Future?

We test the assumption that people desire to be accurate when making predictions about their own future. Results revealed that, across four different scenarios and three manipulated variables (commitment to a decision, agency over the decision, and control over outcomes), participants thought it was better to make optimistically biased predictions than accurate or pessimistically biased predictions. Additionally, participants thought that they and others would be optimistic in the scenarios they read, but insufficiently so. We argue that prescriptions can serve as one standard by which the quality of predictions can be judged, and that this particular standard strongly endorses optimism

Hoping for the Best or Preparing for the Worst? Regulatory Focus and Preferences for Optimism and Pessimism in Predicting Personal Outcomes

People are rarely completely accurate in forecasting their own futures. in- stead, past research has demonstrated tendencies for both optimistic and pessimistic bias in thinking about one’s own outcomes. Furthermore, both biases are thought to be potentially functional. Recently, an “intuitive functionalist” account of forecasting biases has been proposed (Sackett & Armor, 2010; see also Armor, Massey, & Sackett, 2008), which posits that individuals flexibly shift between optimistic or pessimistic outlooks based on the perceived value of each outlook. The present research examines people’s chronic motivational orientations as one factor that influences perceptions of the functional value of optimistic or pessimistic outlooks. Across three studies, we demonstrate that those primarily concerned with growth and advancement (i.e., promotion) prefer optimistic forecasts and perform better when adopting an optimistic outlook, whereas those primarily concerned with safety and security (i.e., prevention) prefer pessimistic forecasts and perform better when adopting a pessimistic outlook

You’re Having Fun When Time Flies: The Hedonic Consequences of Subjective Time Progression

Seven studies tested the hypothesis that people use subjective time progression in hedonic evaluation. When people believe that time has passed unexpectedly quickly, they rate tasks as more engaging, noises as less irritating, and songs as more enjoyable. We propose that felt time distortion operates as a metacognitive cue that people implicitly attribute to their enjoyment of an experience (i.e., time flew, so the experience must have been fun). Consistent with this attribution account, the effects of felt time distortion on enjoyment ratings were moderated by the need for attribution, the strength of the “time flies” naive theory, and the presence of an alternative attribution. These findings suggest a previously unexplored process through which subjective time progression can influence the hedonic evaluation of experiences

Goals as Reference Points in Marathon Running: A Novel Test of Reference Dependence

In a large-scale field study of marathon runners, we test whether goals act as reference points in shaping the valuation of outcomes. Theories of reference-dependent preferences, such as Prospect Theory, imply that outcomes that are just below or just above a reference point are evaluated differently. Consistent with the Prospect Theory value function, we find that satisfaction as a function of relative performance (the difference between a runner’s finishing time goal and her actual finishing time) exhibits loss aversion and diminishing sensitivity in both predictions of and actual experienced satisfaction. However, in contrast to Prospect Theory, we observe that loss aversion is partially driven by a discontinuity or jump at the reference point. In addition, we find that a runner’s time goal as well as their previous marathon times simultaneously impact runner satisfaction, providing support for the impact of multiple reference points on satisfaction

Response of macroarthropod assemblages to the loss of hemlock (Tsuga canadensis), a foundation species

In eastern North American forests, eastern hemlock (Tsuga canadensis) is a foundation species. As hemlock is lost from forests due to the invasive hemlock woolly adelgid (Adelges tsugae) and preemptive salvage logging, the structure of assemblages of species associated with hemlock is expected to change. We manipulated hemlock canopy structure at hectare scales to investigate the effects of hemlock death on assemblages of ants, beetles, and spiders in a New England forest. Relative to reference hemlock stands, both in situ death of hemlock and logging and removal of hemlock altered composition and diversity of beetles and spiders, and logging increased the species richness and evenness of ant assemblages. Species composition of ant assemblages in disturbed habitats was non-random relative to the regional species pool, but we found no evidence that interspecific competition shaped the structure of ant, beetle, or spider assemblages, in either manipulated or intact forest stands. Environmental filtering by hemlock appears to maintain low levels of species richness and evenness in forest stands, suggesting that the loss of hemlock due to the hemlock woolly adelgid or human activities will not likely lead to extirpations of ant, beetle, or spider species at local scales