Competition and moral behavior: A meta-analysis of forty-five crowd-sourced experimental designs

Significance Using experiments involves leeway in choosing one out of many possible experimental designs. This choice constitutes a source of uncertainty in estimating the underlying effect size which is not incorporated into common research practices. This study presents the results of a crowd-sourced project in which 45 independent teams implemented research designs to address the same research question: Does competition affect moral behavior? We find a small adverse effect of competition on moral behavior in a meta-analysis involving 18,123 experimental participants. Importantly, however, the variation in effect size estimates across the 45 designs is substantially larger than the variation expected due to sampling errors. This “design heterogeneity” highlights that the generalizability and informativeness of individual experimental designs are limited. Abstract Does competition affect moral behavior? This fundamental question has been debated among leading scholars for centuries, and more recently, it has been tested in experimental studies yielding a body of rather inconclusive empirical evidence. A potential source of ambivalent empirical results on the same hypothesis is design heterogeneity—variation in true effect sizes across various reasonable experimental research protocols. To provide further evidence on whether competition affects moral behavior and to examine whether the generalizability of a single experimental study is jeopardized by design heterogeneity, we invited independent research teams to contribute experimental designs to a crowd-sourced project. In a large-scale online data collection, 18,123 experimental participants were randomly allocated to 45 randomly selected experimental designs out of 95 submitted designs. We find a small adverse effect of competition on moral behavior in a meta-analysis of the pooled data. The crowd-sourced design of our study allows for a clean identification and estimation of the variation in effect sizes above and beyond what could be expected due to sampling variance. We find substantial design heterogeneity—estimated to be about 1.6 times as large as the average standard error of effect size estimates of the 45 research designs—indicating that the informativeness and generalizability of results based on a single experimental design are limited. Drawing strong conclusions about the underlying hypotheses in the presence of substantive design heterogeneity requires moving toward much larger data collections on various experimental designs testing the same hypothesis

Meeting the Contact-Mechanics Challenge

This paper summarizes the submissions to a recently announced contact-mechanics modeling challenge. The task was to solve a typical, albeit mathematically fully defined problem on the adhesion between nominally flat surfaces. The surface topography of the rough, rigid substrate, the elastic properties of the indenter, as well as the short-range adhesion between indenter and substrate, were specified so that diverse quantities of interest, e.g., the distribution of interfacial stresses at a given load or the mean gap as a function of load, could be computed and compared to a reference solution. Many different solution strategies were pursued, ranging from traditional asperity-based models via Persson theory and brute-force computational approaches, to real-laboratory experiments and all-atom molecular dynamics simulations of a model, in which the original assignment was scaled down to the atomistic scale. While each submission contained satisfying answers for at least a subset of the posed questions, efficiency, versatility, and accuracy differed between methods, the more precise methods being, in general, computationally more complex. The aim of this paper is to provide both theorists and experimentalists with benchmarks to decide which method is the most appropriate for a particular application and to gauge the errors associated with each one