Shortcomings of experimental economics to study human behavior: a reanalysis of Cohn et al. 2014, Nature 516, 86–89, ‘‘Business culture and dishonesty in the banking industry’’

Abstract

In the wake of financial scandals, Cohn and collaborators published a headline-grabber study in the field of behavioral economics. M.C. Villeval (2014) summarized the main message as follows, in News and Views of the Nature issue where the Cohn study was published: the “experiment shows that although bank employees behave honestly on average, their dishonesty increases when they make decisions after having been primed to think about their professional identity.” Cohn et al. thus provide evidence that “the incentives and the business culture developed in the financial sector may undermine the honesty norms of ordinary employees.” This study may have important consequences for policy, since, Villeval continues, “it is crucial to ensure a business culture of honesty in this industry to restore trust in it.” Villeval also argues that “from a scientific perspective, this study […] supports the economic theory of social identity […], links this theory with the economic analysis of lying behavior [… and] shows how behavioural economists can contribute to a broader reflection in science about how people manage their 'multiple selves' ”. Here I show that the use of flawed statistics methods, yet employed routinely in so-called “evidence-based” science, led the authors to distort the “evidence”. I am also using this data-set as an interesting example to explore how we can use modeling and simulations to provide a fair account of the information and uncertainty conveyed by the data, based on Confidence Intervals. I provide the R-code. Based on this paper, I question the contribution of behavioral economics to the understanding of human behavior and conclude with considerations on honesty and science