Controlling the uncontrolled: Are there incidental experimenter effects on physiologic responding?

Abstract

The degree to which experimenters shape participant behavior has long been of interest in experimental social science research. Here, we extend this question to the domain of peripheral psychophysiology, where experimenters often have direct, physical contact with participants, yet researchers do not consistently test for their influence. We describe analytic tools for examining experimenter effects in peripheral physiology. Using these tools, we investigate nine data sets totaling 1,341 participants and 160 experimenters across different roles (e.g., lead research assistants, evaluators, confederates) to demonstrate how researchers can test for experimenter effects in participant autonomic nervous system activity during baseline recordings and reactivity to study tasks. Our results showed (a) little to no significant variance in participants' physiological reactivity due to their experimenters, and (b) little to no evidence that three characteristics of experimenters that are well known to shape interpersonal interactions-status (using five studies with 682 total participants), gender (using two studies with 359 total participants), and race (in two studies with 554 total participants)-influenced participants' physiology. We highlight several reasons that experimenter effects in physiological data are still cause for concern, including the fact that experimenters in these studies were already restricted on a number of characteristics (e.g., age, education). We present recommendations for examining and reducing experimenter effects in physiological data and discuss implications for replication