On the Generalizability of Experimental Results

The age-old question of the generalizability of the results of experiments that are conducted in artificial laboratory settings to more realistic inferential and decision making situations is considered in this paper. Conservatism in probability revision provides an example of a result that 1) has received wide attention, including attention in terms of implications for real-world decision making, on the basis of experiments conducted in artificial settings and 2) is now apparently thought by many to be highly situational and not at all a ubiquitous phenomenon, in which case its implications for real-world decision making are not as extensive as originally claimed. In this paper we consider the questions of generalizations from the laboratory to the real world in some detail, both within the context of the experiments regarding conservatism and within a more general context. In addition, we discuss some of the difficulties inherent in experimentation in realistic settings, suggest possible procedures for avoiding or at least alleviating such difficulties, and make a plea for more realistic experiments