Designing and Deploying Online Field Experiments

Online experiments are widely used to compare specific design alternatives, but they can also be used to produce generalizable knowledge and inform strategic decision making. Doing so often requires sophisticated experimental designs, iterative refinement, and careful logging and analysis. Few tools exist that support these needs. We thus introduce a language for online field experiments called PlanOut. PlanOut separates experimental design from application code, allowing the experimenter to concisely describe experimental designs, whether common "A/B tests" and factorial designs, or more complex designs involving conditional logic or multiple experimental units. These latter designs are often useful for understanding causal mechanisms involved in user behaviors. We demonstrate how experiments from the literature can be implemented in PlanOut, and describe two large field experiments conducted on Facebook with PlanOut. For common scenarios in which experiments are run iteratively and in parallel, we introduce a namespaced management system that encourages sound experimental practice.Comment: Proceedings of the 23rd international conference on World wide web, 283-29

Field experiments.

The season of 1900 was an unusual one throughout. The large rainfall during the growing months was in most respects favorable to plant growth as indicated by the magnificent crops that were harvested, though for the best results in some forms of experimental work, a drier season is essential. For example, an experiment was undertaken to determine the effect of cutting clover in different stages of maturity upon the quantity and quality of the cured hay from two cuttings, but owing to the difficulty in curing on account of wet weather, the results were not ascertained. The favorable conditions of growth also caused small grains to lodge badly, which greatly interfered with their proper maturity, and also hindered the collection of data regarding the characteristics of varieties. Attention is called to the ample yields recorded in some of the reports which follow. As was said before this was partly due to the favorable growing season together with the fact that many of the experiments were conducted in small areas, a condition generally favorable to large returns. However it must be said that this should not be considered a source of error, as the largest areas were always used consistent with the supplying of uniform conditions throughout each experiment. The size of plot varied from one-fortieth of an acre to five acres. In case of some of the corn experiments whole fields, including as much as twenty acres, on the college farm were devoted to the growing of the best varieties, this being cited to corroborate the results obtained in the smaller plots

Field Experiments on Anchoring of Economic Valuations

A pillar of behavioral research is the view that preferences are constructed during the value elicitation process, but it is unclear whether, and to what extent, such biases influence real market equilibria. This paper examines the “anchoring” phenomenon in the field. The first experiment produces evidence that inexperienced consumers can be anchored in the value elicitation process, yet there is little evidence that experienced agents are influenced by anchors. The second experiment finds that anchors have only transient effects on prices and quantities traded: aggregate market outcomes converge to the intersection of supply and demand after a few market periods.field experiment, anchoring, valuation, experience

Methodological Frontiers of Public Finance Field Experiments

The purpose of this article is to demonstrate how a rich array of methods can be applied to increase the relevance of field experiments in public economics. Two cross-cutting themes are important in multiple phases of the research. First, greater statistical sophistication can draw more value from a field experiment without obscuring the simple and compelling information from the differences in average outcomes of intervention and control groups. Second, the methodological frontier is interdisciplinary, drawing on knowledge and techniques developed in psychology, anthropology, and sociology that can be adapted in order to make public finance field experiments more useful.

Models suggesting field experiments to test two hypotheses explaining successional diversity

A simple mathematical model of competition is developed that includes two alternative mechanisms promoting successional diversity. The first underpins the competition-colonization hypothesis in which early successional species are able to persist because they colonize disturbed habitats before the arrival of late successional dominant competitors. The second underpins the niche hypothesis, in which early successional species are able to persist, even with unlimited colonization by late successional dominants, because they specialize on the resource-rich conditions typical of recently disturbed sites. We modify the widely studied competition-colonization model so that it also includes the mechanism behind the niche hypothesis. Analysis of this model suggests simple experiments that determine whether the successional diversity of a field system is maintained primarily by the competition-colonization mechanism, primarily by the niche mechanism, by neither, or by both. We develop quantitative metrics of the relative importance of the two mechanisms. We also discuss the implications for the management of biodiversity in communities structured by the two mechanisms

Reducing Exclusionary Attitudes through Interpersonal Conversation: Evidence from Three Field Experiments

Exclusionary attitudes-prejudice toward outgroups and opposition to policies that promote their well-being-are presenting challenges to democratic societies worldwide. Drawing on insights from psychology, we argue that non-judgmentally exchanging narratives in interpersonal conversations can facilitate durable reductions in exclusionary attitudes. We support this argument with evidence from three pre-registered field experiments targeting exclusionary attitudes toward unauthorized immigrants and transgender people. In these experiments, 230 canvassers conversed with 6,869 voters across 7 US locations. In Experiment 1, face-To-face conversations deploying arguments alone had no effects on voters' exclusionary immigration policy or prejudicial attitudes, but otherwise identical conversations also including the non-judgmental exchange of narratives durably reduced exclusionary attitudes for at least four months (d = 0.08). Experiments 2 and 3, targeting transphobia, replicate these findings and support the scalability of this strategy (ds = 0.08, 0.04). Non-judgmentally exchanging narratives can help overcome the resistance to persuasion often encountered in discussions of these contentious topics

Poverty, politics, and preferences: Field experiments and survey data from Vietnam

We conducted field experiments to investigate how wealth, political history, occupation, and other demographic variables (from a comprehensive earlier household survey) are correlated with risk, time discounting and trust in Vietnam. Our experiments suggest risk and time preferences depend on the stage of economic development. In wealthier villages, people are less loss-averse and more patient. Our research also shows people who participate in ROSCAs (rotating credit associations) are more patient than non-participant, but those who participate in bidding ROSCAs are less patient and more risk averse than those who participate in fixed ROSCAs. Results from a trust game demonstrate both positive and negative effects of communism. Villagers in the South tend to invest more in low-income partners without expecting repayment. On the other hand, people in the north are more trustworthy but do not pass on more money to the poor. Our findings also suggest market activities, like starting a small trade business, are correlated with trust and trustworthiness. We also contribute to experimental methodology by using choices that separate different aspects of risk aversion and time preferences in behavioral economics specifications

Real Wealth and Experimental Cooperation: Evidence from Field Experiments

This paper explores how wealth and inequality can affect self-governed solutions to commons dilemmas by constraining group cooperation. It reports a series of experiments in the field where subjects are actual commons users. Household data about the participants-context explain statistically the usually observed wide variation found within and across groups in similar experiments. Participants-wealth and inequality reduced cooperation when groups were allowed to have face-toface communication between rounds. There are implications for a greater awareness of nonpayoff asymmetries affecting cooperation in heterogeneous groups, apart from heterogeneity in the payoffs structure of the game.Cooperation, Collective