Design and Analysis of Experiments in Networks: Reducing Bias from Interference

Estimating the effects of interventions in networks is complicated due to interference, such that the outcomes for one experimental unit may depend on the treatment assignments of other units. Familiar statistical formalism, experimental designs, and analysis methods assume the absence of this interference, and result in biased estimates of causal effects when it exists. While some assumptions can lead to unbiased estimates, these assumptions are generally unrealistic in the context of a network and often amount to assuming away the interference. In this work, we evaluate methods for designing and analyzing randomized experiments under minimal, realistic assumptions compatible with broad interference, where the aim is to reduce bias and possibly overall error in estimates of average effects of a global treatment. In design, we consider the ability to perform random assignment to treatments that is correlated in the network, such as through graph cluster randomization. In analysis, we consider incorporating information about the treatment assignment of network neighbors. We prove sufficient conditions for bias reduction through both design and analysis in the presence of potentially global interference; these conditions also give lower bounds on treatment effects. Through simulations of the entire process of experimentation in networks, we measure the performance of these methods under varied network structure and varied social behaviors, finding substantial bias reductions and, despite a bias–variance tradeoff, error reductions. These improvements are largest for networks with more clustering and data generating processes with both stronger direct effects of the treatment and stronger interactions between units. Keywords: causal inference; field experiments; peer effects; spillovers; social contagion; social network analysis; graph partitionin

Exact P-values for Network Interference

We study the calculation of exact p-values for a large class o f non- sharp null hypotheses about treatment effects in a setting wi th data from experiments involving members of a single connected ne twork. The class includes null hypotheses that limit the effect of on e unit’s treatment status on another according to the distance betwe en units; for example, the hypothesis might specify that the treatmen t status of immediate neighbors has no effect, or that units more than two edges away have no effect. We also consider hypotheses concerning t he valid- ity of sparsification of a network (for example based on the st rength of ties) and hypotheses restricting heterogeneity in peer effe cts (so that, for example, only the number or fraction treated among neigh boring units matters). Our general approach is to define an artificia l experi- ment, such that the null hypothesis that was not sharp for the original experiment is sharp for the artificial experiment, and such t hat the randomization analysis for the artificial experiment is val idated by the design of the original experiment

Estimating peer effects in networks with peer encouragement designs

Peer effects, in which the behavior of an individual is affected by the behavior of their peers, are central to social science. Because peer effects are often confounded with homophily and common external causes, recent work has used randomized experiments to estimate effects of specific peer behaviors. These experiments have often relied on the experimenter being able to randomly modulate mechanisms by which peer behavior is transmitted to a focal individual. We describe experimental designs that instead randomly assign individuals’ peers to encouragements to behaviors that directly affect those individuals. We illustrate this method with a large peer encouragement design on Facebook for estimating the effects of receiving feedback from peers on posts shared by focal individuals. We find evidence for substantial effects of receiving marginal feedback on multiple behaviors, including giving feedback to others and continued posting. These findings provide experimental evidence for the role of behaviors directed at specific individuals in the adoption and continued use of communication technologies. In comparison, observational estimates differ substantially, both underestimating and overestimating effects, suggesting that researchers and policy makers should be cautious in relying on them

Learning Causal Effects From Many Randomized Experiments Using Regularized Instrumental Variables

Scientific and business practices are increasingly resulting in large collections of randomized experiments. Analyzed together multiple experiments can tell us things that individual experiments cannot. We study how to learn causal relationships between variables from the kinds of collections faced by modern data scientists: the number of experiments is large, many experiments have very small effects, and the analyst lacks metadata (e.g., descriptions of the interventions). We use experimental groups as instrumental variables (IV) and show that a standard method (two-stage least squares) is biased even when the number of experiments is infinite. We show how a sparsity-inducing l0 regularization can (in a reversal of the standard bias--variance tradeoff) reduce bias (and thus error) of interventional predictions. We are interested in estimating causal effects, rather than just predicting outcomes, so we also propose a modified cross-validation procedure (IVCV) to feasibly select the regularization parameter. We show, using a trick from Monte Carlo sampling, that IVCV can be done using summary statistics instead of raw data. This makes our full procedure simple to use in many real-world applications

Social Influence and Reciprocity in Online Gift Giving

© 2018 Copyright is held by the owner/author(s). Giving gifts is a fundamental part of human relationships that is being affected by technology. The Internet enables people to give at the last minute and over long distances, and to observe friends giving and receiving gifts. How online gift giving spreads in social networks is therefore important to understand. We examine 1.5 million gift exchanges on Facebook and show that receiving a gift causes individuals to be 56% more likely to give a gift in the future. Additional surveys show that online gift giving was more socially acceptable to those who learned about it by observing friends' participation instead of a non-social encouragement. Most receivers pay the gift forward instead of reciprocating directly online, although surveys revealed additional instances of direct reciprocity, where the initial gifting occurred offline. Thus, social influence promotes the spread of online gifting, which both complements and substitutes for offline gifting

Interdependence and the cost of uncoordinated responses to COVID-19

Social distancing is the core policy response to coronavirus disease 2019 (COVID-19). But, as federal, state and local governments begin opening businesses and relaxing shelter-in-place orders worldwide, we lack quantitative evidence on how policies in one region affect mobility and social distancing in other regions and the consequences of uncoordinated regional policies adopted in the presence of such spillovers. To investigate this concern, we combined daily, county-level data on shelter-in-place policies with movement data from over 27 million mobile devices, social network connections among over 220 million Facebook users, daily temperature and precipitation data from 62,000 weather stations, and county-level census data on population demographics to estimate the geographic and social network spillovers created by regional policies across the United States. Our analysis shows that the contact patterns of people in a given region are significantly influenced by the policies and behaviors of people in other, sometimes distant, regions. When just one-third of a state’s social and geographic peer states adopt shelter-in-place policies, it creates a reduction in mobility equal to the state’s own policy decisions. These spillovers are mediated by peer travel and distancing behaviors in those states. A simple analytical model calibrated with our empirical estimates demonstrated that the “loss from anarchy” in uncoordinated state policies is increasing in the number of noncooperating states and the size of social and geographic spillovers. These results suggest a substantial cost of uncoordinated government responses to COVID-19 when people, ideas, and media move across borders