A multi-cell experimental design to recover policy relevant treatment effects, with an application to online advertising

Abstract

Experiments are an important tool to measure the impacts of interventions. However, in experimental settings with one-sided noncompliance, extant empirical approaches may not produce the estimands a decision-maker needs to solve their problem. For example, these experimental designs are common in digital advertising settings, but typical methods do not yield effects that inform the intensive margin -- how much should be spent or how many consumers should be reached with a campaign. We propose a solution that combines a novel multi-cell experimental design with modern estimation techniques that enables decision-makers to recover enough information to solve problems with an intensive margin. Our design is straightforward to implement. Using data from advertising experiments at Facebook, we demonstrate our approach outperforms standard techniques in recovering treatment effect parameters. Through a simple advertising reach decision problem, we show that our approach generates better decisions relative to standard techniques