6 research outputs found
The Power of Optimization Over Randomization in Designing Experiments Involving Small Samples
Random assignment, typically seen as the standard in controlled trials, aims to make experimental groups statistically equivalent before treatment. However, with a small sample, which is a practical reality in many disciplines, randomized groups are often too dissimilar to be useful. We propose an approach based on discrete linear optimization to create groups whose discrepancy in their means and variances is several orders of magnitude smaller than with randomization. We provide theoretical and computational evidence that groups created by optimization have exponentially lower discrepancy than those created by randomization and that this allows for more powerful statistical inference.National Science Foundation (U.S.). Graduate Research Fellowship (Grant 1122374
Optimal Experimental Design for Staggered Rollouts
Experimentation has become an increasingly prevalent tool for guiding
decision-making and policy choices. A common hurdle in designing experiments is
the lack of statistical power. In this paper, we study the optimal multi-period
experimental design under the constraint that the treatment cannot be easily
removed once implemented; for example, a government might implement a public
health intervention in different geographies at different times, where the
treatment cannot be easily removed due to practical constraints. The treatment
design problem is to select which geographies (referred by units) to treat at
which time, intending to test hypotheses about the effect of the treatment.
When the potential outcome is a linear function of unit and time effects, and
discrete observed/latent covariates, we provide an analytically feasible
solution to the optimal treatment design problem where the variance of the
treatment effect estimator is at most 1+O(1/N^2) times the variance using the
optimal treatment design, where N is the number of units. This solution assigns
units in a staggered treatment adoption pattern - if the treatment only affects
one period, the optimal fraction of treated units in each period increases
linearly in time; if the treatment affects multiple periods, the optimal
fraction increases non-linearly in time, smaller at the beginning and larger at
the end. In the general setting where outcomes depend on latent covariates, we
show that historical data can be utilized in designing experiments. We propose
a data-driven local search algorithm to assign units to treatment times. We
demonstrate that our approach improves upon benchmark experimental designs via
synthetic interventions on the influenza occurrence rate and synthetic
experiments on interventions for in-home medical services and grocery
expenditure