To Train or Not To Train: Optimal Treatment Assignment Rules Using Welfare-to-Work Experiments

Planners often face the especially difficult and important task of assigning programs or treatments to optimize outcomes. Using the recent welfare-to-work reforms as an illustration, this paper considers the normative problem of how administrators might use data from randomized experiments to assign treatments. Under the new welfare system, state governments must design welfare programs to optimize employment. With experimental results in-hand, planners observe the average effect of training on employment but may not observe the individual effect of training. If the effect of a treatment varies across individuals, the planner faces a decision problem under ambiguity (Manski, 1998). In this setting, I find a straightforward proposition formalizes conditions under which a planner should reject particular decision rules as being inferior. An optimal decision rule, however, may not be revealed. In the absence of strong assumptions about the degree of heterogeneity in the population or the information known by the planner, the data are inconclusive about the efficacy of most assignment rules.ambiguity, randomized experiments, treatment choice, welfare-to-work programs

Agnostic notes on regression adjustments to experimental data: Reexamining Freedman's critique

Freedman [Adv. in Appl. Math. 40 (2008) 180-193; Ann. Appl. Stat. 2 (2008) 176-196] critiqued ordinary least squares regression adjustment of estimated treatment effects in randomized experiments, using Neyman's model for randomization inference. Contrary to conventional wisdom, he argued that adjustment can lead to worsened asymptotic precision, invalid measures of precision, and small-sample bias. This paper shows that in sufficiently large samples, those problems are either minor or easily fixed. OLS adjustment cannot hurt asymptotic precision when a full set of treatment-covariate interactions is included. Asymptotically valid confidence intervals can be constructed with the Huber-White sandwich standard error estimator. Checks on the asymptotic approximations are illustrated with data from Angrist, Lang, and Oreopoulos's [Am. Econ. J.: Appl. Econ. 1:1 (2009) 136--163] evaluation of strategies to improve college students' achievement. The strongest reasons to support Freedman's preference for unadjusted estimates are transparency and the dangers of specification search.Comment: Published in at http://dx.doi.org/10.1214/12-AOAS583 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Statistical Treatment Rules for Heterogeneous Populations: With Application to Randomized Experiments

This paper uses Wald's concept of the risk of a statistical decision function to address the question: How should sample data on treatment response be used to guide treatment choices in a heterogeneous population? Statistical treatment rules (STRs) are statistical decision functions that map observed covariates of population members and sample data on treatment response into treatment choices. I propose evaluation of STRs by their expected welfare (negative risk in Wald's terms), and I apply this criterion to compare two STRs when the sample data are generated by a classical randomized experiment. The rules compared both embody the reasonable idea that persons should be assigned the treatment with the best empirical success rate, but they differ in their use of covariate information. The conditional success (CS) rule selects treatments with the best empirical success rates conditional on specified covariates and the unconditional success (US) rule selects a treatment with the best unconditional empirical success rate. The main finding is a proposition giving finite-sample bounds on expected welfare under the two rules. The bounds, which rest on a large-deviations theorem of Hoeffding, yield explicit sample-size and distributional conditions under which the CS Rule is superior to the US rule.

Estimating the impact of city-wide Aedes aegypti population control: An observational study in Iquitos, Peru.

During the last 50 years, the geographic range of the mosquito Aedes aegypti has increased dramatically, in parallel with a sharp increase in the disease burden from the viruses it transmits, including Zika, chikungunya, and dengue. There is a growing consensus that vector control is essential to prevent Aedes-borne diseases, even as effective vaccines become available. What remains unclear is how effective vector control is across broad operational scales because the data and the analytical tools necessary to isolate the effect of vector-oriented interventions have not been available. We developed a statistical framework to model Ae. aegypti abundance over space and time and applied it to explore the impact of citywide vector control conducted by the Ministry of Health (MoH) in Iquitos, Peru, over a 12-year period. Citywide interventions involved multiple rounds of intradomicile insecticide space spray over large portions of urban Iquitos (up to 40% of all residences) in response to dengue outbreaks. Our model captured significant levels of spatial, temporal, and spatio-temporal variation in Ae. aegypti abundance within and between years and across the city. We estimated the shape of the relationship between the coverage of neighborhood-level vector control and reductions in female Ae. aegypti abundance; i.e., the dose-response curve. The dose-response curve, with its associated uncertainties, can be used to gauge the necessary spraying effort required to achieve a desired effect and is a critical tool currently absent from vector control programs. We found that with complete neighborhood coverage MoH intra-domicile space spray would decrease Ae. aegypti abundance on average by 67% in the treated neighborhood. Our framework can be directly translated to other interventions in other locations with geolocated mosquito abundance data. Results from our analysis can be used to inform future vector-control applications in Ae. aegypti endemic areas globally