Interpreting OLS Estimands When Treatment Effects Are Heterogeneous: Smaller Groups Get Larger Weights

Applied work often studies the effect of a binary variable ("treatment") using linear models with additive effects. I study the interpretation of the OLS estimands in such models when treatment effects are heterogeneous. I show that the treatment coefficient is a convex combination of two parameters, which under certain conditions can be interpreted as the average treatment effects on the treated and untreated. The weights on these parameters are inversely related to the proportion of observations in each group. Reliance on these implicit weights can have serious consequences for applied work, as I illustrate with two well-known applications. I develop simple diagnostic tools that empirical researchers can use to avoid potential biases. Software for implementing these methods is available in R and Stata. In an important special case, my diagnostics only require the knowledge of the proportion of treated units

Mostly Harmless Simulations? Using Monte Carlo Studies for Estimator Selection

We consider two recent suggestions for how to perform an empirically motivated Monte Carlo study to help select a treatment effect estimator under unconfoundedness. We show theoretically that neither is likely to be informative except under restrictive conditions that are unlikely to be satisfied in many contexts. To test empirical relevance, we also apply the approaches to a real-world setting where estimator performance is known. Both approaches are worse than random at selecting estimators which minimise absolute bias. They are better when selecting estimators that minimise mean squared error. However, using a simple bootstrap is at least as good and often better. For now researchers would be best advised to use a range of estimators and compare estimates for robustness

Population average gender effects

In this paper I develop a consistent estimator of the population average treatment effect (PATE) which is based on a nonstandard version of the Oaxaca-Blinder decomposition. As a result, I extend the recent literature which has utilized the treatment effects framework to reinterpret this technique, and propose an alternative solution to its fundamental problem of comparison group choice. I also use the Oaxaca-Blinder decomposition and its semiparametric extension to decompose gender wage differentials with the UK Labour Force Survey (LFS) data, while providing separate estimates of the average gender effect on men, women, and the whole population

When Should We (Not) Interpret Linear IV Estimands as LATE?

In this paper I revisit the interpretation of the linear instrumental variables (IV) estimand as a weighted average of conditional local average treatment effects (LATEs). I focus on a practically relevant situation in which additional covariates are required for identification while the reduced-form and first-stage regressions implicitly restrict the effects of the instrument to be homogeneous, and are thus possibly misspecified. I show that the weights on some conditional LATEs are negative and the IV estimand is no longer interpretable as a causal effect under a weaker version of monotonicity, i.e. when there are compliers but no defiers at some covariate values and defiers but no compliers elsewhere. The problem of negative weights disappears in the overidentified specification of Angrist and Imbens (1995) and in an alternative method, termed "reordered IV," that I also develop. Even if all weights are positive, the IV estimand in the just identified specification is not interpretable as the unconditional LATE parameter unless the groups with different values of the instrument are roughly equal sized. I illustrate my findings in an application to causal effects of college education using the college proximity instrument. The benchmark estimates suggest that college attendance yields earnings gains of about 60 log points, which is well outside the range of estimates in the recent literature. I demonstrate that this result is driven by the existence of defiers and the presence of negative weights. Corrected estimates indicate that attending college causes earnings to be roughly 20% higher

Mostly harmless simulations? Using Monte Carlo studies for estimator selection

We consider two recent suggestions for how to perform an empirically motivated Monte Carlo study to help select a treatment effect estimator under unconfoundedness. We show theoretically that neither is likely to be informative except under restrictive conditions that are unlikely to be satisfied in many contexts. To test empirical relevance, we also apply the approaches to a real-world setting where estimator performance is known. Both approaches are worse than random at selecting estimators which minimise absolute bias. They are better when selecting estimators that minimise mean squared error. However, using a simple bootstrap is at least as good and often better. For now researchers would be best advised to use a range of estimators and compare estimates for robustness

New Evidence on Linear Regression and Treatment Effect Heterogeneity

In this paper I provide new evidence on the implications of treatment effect heterogeneity for least squares estimation when the effects are inappropriately assumed to be homogenous. I prove that under a set of benchmark assumptions linear regression provides a consistent estimator of the population average treatment effect on the treated times the population proportion of the nontreated individuals plus the population average treatment effect on the nontreated times the population proportion of the treated individuals. Consequently, in many empirical applications the linear regression estimates might not be close to any of the standard average treatment effects of interest

Koncepcja przestrzeni Józefa Szujskiego jako rewizja romantycznego obrazu misji Polski na Wschodzie

HENRYK SŁOCZYŃSKI, dr hab., pracownik Zakład Dziejów Historiografii i Metodologii Historii w Instytucie Historii Uniwersytetu Jagiellońskiego. Zainteresowania badawcze: dzieje polskiej myśli historycznej i politycznej w okresie zaborów; szczególnie koncepcje myślicieli epoki romantyzmu i krakowskich konserwatystów oraz malarstwo historyczne Jana Matejki. Autor trzech publikacji książkowych: Matejko (Wrocław 2000), Jan Matejko (Kraków 2005) oraz Światło w dziejarskiej ciemnicy. Koncepcja dziejów i interpretacja przeszłości Polski Joachima Lelewela (Kraków 2010).30732

New Evidence on Linear Regression and Treatment Effect Heterogeneity

It is standard practice in applied work to rely on linear least squares regression to estimate the effect of a binary variable ("treatment") on some outcome of interest. In this paper I study the interpretation of the regression estimand when treatment effects are in fact heterogeneous. I show that the coefficient on treatment is identical to the outcome of the following three-step procedure: first, calculate the linear projection of treatment on the vector of other covariates ("propensity score"); second, calculate average partial effects for both groups of interest from a regression of outcome on treatment, the propensity score, and their interaction; third, calculate a weighted average of these two effects, with weights being inversely related to the unconditional probability that a unit belongs to a given group. Each of these steps is potentially problematic, but this last property – the reliance on implicit weights which are inversely related to the proportion of each group – can have particularly devastating consequences for applied work. To illustrate the severity of this issue, I perform Monte Carlo simulations as well as replicate two prominent applied papers: Berger, Easterly, Nunn and Satyanath (2013) on the effects of successful CIA interventions during the Cold War on imports from the US; and Martinez-Bravo (2014) on the effects of appointed officials on village-level electoral results in Indonesia. In both cases some of the conclusions change dramatically after allowing for heterogeneity in effects

Doubly Robust Estimation of Local Average Treatment Effects Using Inverse Probability Weighted Regression Adjustment

We revisit the problem of estimating the local average treatment effect (LATE) and the local average treatment effect on the treated (LATT) when control variables are available, either to render the instrumental variable (IV) suitably exogenous or to improve precision. Unlike previous approaches, our doubly robust (DR) estimation procedures use quasi-likelihood methods weighted by the inverse of the IV propensity score - so-called inverse probability weighted regression adjustment (IPWRA) estimators. By properly choosing models for the propensity score and outcome models, fitted values are ensured to be in the logical range determined by the response variable, producing DR estimators of LATE and LATT with appealing small sample properties. Inference is relatively straightforward both analytically and using the nonparametric bootstrap. Our DR LATE and DR LATT estimators work well in simulations. We also propose a DR version of the Hausman test that compares different estimates of the average treatment effect on the treated (ATT) under one-sided noncompliance

New Evidence on Linear Regression and Treatment Effect Heterogeneity

It is standard practice in applied work to rely on linear least squares regression to estimate the effect of a binary variable ("treatment") on some outcome of interest. In this paper I study the interpretation of the regression estimand when treatment effects are in fact heterogeneous. I show that the coefficient on treatment is identical to the outcome of the following three-step procedure: first, calculate the linear projection of treatment on the vector of other covariates ("propensity score"); second, calculate average partial effects for both groups of interest from a regression of outcome on treatment, the propensity score, and their interaction; third, calculate a weighted average of these two effects, with weights being inversely related to the unconditional probability that a unit belongs to a given group. Each of these steps is potentially problematic, but this last property – the reliance on implicit weights which are inversely related to the proportion of each group – can have particularly devastating consequences for applied work. To illustrate the severity of this issue, I perform Monte Carlo simulations as well as replicate two prominent applied papers: Berger, Easterly, Nunn and Satyanath (2013) on the effects of successful CIA interventions during the Cold War on imports from the US; and Martinez-Bravo (2014) on the effects of appointed officials on village-level electoral results in Indonesia. In both cases some of the conclusions change dramatically after allowing for heterogeneity in effects