Matching estimators and optimal bandwidth choice

Optimal bandwidth choice for matching estimators and their finite sample properties are examined. An approximation to their MSE is derived, as a basis for a plug-in bandwidth selector. In small samples, this approximation is not very accurate, though. Alternatively, conventional cross-validation bandwidth selection is considered and performs rather well in simulation studies: Compared to standard pair-matching, kernel and ridge matching achieve reductions in MSE of about 25 to 40%. Local linear matching and weighting perform poorly. Furthermore, the scope for developing better bandwidth selectors seems to be limited for ridge matching, but non-negligible for kernel and local linear matchin

A PRIMER ON PROPENSITY SCORE MATCHING ESTIMATORS

Nonparametric matching estimators are frequently applied in evaluation studies. The general idea of the methodology is to determine the impact of treatment on the treated using information from treated and from similar non-treated observations to build a counterfactual of no treatment. I discuss the methodology for both the binary treatment case as well as for the multiple treatment case.Propensity score matching, binary treatment, multiple treatments

A Non-Experimental Evaluation of Curriculum Effectiveness in Math

We use non-experimental data from a large panel of schools and districts in Indiana to evaluate the impacts of math curricula on student achievement. Using matching methods, we obtain causal estimates of curriculum effects at just a fraction of what it would cost to produce experimental estimates. Furthermore, external validity concerns that are particularly cogent in experimental curricular evaluations suggest that our non-experimental estimates may be preferred. In the short term, we find large differences in effectiveness across some math curricula. However, as with many other educational inputs, the effects of math curricula do not persist over time. Across curriculum adoption cycles, publishers that produce less effective curricula in one cycle do not lose market share in the next cycle. One explanation for this result is the dearth of information available to administrators about curricular effectiveness

Testing the suitability of polynomial models in errors-in-variables problems

A low-degree polynomial model for a response curve is used commonly in practice. It generally incorporates a linear or quadratic function of the covariate. In this paper we suggest methods for testing the goodness of fit of a general polynomial model when there are errors in the covariates. There, the true covariates are not directly observed, and conventional bootstrap methods for testing are not applicable. We develop a new approach, in which deconvolution methods are used to estimate the distribution of the covariates under the null hypothesis, and a ``wild'' or moment-matching bootstrap argument is employed to estimate the distribution of the experimental errors (distinct from the distribution of the errors in covariates). Most of our attention is directed at the case where the distribution of the errors in covariates is known, although we also discuss methods for estimation and testing when the covariate error distribution is estimated. No assumptions are made about the distribution of experimental error, and, in particular, we depart substantially from conventional parametric models for errors-in-variables problems.Comment: Published in at http://dx.doi.org/10.1214/009053607000000361 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Asymptotic Expansions for Some Semiparametric Program Evaluation Estimators

We investigate the performance of a class of semiparametric estimators of the treatment effect via asymptotic expansions. We derive approximations to the first two moments of the estimator that are valid to 'second order'. We use these approximations to define a method of bandwidth selection. We also propose a degrees- of-freedom like bias correction that improves the second order properties of the estimator but without requiring estimation of higher order derivatives of the unknown propensity score. We provide some numerical calibrations of the results.Bandwidth selection, kernel estimation, program evaluation, semiparametric estimation, treatment effect.

Effect of mean on variance function estimation in nonparametric regression

Variance function estimation in nonparametric regression is considered and the minimax rate of convergence is derived. We are particularly interested in the effect of the unknown mean on the estimation of the variance function. Our results indicate that, contrary to the common practice, it is not desirable to base the estimator of the variance function on the residuals from an optimal estimator of the mean when the mean function is not smooth. Instead it is more desirable to use estimators of the mean with minimal bias. On the other hand, when the mean function is very smooth, our numerical results show that the residual-based method performs better, but not substantial better than the first-order-difference-based estimator. In addition our asymptotic results also correct the optimal rate claimed in Hall and Carroll [J. Roy. Statist. Soc. Ser. B 51 (1989) 3--14].Comment: Published in at http://dx.doi.org/10.1214/009053607000000901 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Post-Secondary Education in Canada: Can Ability Bias Explain the Earnings Gap Between College and University Graduates?

Post-Secondary Education in Canada: Can Ability Bias Explain the Earnings Gap Between College and University Graduates? Using the Canadian General Social Survey we compute returns to post-secondary education relative to high-school. Unlike previous research using Canadian data, our dataset allows us to control for ability selection into higher education. We find strong evidence of positive ability selection into all levels of post-secondary education for men and weaker positive selection for women. Since the ability selection is stronger for higher levels of education, particularly for university, the difference in returns between university and college or trades education decreases slightly after accounting for ability bias. However, a puzzling large gap persists, with university-educated men still earning over 20% more than men with college or trades education. Moreover, contrary to previous Canadian literature that reports higher returns for women, we document that the OLS hourly wage returns to university education are the same for men and women. OLS returns are higher for women only if weekly or yearly wages are considered instead, because university-educated women work more hours than the average. Nevertheless, once we account for ability selection into post-secondary education, we generally find higher returns for women than for men for all wage measures as a result of the stronger ability selection for men.returns to university, returns to college, returns to trades, unobserved ability, selection bias

How to Control for Many Covariates? Reliable Estimators Based on the Propensity Score

We investigate the finite sample properties of a large number of estimators for the average treatment effect on the treated that are suitable when adjustment for observable covariates is required, like inverse probability weighting, kernel and other variants of matching, as well as different parametric models. The simulation design used is based on real data usually employed for the evaluation of labour market programmes in Germany. We vary several dimensions of the design that are of practical importance, like sample size, the type of the outcome variable, and aspects of the selection process. We find that trimming individual observations with too much weight as well as the choice of tuning parameters is important for all estimators. The key conclusion from our simulations is that a particular radius matching estimator combined with regression performs best overall, in particular when robustness to misspecifications of the propensity score is considered an important property.propensity score matching, kernel matching, inverse probability weighting, selection on observables, empirical Monte Carlo study, finite sample properties

How to control for many covariates? Reliable estimators based on the propensity score

We investigate the finite sample properties of a large number of estimators for the average treatment effect on the treated that are suitable when adjustment for observable covariates is required, like inverse pro¬bability weighting, kernel and other variants of matching, as well as different parametric models. The simulation design used is based on real data usually employed for the evaluation of labour market programmes in Germany. We vary several dimensions of the design that are of practical importance, like sample size, the type of the outcome variable, and aspects of the selection process. We find that trimming individual observations with too much weight as well as the choice of tuning parameters is important for all estimators. The key conclusion from our simulations is that a particular radius matching estimator combined with regression performs best overall, in particular when robustness to misspecifications of the propensity score is considered an important property.Propensity score matching, kernel matching, inverse probability weighting, selection on observables, empirical Monte Carlo study, finite sample properties