Was There a Riverside Miracle? A Framework for Evaluating Multi-Site Programs

This paper uses data from the Greater Avenues for Independence (GAIN) initiative to discuss the evaluation of programs that are implemented at multiple sites. Two frequently used methods are to pool the data or to use fixed effects (an extreme version of which estimates separate models for each site). The former approach, however, ignores site effects. Though the latter estimates site effects, it lacks a framework for predicting the impact in subsequent implementations of the program (e.g., will a new implementation resemble Riverside or Alameda?). I develop a model for earnings that lies between these two extremes. For the GAIN data, I show that most of the differences across sites are due to differences in the composition of participants. I show also that uncertainty regarding predicting site effects is important; when the predictive uncertainty is ignored, the treatment impact for the Riverside sites is significant, but when we consider predictive uncertainty, the impact for the Riverside sites is insignificant. Finally, I demonstrate that the model is able to extrapolate site effects with reasonable accuracy, when the site for which the prediction is being made does not differ substantially from the sites already observed. For example, the San Diego treatment effects could have been predicted based on observable site characteristics, but the Riverside effects are consistently underestimated.

The Effect of Automobile Insurance and Accident Liability Laws in Traffic Fatalities

This paper investigates the incentive effects of automobile insurance, compulsory insurance laws, and no-fault liability laws on driver behavior and traffic fatalities. We analyze a panel of 50 U.S. states and the District of Columbia from 1970-1998, a period in which many states adopted compulsory insurance regulations and/or no-fault laws. Using an instrumental variables approach, we find evidence that automobile insurance has moral hazard costs, leading to an increase in traffic fatalities. We also find that reductions in accident liability produced by no-fault liability laws have led to an increase in traffic fatalities (estimated to be on the order of 6%). Overall, our results indicate that, whatever other benefits they might produce, increases in the incidence of automobile insurance and moves to no-fault liability systems have significant negative effects on traffic fatalities.

Child Labor: The Role of Income Variability and Access to Credit Across Countries

This paper examines the relationship between child labor and access to credit at a cross-country level. Even though this link is theoretically central to child labor, so far there has been little work done to assess its importance empirically. We measure child labor as a country aggregate, and credit constraints are proxied by the extent of financial development. These two variables display a strong negative relationship, which we show is robust to selection on observables (by controlling for a wide range of variables such as GDP per capita, urbanization, initial child labor, schooling, fertility, legal institutions, inequality, and openness, and by allowing for a nonparametric functional form), and to selection on unobservables (by allowing for fixed effects). We find that the magnitude of the association between our proxy of access to credit and child labor is large in the sub-sample of poor countries. Moreover, in the absence of developed financial markets, households appear to resort substantially to child labor in order to cope with income variability. This evidence suggests that policies aimed at widening households' access to credit could be effective in reducing the extent of child labor.

Propensity Score Matching Methods for Non-experimental Causal Studies

This paper considers causal inference and sample selection bias in non-experimental settings in which: (i) few units in the non-experimental comparison group are comparable to the treatment units, and (ii) selecting a subset of comparison units similar to the treatment units is difficult because units must be compared across a high-dimensional set of pre-treatment characteristics. We propose the use of propensity score matching methods and implement them using data from the NSW experiment. Following Lalonde (1986), we pair the experimental treated units with non-experimental comparison units from the CPS and PSID and compare the estimates of the treatment effect obtained using our methods to the benchmark results from the experiment. We show that the methods succeed in focusing attention on the small subset of the comparison units comparable to the treated units and, hence, in alleviating the bias due to systematic differences between the treated and comparison units.

Causal Effects in Non-Experimental Studies: Re-Evaluating the Evaluation of Training Programs

This paper uses propensity score methods to address the question: how well can an observational study estimate the treatment impact of a program? Using data from Lalonde's (1986) influential evaluation of non-experimental methods, we demonstrate that propensity score methods succeed in estimating the treatment impact of the National Supported Work Demonstration. Propensity score methods reduce the task of controlling for differences in pre-intervention variables between the treatment and the non-experimental comparison groups to controlling for differences in the estimated propensity score (the probability of assignment to treatment, conditional on covariates). It is difficult to control for differences in pre-intervention variables when they are numerous and when the treatment and comparison groups are dissimilar, whereas controlling for the estimated propensity score, a single variable on the unit interval, is a straightforward task. We apply several methods, such as stratification on the propensity score and matching on the propensity score, and show that they result in accurate estimates of the treatment impact.

Program Evaluation as a Decision Problem

I argue for thinking of program evaluation as a decision problem. In the context of California's GAIN experiment (a randomized trial of a welfare-to-work alternative to AFDC), I show that GAIN first-order stochastically dominates AFDC when considering the choice between the treatment and control programs in terms of average earnings, even though the treatment effect is not statistically significant. I also argue for incorporating the post-evaluation assignment mechanism for the program under consideration into the evaluation process. I show that if policies, such as allowing a career counselor to choose which program individuals join, are included in the evaluation, then GAIN is superior to AFDC whereas the opposite ranking emerges from the standard treatment versus control comparison which ignores potential heterogeneity in the treatment impact.

Why Should We Care About Child Labor? The Education, Labor Market, and Health Consequences of Child Labor

Although there is an extensive literature on the determinants of child labor and many initiatives aimed at combating it, there is limited evidence on the consequences of child labor on socio-economic outcomes such as education, wages, and health. We evaluate the causal effect of child labor participation on these outcomes using panel data from Vietnam and an instrumental variables strategy. Five years subsequent to the child labor experience, we find significant negative impacts on school participation and educational attainment, but also find substantially higher earnings for those (young) adults who worked as children. We find no significant effects on health. Over a longer horizon, we estimate that from age 30 onward the forgone earnings attributable to lost schooling exceed any earnings gain associated with child labor and that the net present discounted value of child labor is positive for discount rates of 11.5 percent or higher. We show that child labor is prevalent among households likely to have higher borrowing costs, that are farther from schools, and whose adult members experienced negative returns to their own education. This evidence suggests that reducing child labor will require facilitating access to credit and will also require households to be forward looking.

Was there a Riverside miracle? An hierarchical framework for evaluating programs with grouped data

This paper uses data from the Greater Avenues for Independence (GAIN) demonstration to discuss the evaluation of programs that are implemented at multiple sites. Two frequently used methods are pooling the data or using fixed effects (an extreme version of which estimates separate models for each site). The former approach, however, ignores site effects. Though the latter incorporates site effects, it lacks a framework for predicting the impact of subsequent implementations of the program (e.g., will a new implementation resemble Riverside or Alameda?). I present an hierarchical model that lies between these two extremes. For the GAIN data, I demonstrate that the model captures much of the site-to-site variation of treatment effects, but has less uncertainty than a model which estimates treatment effects separately for each site. I also show that uncertainty in predicting site effects is important: when the predictive uncertainty is ignored, the treatment impact for the Riverside sites is significant, but when we consider predictive uncertainty, the impact for the Riverside sites is insignificant. Finally, I demonstrate that the model is able to extrapolate site effects with reasonable accuracy, when the site for which the prediction is being made does not differ substantially from the sites already observed. For example, the San Diego treatment effects could have been predicted based on observable site characteristics, but the Riverside effects are consistently underestimated

Do Financial Incentives Affect Fertility?

This paper investigates how fertility responds to changes in the price of a marginal child and in household income. We construct a large, individual-level panel data set of married Israeli women during the period 1999–2005 that contains fertility histories and detailed controls. We exploit variation in Israel’s child subsidy program to identify changes in the price of a marginal child (using changes in the subsidy for a marginal child) and to instrument for household income (using changes in the subsidy for infra-marginal children). We find a significant and positive price effect on fertility: the mean level of marginal child subsidy produces a 7.8 percent increase in fertility. There is a positive effect within all religious and ethnic subgroups, including the ultra-Orthodox Jewish population, whose social and religious norms discourage family planning. There is also a significant price effect on fertility among women who are close to the end of their lifetime fertility, suggesting that at least part of the price effect is due to a reduction in total fertility. As expected, the child subsidy has no effect in the upper range of the income distribution. Finally, consistent with the predictions of Becker (1960) and Becker and Tomes (1976), we find that the income effect is small in magnitude and is negative at low income levels and positive at high levels.

Do interest rates matter? credit demand in the Dhaka Slums

If the demand for credit by the poor changes little when interest rates increase, lenders can raise fees to cost-covering levels without losing customers. This claim is at the core of sustainable microfinance strategies that aim to provide banking services to the poor while eschewing long-term subsidies, but, so far, there is little direct evidence of this. This paper uses data from SafeSave, a credit cooperative in the slums of Dhaka, Bangladesh, to examine how sensitive borrowers are to increases in the interest rate on loans. Using unanticipated between-branch variation in the interest rate we estimate interest elasticities of loan demand ranging from -0.73 to -1.04. Less wealthy accountholders are more sensitive to the interest rate than (relatively) wealthier borrowers (an elasticity of -0.86 compared to -0.26), and consequently the bank’s portfolio shifts away from its poorest borrowers when it increases the interest rate.microfinance; credit; demand