The causal interpretation of two-stage least squares with multiple instrumental variables

Empirical researchers often combine multiple instrumental variables (IVs) for a single treatment using two-stage least squares (2SLS). When treatment effects are heterogeneous, a common justification for including multiple IVs is that the 2SLS estimand can be given a causal interpretation as a positively weighted average of local average treatment effects (LATEs). This justification requires the well-known monotonicity condition. However, we show that with more than one instrument, this condition can only be satisfied if choice behavior is effectively homogeneous. Based on this finding, we consider the use of multiple IVs under a weaker, partial monotonicity condition. We characterize empirically verifiable sufficient and necessary conditions for the 2SLS estimand to be a positively weighted average of LATEs under partial monotonicity. We apply these results to an empirical analysis of the returns to college with multiple instruments. We show that the standard monotonicity condition is at odds with the data. Nevertheless, our empirical checks reveal that the 2SLS estimate retains a causal interpretation as a positively weighted average of the effects of college attendance among complier groups.publishedVersio

Who Benefits from KIPP?

The nation's largest charter management organization is the Knowledge is Power Program (KIPP). KIPP schools are emblematic of the No Excuses approach to public education, a highly standardized and widely replicated charter model that features a long school day, an extended school year, selective teacher hiring, strict behavior norms, and a focus on traditional reading and math skills. No Excuses charter schools are sometimes said to focus on relatively motivated high achievers at the expense of students who are most diffiult to teach, including limited English proficiency (LEP) and special education (SPED) students, as well as students with low baseline achievement levels. We use applicant lotteries to evaluate the impact of KIPP Academy Lynn, a KIPP school in Lynn, Massachusetts that typifies the KIPP approach. Our analysis focuses on special needs students that may be underserved. The results show average achievement gains of 0.36 standard deviations in math and 0.12 standard deviations in reading for each year spent at KIPP Lynn, with the largest gains coming from the LEP, SPED, and low-achievement groups. The average reading gains are driven almost completely by SPED and LEP students, whose reading scores rise by roughly 0.35 standard deviations for each year spent at KIPP Lynn.human capital, charter schools, achievement

Charter Schools and the Road to College Readiness: The Effects on College Preparation, Attendance and Choice

The analysis here focuses on Boston's charter high schools. For the purpose of this report, an analysis of high schools is both a necessity and a virtue. It is necessary to study high schools because most students applying to charters in earlier grades are not yet old enough to generate data on postsecondary outcomes. Charter high schools are also of substantial policy interest: a growing body of research argues that high school may be too late for cost-effective human capital interventions. Indeed, impact analyses of interventions for urban youth have mostly generated disappointing results.This report is interested in ascertaining whether charter schools, which in Massachusetts are largely budget-neutral, can have a substantial impact on the life course of affected students. The set of schools studied here comes from an earlier investigation of the effects of charter attendance in Boston on test scores.The high schools from the earlier study, which enroll the bulk of charter high school students in Boston, generate statistically and socially significant gains on state assessments in the 10th grade. This report questions whether these gains are sustained

School choice, school quality, and human capital : three essays

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Economics, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (p. 173-180).This dissertation consists of three essays covering topics in the economics of education. Two common threads connect these essays: first, a focus on the inputs and practices driving variation in effectiveness across educational programs; and second, an interest in the relationships between students' preferences, characteristics, and returns to human capital investment. In the first chapter, I develop and estimate a structural model of school choice that links students' decisions to apply to and attend charter schools in Boston, Massachusetts to their potential achievement test scores in charter schools and public schools. This chapter is motivated by a growing literature that uses randomized entrance lotteries to show that urban charter schools, including those in Boston, substantially increase test scores and close racial achievement gaps among their applicants. A key policy question is whether charter expansion is likely to produce similar effects on a larger scale. To address this question, I use the structural model to predict the effects of charter expansion for the citywide achievement distribution in Boston. Estimates of the model suggest that charter applicants are negatively selected on achievement gains: low-income students and students with low prior achievement gain the most from charter attendance, but are unlikely to apply to charter schools. This form of selection implies that lottery-based estimates understate gains for broader groups of students, and that charter schools will produce substantial gains for marginal applicants drawn in by expansion. Simulations suggest that realistic expansions are likely to reduce the gap in math scores between Boston and the rest of Massachusetts by up to 8 percent, and reduce racial achievement gaps by roughly 5 percent. Nevertheless, the estimates also imply that perceived application costs are high and that most students prefer traditional public schools to charter schools, so large expansions may leave many charter seats empty. These results suggest that in the absence of significant behavioral or institutional changes, the potential gains from charter expansion may be limited as much by demand as by supply. The second chapter, written jointly with Joshua Angrist and Parag Pathak, seeks to explain differences in effectiveness across charter schools. Using a large sample of lotteried applicants to charter schools throughout Massachusetts, we show that urban charter schools boost student achievement, while charter schools in other settings do not. We then explore student-level and school-level explanations for this difference. In an econometric framework that isolates sources of charter effect heterogeneity, we show that urban charter schools boost achievement well beyond that of urban public school students, while non-urban charters reduce achievement from a higher baseline. Student demographics explain some of these gains since urban charters are most effective for non-whites and low-baseline achievers. At the same time, non-urban charter schools are uniformly ineffective. Our estimates also reveal important school-level heterogeneity within the urban charter sample. A non-lottery analysis suggests that urban charters with binding, well-documented admissions lotteries generate larger score gains than under-subscribed urban charter schools with poor lottery records. Using a detailed survey of school practices and characteristics, we link charter impacts to inputs such as instructional time, classroom techniques and school philosophy. The relative effectiveness of urban lottery-sample charters is accounted for by these schools' embrace of the No Excuses approach to urban education, a package of policies that includes strict discipline, increased instructional time, selective teacher-hiring, and a focus on traditional skills. In the third chapter, I use data from the Head Start Impact Study (HSIS), a nationwide randomized trial of the Head Start program, to study the relationship between site-level treatment effects and educational inputs within Head Start. Studies of small-scale, intensive early-childhood programs, including the High/Scope Perry Preschool Project, show that such programs can have transformative effects on human capital and economic outcomes. Evidence for larger-scale programs like Head Start is more mixed. I use the HSIS data to ask whether Head Start centers using practices more similar to successful model programs produce larger short-run effects on cognitive and non-cognitive skills. My results show that while there is significant variation in effectiveness across Head Start centers, centers that are more similar to the Perry Preschool Project on observed dimensions are not more effective. Specifically, Head Start centers using the High/Scope curriculum, the centerpiece of the Perry experiment, do not produce larger gains relative to other centers. Other inputs often cited as essential to the success of the Perry Project, including teacher education, teacher certification, teacher/student ratios, instructional time, and frequency of home visiting, are also unrelated to effectiveness in Head Start. These results suggest that replicating the success of small-scale programs may be difficult, as the effectiveness of such programs may be due to idiosyncratic, unmeasured inputs. JEL Classification: 121, C51, J24by Christopher Ross Walters.Ph.D

A Discrimination Report Card

We develop an Empirical Bayes grading scheme that balances the informativeness of the assigned grades against the expected frequency of ranking errors. Applying the method to a massive correspondence experiment, we grade the racial biases of 97 U.S. employers. A four-grade ranking limits the chances that a randomly selected pair of firms is mis-ranked to 5% while explaining nearly half of the variation in firms' racial contact gaps. The grades are presented alongside measures of uncertainty about each firm's contact gap in an accessible rubric that is easily adapted to other settings where ranks and levels are of simultaneous interest

Explaining Charter School Effectiveness

Estimates using admissions lotteries suggest that urban charter schools boost student achievement, while charter schools in other settings do not. We explore student-level and school-level explanations for these differences using a large sample of Massachusetts charter schools. Our results show that urban charter schools boost achievement well beyond ambient non-charter levels (that is, the average achievement level for urban non-charter students), and beyond non-urban achievement in math. Student demographics explain some of these gains since urban charters are most effective for non-whites and low-baseline achievers. At the same time, non-urban charter schools are uniformly ineffective. Our estimates also reveal important school-level heterogeneity in the urban charter sample. A non-lottery analysis suggests that urban schools with binding, well-documented admissions lotteries generate larger score gains than under-subscribed urban charter schools with poor lottery records. We link the magnitude of charter impacts to distinctive pedagogical features of urban charters such as the length of the school day and school philosophy. The relative effectiveness of urban lottery-sample charters is accounted for by over-subscribed urban schools' embrace of the No Excuses approach to education.

Stand and Deliver: Effects of Boston’s Charter High Schools on College Preparation, Entry, and Choice

We use admissions lotteries to estimate effects of attendance at Boston’s charter high schools on college preparation and enrollment. Charter schools increase pass rates on Massachusetts’ high-stakes exit exam, with large effects on the likelihood of qualifying for a state-sponsored scholarship. Charter attendance also boosts SAT scores sharply and increases the likelihood of taking an Advanced Placement (AP) exam, the number of AP exams taken, and AP scores. Charters induce a substantial shift from 2- to 4-year institutions, though the effect on overall college enrollment is modest. Charter effects on college-related outcomes are strongly correlated with charter effects on earlier tests.Institute of Education Sciences (U.S.) (Grant 08120031)NewSchools Venture Fun

Charter School Demand and Effectiveness: A Boston Update

This report begins with a focus on applications to charter schools. While much attention has been focused on charter school waitlists, waitlist data can be misleading. It includes duplicates, but also includes waitlists that have been rolled over from year to year and might be an unrealistic measure of demand. Instead, the authors investigate three factors related to demand: the yearly percentage of each middle school and high school class that applies to a charter in our lottery sample; the percentage of applicants that receive an offer from a charter school; and where students ultimately attend. This report also examines the demographic makeup of charter school enrollees and compare it to BPS. The next section follows the path of charter school students and report their performance in charters using the evidence from the lotteries. The lottery sample now covers many more charter schools. The 2009 report included findings from eight schools. This report includes MCAS results through 2012 from 12 schools and many more cohorts from the original schools, with additional newly opened schools contributing to the demand analysis. In addition to updating the test score results from the 2009 report, this report breaks down the test score effects by student subgroups. Trends are investigated over time in charter performance and by school groups. Finally, this report gives results using statistical controls, which allow the authros to estimate effects for attending charter schools that do not have sufficient lottery records for the more rigorous lottery based analysis. The lottery sample now contains almost all Boston charter schools with entry grades at middle or high school

Who Benefits from KIPP?

Charter schools affiliated with the Knowledge is Power Program (KIPP) are emblematic of the No Excuses approach to public education. These schools feature a long school day, an extended school year, selective teacher hiring, strict behavior norms and a focus on traditional reading and math skills. We use applicant lotteries to evaluate the impact of KIPP Academy Lynn, a KIPP charter school that is mostly Hispanic and has a high concentration of limited English proficiency (LEP) and special-need students, groups that charter critics have argued are typically under-served. The results show overall gains of 0.35 standard deviations in math and 0.12 standard deviations in reading for each year spent at KIPP Lynn. LEP students, special education students, and those with low baseline scores benefit more from time spent at KIPP than do other students, with reading gains coming almost entirely from the LEP group.