Charter Schools and the Road to College Readiness: The Effects on College Preparation, Attendance and Choice

The analysis here focuses on Boston's charter high schools. For the purpose of this report, an analysis of high schools is both a necessity and a virtue. It is necessary to study high schools because most students applying to charters in earlier grades are not yet old enough to generate data on postsecondary outcomes. Charter high schools are also of substantial policy interest: a growing body of research argues that high school may be too late for cost-effective human capital interventions. Indeed, impact analyses of interventions for urban youth have mostly generated disappointing results.This report is interested in ascertaining whether charter schools, which in Massachusetts are largely budget-neutral, can have a substantial impact on the life course of affected students. The set of schools studied here comes from an earlier investigation of the effects of charter attendance in Boston on test scores.The high schools from the earlier study, which enroll the bulk of charter high school students in Boston, generate statistically and socially significant gains on state assessments in the 10th grade. This report questions whether these gains are sustained

Who Benefits from KIPP?

The nation's largest charter management organization is the Knowledge is Power Program (KIPP). KIPP schools are emblematic of the No Excuses approach to public education, a highly standardized and widely replicated charter model that features a long school day, an extended school year, selective teacher hiring, strict behavior norms, and a focus on traditional reading and math skills. No Excuses charter schools are sometimes said to focus on relatively motivated high achievers at the expense of students who are most diffiult to teach, including limited English proficiency (LEP) and special education (SPED) students, as well as students with low baseline achievement levels. We use applicant lotteries to evaluate the impact of KIPP Academy Lynn, a KIPP school in Lynn, Massachusetts that typifies the KIPP approach. Our analysis focuses on special needs students that may be underserved. The results show average achievement gains of 0.36 standard deviations in math and 0.12 standard deviations in reading for each year spent at KIPP Lynn, with the largest gains coming from the LEP, SPED, and low-achievement groups. The average reading gains are driven almost completely by SPED and LEP students, whose reading scores rise by roughly 0.35 standard deviations for each year spent at KIPP Lynn.human capital, charter schools, achievement

Above and beyond the call. Long-term real earnings effects of British male military conscription in the post-war years

We add to the literature on the long-term economic effects of male military service. We concentrate on post-war British conscription into the armed services from 1949 to 1960. It was called National Service and applied to males aged 18 to 26. Based on a regression discontinuity design we estimate the effect of military service on the earnings of those required to serve through conscription. We argue that, in general, we should not expect to find large long-term real earnings among conscripts compared to later birth cohorts of males who were not eligible for call-up. Our empirical evidence firmly rejects the view that conscription entails relative long-term real earnings differences

Research Design Meets Market Design: Using Centralized Assignment for Impact Evaluation

A growing number of school districts use centralized assignment mechanisms to allocate school seats in a manner that reflects student preferences and school priorities. Many of these assignment schemes use lotteries to ration seats when schools are oversubscribed. The resulting random assignment opens the door to credible quasi-experimental research designs for the evaluation of school effectiveness. Yet the question of how best to separate the lottery-generated variation integral to such designs from non-random preferences and priorities remains open. This paper develops easily-implemented empirical strategies that fully exploit the random assignment embedded in a wide class of mechanisms, while also revealing why seats are randomized at one school but not another. We use these methods to evaluate charter schools in Denver, one of a growing number of districts that combine charter and traditional public schools in a unified assignment system. The resulting estimates show large achievement gains from charter school attendance. Our approach generates efficiency gains over ad hoc methods, such as those that focus on schools ranked first, while also identifying a more representative average causal effect. We also show how to use centralized assignment mechanisms to identify causal effects in models with multiple school sectors

The Elite Illusion: Achievement Effects at Boston and New York Exam Schools

Talented students compete fiercely for seats at Boston and New York exam schools. These schools are characterized by high levels of peer achievement and a demanding curriculum tailored to each district's highest achievers. While exam school students do very well in school, the question of whether an exam school education adds value relative to a regular public education remains open. We estimate the causal effect of exam school attendance using a regression-discontinuity design, reporting both parametric and non- parametric estimates. The outcomes studied here include scores on state standardized achievement tests, PSAT and SAT participation and scores, and AP scores. Our estimates show little effect of exam school offers on most students' achievement. We use two-stage least squares to convert reduced form estimates of the effects of exam school offers into estimates of peer and tracking effects, arguing that these appear to be unimportant in this context. Finally, we explore the external validity of RD estimates, arguing that as best we can tell, there is little effect of an exam school education on achievement even for the highest-ability marginal applicants and for applicants to the right of admissions cutoffs. On the other hand, a Boston exam school education seems to have a modest effect on high school English scores for minority applicants. A small group of 9th grade applicants also appears to do better on SAT Reasoning. These localized gains notwithstanding, the intense competition for exam school seats does not appear to be justified by improved learning for a broad set of students.

Measuring human capital using global learning data

Human capital—that is, resources associated with the knowledge and skills of individuals—is a critical component of economic development1,2. Learning metrics that are comparable for countries globally are necessary to understand and track the formation of human capital. The increasing use of international achievement tests is an important step in this direction3. However, such tests are administered primarily in developed countries4, limiting our ability to analyse learning patterns in developing countries that may have the most to gain from the formation of human capital. Here we bridge this gap by constructing a globally comparable database of 164 countries from 2000 to 2017. The data represent 98% of the global population and developing economies comprise two-thirds of the included countries. Using this dataset, we show that global progress in learning—a priority Sustainable Development Goal—has been limited, despite increasing enrolment in primary and secondary education. Using an accounting exercise that includes a direct measure of schooling quality, we estimate that the role of human capital in explaining income differences across countries ranges from a fifth to half; this result has an intermediate position in the wide range of estimates provided in earlier papers in the literature5–13. Moreover, we show that average estimates mask considerable heterogeneity associated with income grouping across countries and regions. This heterogeneity highlights the importance of including countries at various stages of economic development when analysing the role of human capital in economic development. Finally, we show that our database provides a measure of human capital that is more closely associated with economic growth than current measures that are included in the Penn world tables version 9.014 and the human development index of the United Nations15

Breaking Ties: Regression Discontinuity Design Meets Market Design

Many schools in large urban districts have more applicants than seats. Centralized school assignment algorithms ration seats at over-subscribed schools using randomly assigned lottery numbers, non-lottery tie-breakers like test scores, or both. The New York City public high school match illustrates the latter, using test scores and other criteria to rank applicants at \screened” schools, combined with lottery tie-breaking at unscreened \lottery” schools. We show how to identify causal effects of school attendance in such settings. Our approach generalizes regression discontinuity methods to allow for multiple treatments and multiple running variables, some of which are randomly assigned. The key to this generalization is a local propensity score that quantifies the school assignment probabilities induced by lottery and non-lottery tie-breakers. The local propensity score is applied in an empirical assessment of the predictive value of New York City’s school report cards. Schools that receive a high grade indeed improve SAT math scores and increase graduation rates, though by much less than OLS estimates suggest. Selection bias in OLS estimates is egregious for screened schools

Explaining Charter School Effectiveness

Estimates using admissions lotteries suggest that urban charter schools boost student achievement, while charter schools in other settings do not. We explore student-level and school-level explanations for these differences using a large sample of Massachusetts charter schools. Our results show that urban charter schools boost achievement well beyond ambient non-charter levels (that is, the average achievement level for urban non-charter students), and beyond non-urban achievement in math. Student demographics explain some of these gains since urban charters are most effective for non-whites and low-baseline achievers. At the same time, non-urban charter schools are uniformly ineffective. Our estimates also reveal important school-level heterogeneity in the urban charter sample. A non-lottery analysis suggests that urban schools with binding, well-documented admissions lotteries generate larger score gains than under-subscribed urban charter schools with poor lottery records. We link the magnitude of charter impacts to distinctive pedagogical features of urban charters such as the length of the school day and school philosophy. The relative effectiveness of urban lottery-sample charters is accounted for by over-subscribed urban schools' embrace of the No Excuses approach to education.

Breaking Ties: Regression Discontinuity Design Meets Market Design

Centralized school assignment algorithms must distinguish between applicants with the same preferences and priorities. This is done with randomly assigned lottery numbers, nonlottery tie-breakers like test scores, or both. The New York City public high school match illustrates the latter, using test scores, grades, and interviews to rank applicants to screened schools, combined with lottery tie-breaking at unscreened schools. We show how to identify causal effects of school attendance in such settings. Our approach generalizes regression discontinuity designs to allow for multiple treatments and multiple running variables, some of which are randomly assigned. Lotteries generate assignment risk at screened as well as unscreened schools. Centralized assignment also identifies screened school effects away from screened school cutoffs. These features of centralized assignment are used to assess the predictive value of New York City’s school report cards. Grade A schools improve SAT math scores and increase the likelihood of graduating, though by less than OLS estimates suggest. Selection bias in OLS estimates is egregious for Grade A screened schools

'Making it count': incentives, student effort and performance

This paper examines how incentives to participate in online assessments (quizzes) affect students’ effort and performance. Our identification strategy exploits within-student weekly variation in incentives to attempt online quizzes. We find tournament incentives and participation incentives to be ineffective in increasing quiz participation. In contrast, making the quiz counts towards the final grade substantially increases participation. We find no evidence of displacement of effort between weeks. Using a natural experiment which provides variation in assessment weighting of the quizzes between two cohorts, we find that affected students obtain better exam grades. We estimate the return to 10% assessment weighting to be around 0.27 of a standard deviation in the in-term exam grade. We find no evidence that assessment weighting has unintended consequences, i.e., that increased quiz effort: displaces effort over the year; reduces other forms of effort; or reduces (effort and thus) performance in other courses. Finally, assessment weighting induced effort increases most for students at and below median ability, resulting in a reduction of the grade gap by 17%