Improving School Accountability Measures

A growing number of states are using annual school-level test scores as part of their school accountability systems. We highlight an under-appreciated weakness of that approach the imprecision of school-level test score means -- and propose a method for better discerning signal from noise in annual school report cards. For an elementary school of average size in North Carolina, we estimate that 28 percent of the variance in 5th grade reading scores is due to sampling variation and about 10 percent is due to other non-persistent sources. More troubling, we estimate that less than half of the variance in the mean gain in reading performance between 4th and 5th grade is due to persistent differences between schools. We use these estimates of the variance components in an empirical Bayes framework to generate filtered' predictions of school performance, which have much greater predictive value than the mean for a single year. We also identify evidence of within-school heterogeneity in classroom level gains, which suggests the importance of teacher effects.

Productivity Spillovers in Healthcare: Evidence from the Treatment of Heart Attacks

A large literature in medicine documents variation across areas in the use of surgical treatments that is unrelated to outcomes. Observers of this phenomena have invoked “flat of the curve medicine” to explain these facts, and have advocated for reductions in spending in high-use areas. In contrast, we develop a simple Roy model of patient treatment choice with productivity spillovers that can generate the empirical facts. Our model predicts that high-use areas will have higher returns to surgery, better outcomes among patients most appropriate for surgery, and worse outcomes among patients least appropriate for surgery, while displaying no relationship between treatment intensity and overall outcomes. Using data on treatments for heart attacks, we find strong empirical support for these and other predictions of our model, and reject alternative explanations such as waste or supplier induced demand, for geographic variation in medical car

Preferences and Heterogeneous Treatment Effects in a Public School Choice Lottery

This paper combines a model of parental school choice with randomized school lotteries in order to understand the effects of being assigned to a first-choice school on academic outcomes. We outline a simple framework in which those who place the highest weight on academics when choosing a school benefit the most academically when admitted. Although the average student does not improve academically when winning a school lottery, this average impact conceals a range of impacts for identifiable subgroups of students. Children of parents whose choices revealed a strong preference for academic quality experienced significant gains in test scores as a result of attending their chosen school, while children whose parents weighted academic characteristics less heavily experienced academic losses. This differential effect is largest for children of parents who forfeit the most in terms of utility gains from proximity and racial match to choose a school with stronger academics. Depending on one's own race and neighborhood, a preference for academic quality can either conflict with or be reinforced by other objectives, such as a desire for proximity and same-race peers.

What Does Certification Tell Us About Teacher Effectiveness? Evidence from New York City

We use six years of data on student test performance to evaluate the effectiveness of certified, uncertified, and alternatively certified teachers in the New York City public schools. On average, the certification status of a teacher has at most small impacts on student test performance. However, among those with the same certification status, there are large and persistent differences in teacher effectiveness. This evidence suggests that classroom performance during the first two years, rather than certification status, is a more reliable indicator of a teacher's future effectiveness. We also evaluate turnover among teachers with different certification status, and the impact on student achievement of hiring teachers with predictably high turnover. Given relatively modest estimates of experience differentials, even high turnover groups (such as Teach for America participants) would have to be only slightly more effective in their first year to offset the negative effects of their high exit rates.

Identifying Provider Prejudice in Healthcare

We use simple economic insights to develop a framework for distinguishing between prejudice and statistical discrimination using observational data. We focus our inquiry on the enormous literature in healthcare where treatment disparities by race and gender are not explained by access, preferences, or severity. But treatment disparities, by themselves, cannot distinguish between two competing views of provider behavior. Physicians may consciously or unconsciously withhold treatment from minority groups despite similar benefits (prejudice) or because race and gender are associated with lower benefit from treatment (statistical discrimination). We demonstrate that these two views can only be distinguished using data on patient outcomes: for patients with the same propensity to be treated, prejudice implies a higher return from treatment for treated minorities, while statistical discrimination implies that returns are equalized. Using data on heart attack treatments, we do not find empirical support for prejudice-based explanations. Despite receiving less treatment, women and blacks receive slightly lower benefits from treatment, perhaps due to higher stroke risk, delays in seeking care, and providers over-treating minorities due to equity and liability concerns.

Parental Preferences and School Competition: Evidence from a Public School Choice Program

This paper uses data from the implementation of a district-wide public school choice plan in Mecklenburg County, North Carolina to estimate preferences for school characteristics and examine their implications for the local educational market. We use parental rankings of their top three choices of schools matched with student demographic and test score data to estimate a mixed-logit discrete choice demand model for schools. We find that parents value proximity highly and the preference attached to a school's mean test score increases with student's income and own academic ability. We also find considerable heterogeneity in preferences even after controlling for income, academic achievement and race, with strong negative correlations between preferences for academics and school proximity. Simulations of parental responses to test score improvements at a school suggest that the demand response at high-performing schools would be larger than the response at low-performing schools, leading to disparate demand-side pressure to improve performance under school choice.

Searching for Effective Teachers with Imperfect Information

Over the past four decades, empirical researchers -- many of them economists -- have accumulated an impressive amount of evidence on teachers. In this paper, we ask what the existing evidence implies for how school leaders might recruit, evaluate, and retain teachers. We begin by summarizing the evidence on five key points, referring to existing work and to evidence we have accumulated from our research with the nation\u27s two largest school districts: Los Angeles and New York City. First, teachers display considerable heterogeneity in their effects on student achievement gains. Second, estimates of teacher effectiveness based on student achievement data are noisy measures. Third, teachers\u27 effectiveness rises rapidly in the first year or two of their teaching careers but then quickly levels out. Fourth, the primary cost of teacher turnover is not the direct cost of hiring and firing, but rather is the loss to students who will be taught by a novice teacher rather than one with several years of experience. Fifth, it is difficult to identify at the time of hire those teachers who will prove more effective. As a result, better teachers can only be identified after some evidence on their actual job performance has accumulated. We then explore what these facts imply for how principals and school districts should act, using a simple model in which schools must search for teachers using noisy signals of teacher effectiveness. The implications of our analysis are strikingly different from current practice. Rather than screening at the time of hire, the evidence on heterogeneity of teacher performance suggests a better strategy would be identifying large differences between teachers by observing the first few years of teaching performance and retaining only the highest-performing teachers