Estimating the Cost-Effectiveness of a National Program that Impacts High School Graduation and Postsecondary Enrollment

This dissertation was designed to provide an example of an application of the ingredients method of cost-effectiveness analysis of Talent Search, a nation-wide federally-funded program that targets low-income students who promise to be the first generation in their families to attend college. The program serves students in grades 6 to 12 to increase the rates of high school graduation and postsecondary enrollment. Because Talent Search is a multi-site and a multioutput program, the analyses allowed for the exploration of two complexities in conducting cost effectiveness analyses: site-level variation within a program and combining multiple outcomes to evaluate a program’s efficiency. My results show that variation in costs, cost-effectiveness ratios, and benefit-cost ratios was wide across sites. This suggests that future work should include site-level analyses to provide a range for a program’s costs and cost-effectiveness and to provide policy relevant site level examples of the resources utilized to implement a program. Because the outcomes of Talent Search have monetary values in the labor market, I combined the program’s impacts on high school completion and postsecondary enrollment to estimate the additional income generated by the program. These findings suggest that the benefits of Talent Search outweigh the costs on average. However, the variability I find across sites illustrates that more investigation and development is needed to improve the productivity of the program and to reduce inequities across sites. Four important contributions were made with this work: it provides an in depth example of applying the ingredients method to a complex program, it suggests that future work should include site-level analyses, it indicates that retrospective work is limited and future work should be devoted to incorporating the ingredients method contemporaneously into impact evaluations, and it informs policymakers about the cost-effectiveness of Talent Search and the ways in which cost-effectiveness varies across sites

Empirical Support for Establishing Common Assumptions in Cost Research in Education

The economic evaluation of educational policies and programs employing the ingredients method for cost, cost-effectiveness, or benefit-cost analysis is no exception to the critique that economic models require an untenable number of assumptions. Educational economists must make assumptions due to two sources of uncertainty: model uncertainty, as in the well-documented debate over the selection of the appropriate social discount rate to calculate present value and empirical uncertainty due to the infeasibility of gathering sufficiently detailed data on all resources. This paper highlights the frequency of empirical assumptions made in the education literature and proposes a set of harmonized assumptions to address empirical uncertainty that can be used to increase comparability of economic evaluation across programs and across studies. By building consensus on a set of reasonable, empirically derived assumptions that are selected so as to minimally distort the results of evaluations, differences in costs, cost effectiveness, and benefit-cost ratios can be more confidently ascribed to meaningful differences in resource use, program implementation, and program effectiveness, as opposed to differences in choices made by the analyst

Addressing Leandro: Supporting Student Learning by Mitigating Student Hunger

Many students face barriers that prevent them from reaching their full potential in school and beyond. Although some of these barriers are outside the domain of education, solving hunger is one challenge that is both important for school performance and feasible as a policy option. This report reviews the economic importance of investing in strategies to reduce hunger among students, highlights innovative approaches available to schools and districts, and reviews state-level policies to mitigate this challenge for students in North Carolina

Minnesota Reading Corps Pre-K Program Cost Analysis

The Minnesota Reading Corps (MRC) program is a statewide AmeriCorps early literacy initiative that aims to foster emergent literacy skills of children to ensure reading proficiency by the end of grade 3. MRC and its host organization, Reading & Math, Inc. (RMI), aim to address the resource gaps within under-resourced schools by bringing AmeriCorps members into Pre-K classrooms to provide literacy enrichment for the whole class and tutoring services for specific at-risk students. An impact evaluation of the program conducted in 2013-2014 by the University of Chicago-based research center, NORC, showed positive impacts on emergent literacy outcomes for 3-, 4-, and 5-year-olds (Markovitz et al., 2015). Building on the existing evidence on the program effectiveness, this study estimates the costs of providing the MRC Pre-K program that are associated with the impact measured by the 2013-2014 impact evaluation. Rigorous economic evaluations of educational interventions provide important information about the resources necessary to implement a program. Such evaluations bridge the gap between knowledge on program implementation and program impact by identifying the resources utilized to generate outcomes of interest. As such, cost analyses intend to inform policymakers facing decisions to replicate or scale up a program, or trade-offs related to limited resources. Our study used the ingredients method—an approach widely applied to examine costs of educational interventions—to estimate the MRC Pre-K program’s cost (Levin, McEwan, Belfield, Bowden & Shand, 2018). We conducted interviews, surveys, and classroom observations, as well as reviews of program documents, administrative records and past research in order to collect data on all resources utilized to derive program impact based on its theory of change. Wherever important data were missing, we used a Monte Carlo simulation strategy to explore site-level variation on resource use and costs. Overall, the costs of MRC were identified as $1.5 million per year to serve 1,261 students across twenty-five schools, or$1,210 per pupil on average. Costs were found to vary substantially by site, by ingredient category and by who bears the burden of the costs across the 25 sites evaluated. Our analyses of the distribution of who bears the costs suggest that the average cost per student per site borne by schools ranges from $680 to$210, or approximately 25% of the total costs per student. Comparable cost estimates are limited by a lack of similar Pre-K programs that have conducted both impact and cost analysis evaluations. Our study is one of the few rigorous cost analyses in Pre-K programs conducted alongside effectiveness research on a supplemental Pre-K literacy program to date. Nevertheless, these results suggest the Minnesota Reading Corps program leverages a substantial amount of resources into Pre-K classrooms in a way that feasibly distributes costs

Mobilizing Volunteer Tutors to Improve Student Literacy

This report evaluates the implementation, impacts, and costs of Reading Partners, a school-based early-intervention literacy program that relies on volunteer tutors. The work is a partnership between MDRC and CBCSE. The findings illustrate that the program is an efficient option for schools to consider in providing supplemental reading services to students who are not reading at grade-level

A Benefit-Cost Analysis of City Connects

Schools have historically and increasingly played an important role in providing services to meet students’ social and emotional, family, health, and academic needs. Coordinating these services in a way that is strategically aligned with a school’s academic mission and that efficiently addresses the needs of all students is often challenging and costly. This study is an initial investigation of Boston College’s City Connects program, which supports students and schools by evaluating the needs of all students in a school and connecting them to services that are largely provided by community partner organizations. The program aims to help students by connecting them with an individualized set of services to address their academic, social/emotional, family, and health needs. The program also aims to assist schools by connecting them with community agencies and service providers, and streamlining student support referral and management to make the process of providing comprehensive approaches to supporting student learning more strategic and efficient. Prior research has shown evidence of effectiveness of City Connects in terms of increased achievement and educational attainment relative to similar schools that have not implemented the program (City Connects Progress Report, 2014; Walsh, et al., 2014a; 2014b). These positive effects must be weighed against the program’s costs in a benefit-cost analysis to determine whether the program is a worthwhile social investment. This report shows that City Connects provides a whole-school comprehensive service at relatively low cost to the schools—schools themselves only bear about 10% of the core costs of the program. However, the methodological complexity of this work is entailed in the estimation of the total cost when considering the partnerships with community organizations. The results show that the total cost of six years of participation in City Connects from kindergarten through fifth grade (the dosage under which effects were measured) is $4,570 per student, which includes a portion of the costs of the community partner services received by the students in City Connects schools. Depending on what share of the community partner services are considered to be above and beyond the baseline level, the total cost estimate can range from$1,540 to $9,320 per student. Under the model that is most plausible based on implementation data, the benefit-cost ratio is 3.0 and the net benefits are$9,280 per student. This result implies that providing the program to a cohort of 100 students over six years would cost society $457,000 but yield$1,385,000 in social benefits, for a net benefit of $928,000. Even under the most conservative assumptions regarding costs and benefits, the program’s benefits exceed its costs. Sensitivity tests show that the benefit-cost ratio lies somewhere between 1 and 11.8, with a best estimate of$3.00 in benefits per dollar of cost. Further research can investigate the relationship between the program, schools, and community partners and how services provided by partners compare in treatment versus comparison schools

A Benefit-Cost Analysis of a Long-Term Intervention on Social and Emotional Learning in Compulsory School

There is growing evidence that social and emotional skills can be taught to students in school and teaching these skills can have a positive effect on later outcomes, such as better mental health and less drug use. This paper presents a benefit-cost analysis of a longitudinal social and emotional learning intervention in Sweden, using data for 663 students participating in the evaluation. Intervention costs are compared against treatment impact on self-reported drug use. Pre-test and post-test data are available. Since follow-up data for the participants´ drug use as adults is not available, informed projections have been made. Net present monetary values are calculated for the general public and society. The results show that students in the treatment group report decreasing use of drugs over the five year long intervention, the value of which easily outweighs the intervention costs

Cost-effectiveness Analysis of Interventions that Improve High School Completion

This report demonstrates the methods of cost-effectiveness analysis as applied to several educational programs that have been shown to improve the rate of high school completion