Do High-Stakes Placement Exams Predict College Success?

Community colleges are typically assumed to be nonselective, open-access institutions. Yet access to college-level courses at such institutions is far from guaranteed: the vast majority of two-year institutions administer high-stakes exams to entering students that determine their placement into either college-level or remedial education. Despite the stakes involved, there has been relatively little research investigating whether such exams are valid for their intended purpose, or whether other measures of preparedness might be equally or even more effective. This paper contributes to the literature by analyzing the predictive validity of one of the most commonly used assessments, using data on over 42,000 first-time entrants to a large, urban community college system. Using both traditional correlation coefficients as well as more useful decision-theoretic measures of placement accuracy and error rates, I find that placement exams are more predictive of success in math than in English, and more predictive of who is likely to do well in college-level coursework than of who is likely to fail. Utilizing multiple measures to make placement decisions could reduce severe misplacements by about 15 percent without changing the remediation rate, or could reduce the remediation rate by 8 to 12 percentage points while maintaining or increasing success rates in college-level courses. Implications and limitations are discussed

What Accounts for Gaps in Student Loan Default, and What Happens After

A substantial proportion of student loan borrowers default on their repayment within 12 years of initial college entry. Default rates vary dramatically by race/ethnicity and institution sector: Black entrants and for-profit entrants experience default at much higher rates than others. This Brookings Institution report examines whether these disparities can be explained by other factors and what happens after a default. Differences in student and family background characteristics, including family income and wealth, can account for about half of the Black-White gap in defaults. But even accounting for differences in degree attainment, college GPA, and post-college income and employment cannot fully explain the gap, which remains large and statistically significant at 11 percentage points in the most complete model. Similarly, differences in student and family background characteristics can account for nearly half of the gap in default rates between borrowers from for-profits and public two-year colleges. But the gap is not fully explained by differences in attainment, employment, or earnings. Entering a for-profit is associated with a 10-point higher rate of default, even after accounting for everything else in the model. Fifty-four percent of all defaulters resolved at least one of their defaulted loans via rehabilitation, consolidation, paying in full, or having a loan discharged. At least 14 percent emerged from default and re-enrolled in school. While there is no Black-White difference in resolution rates, White defaulters are more likely to rehabilitate defaulted loans, while Black defaulters are more likely to consolidate. Similarly, defaulters from for-profit institutions were more likely to consolidate and less likely to rehabilitate a defaulted loan than defaulters from public two-year institutions

The Cost of Complexity in Federal Student Aid: Lessons from Optimal Tax Theory and Behavioral Economics

The federal system for distributing student financial aid rivals the tax code in its complexity. Both have been a source of frustration and a focus of reform efforts for decades, yet the complexity of the student aid system has received comparatively little attention from economists. We describe the complexity of the aid system, and apply lessons from optimal tax theory and behavioral economics to show that complexity is a serious obstacle to both efficiency and equity in the distribution of student aid. We show that complexity disproportionately burdens those with the least ability to pay and undermines redistributive goals. We use detailed data from federal student aid applications to show that a radically simplified aid process can reproduce the current distribution of aid using a fraction of the information now collected.

Does the Federal Work-Study Program Really Work—and for Whom?

In this brief, the authors discuss current research regarding both the effectiveness of the Federal Work-Study (FWS) program and its equity in terms of the distribution of funds. The authors highlight findings from recent CAPSEE research which suggests that the program does positively influence students’ college attainment and post-college outcomes. The evidence also suggests that these impacts may be greatest for low-income students and students at public institutions. The authors then discuss how the current process for allocating FWS funds to institutions leaves these very students—those who are most likely to benefit—with the least access to the program. The authors conclude with implications for policy, including potential channels that might be used to maximize the effectiveness of the program

Performance Requirements in Need-Based Aid: What Roles Do They Serve, and Do They Work?

Satisfactory academic progress (SAP) requirements have existed in some form in the federal student aid programs for nearly 40 years—and have become increasingly strict—yet only limited research exists regarding their motivations and consequences. In this brief, the authors discuss two recent CAPSEE studies they conducted that examine the consequences of SAP policy for first-time community college students in two separate, anonymous states. They discuss the underlying motivations for the policy, examine how students are affected, and assess the implications for program efficiency and equity

Pell Grants as Performance-Based Aid? An Examination of Satisfactory Academic Progress Requirements in the Nation's Largest Need-Based Aid Program

The Federal Pell Grant Program is the nation’s largest need-based grant program. While students’ initial eligibility for the Pell is based on financial need, renewal of the award is contingent on their making satisfactory academic progress (SAP)—meeting minimum academic standards similar to those proposed in models of performance-based scholarships. It is not clear how many students are affected by failure to meet SAP standards, or how the policies shape student outcomes. In this study, we draw from literature on performance-based funding and academic probation to consider the potential implications of SAP standards. We describe federal guidelines and illustrate how SAP is evaluated in a statewide community college system. Using administrative data with term-by-term measures of Pell receipt, student grades, attempted and earned credits, persistence, degree attainment, and transfer, we employ regression discontinuity and difference-in-differences approaches to examine the magnitude of SAP failure and its effects. Our results suggest that a substantial portion of Pell recipients at community colleges are at risk for Pell ineligibility due to their failure to meet SAP grade point average (GPA) or credit completion requirements. Approximately a quarter fail to meet the GPA standard alone. When the credit completion requirement is taken into consideration, the first-year SAP failure rate approaches 40 percent. Our preferred difference-in-differences estimates show mixed effects of SAP standards: Failing to meet the GPA requirement has a negative impact on persistence into the second year, but it may improve associate degree attainment and transfer among students who are not discouraged from reenrolling

Performance Standards in Need-Based Student Aid

College attendance is a risky investment. But students may not recognize when they are at risk for failure, and financial aid introduces the possibility for moral hazard. Academic performance standards can serve three roles in this context: signaling expectations for success, providing incentives for increased student effort, and limiting financial losses. Such standards have existed in federal need-based aid programs for nearly 40 years in the form of Satisfactory Academic Progress (SAP) requirements, yet have received virtually no academic attention. In this paper, we sketch a simple model to illustrate not only student responses to standards but also the trade-offs faced by a social planner weighing whether to set performance standards in the context of need-based aid. We then use regression discontinuity and difference-in-difference designs to examine the consequences of SAP failure. In line with theoretical predictions, we find heterogeneous effects in the short term, with negative impacts on persistence but positive effects on grades for students who remain enrolled. After three years, the negative effects appear to dominate. Effects on credits attempted are 2–3 times as large as effects on credits earned, suggesting that standards increase the efficiency of aid expenditures. But it also appears to exacerbate inequality in higher education by pushing out low-performing low-income students faster than their equally low-performing, but higher-income peers

Complexity and Targeting in Federal Student Aid: A Quantitative Analysis

A growing body of empirical evidence shows that some financial aid programs increase college enrollment. Puzzlingly, there is little compelling evidence that Pell Grants and Stafford Loans, the primary federal student aid programs, are effective in achieving this goal. In this paper, we provide an in-depth review of this evidence, which taken as a whole suggests that complexity and uncertainty in the federal aid system undermine its efficacy. We document complexity in the aid system, comparing it in particular to complexity in the tax system. We build on our previous work by showing that complexity in the aid process does little to improve the targeting of both student loans and grants, for both dependent and independent students. We conclude that the current targeting of aid can be reproduced with a much simpler aid process. While we show that the targeting benefits of complexity are small, we further document that the costs are large. We offer new estimates of the compliance costs faced by applicants and the administrative costs borne by the government and colleges. These costs total at least $4 billion per year. The perspective of behavioral economics suggests that the true cost is even higher, since complexity and uncertainty may discourage the target population from applying for student aid.

Assessing Developmental Assessment in Community Colleges

Placement exams are high-stakes assessments that determine many students' college trajectories. The majority of community colleges use placement exams—most often the ACCUPLACER, developed by the College Board, or the COMPASS, developed by ACT, Inc.—to sort students into college-level or developmental education courses in math, reading, and sometimes writing. More than half of entering students at community colleges are placed into developmental education in at least one subject as a result. But the evidence on the predictive validity of these tests is not as strong as many might assume, given the stakes involved—and recent research fails to find evidence that the resulting placements into remediation improve student outcomes. While this has spurred debate about the content and delivery of remedial coursework, it is possible that the assessment process itself may be broken; the debate about remediation policy is incomplete without a fuller understanding of the role of assessment. This Brief examines the role of developmental assessment, the validity of the most common assessments currently in use, and emerging directions in assessment policy and practice. Alternative methods of assessment—particularly those involving multiple measures of student preparedness—seem to have the potential to improve student outcomes, but more research is needed to determine what type of change in assessment and placement policy might improve persistence and graduation rates. The Brief concludes with a discussion of implications for policy and research