Review of Jerry Muller, The Tyranny of Metrics and Susan Phillips and Kevin Kinser, Accreditation on the Edg

Olt, Phillip A

English

Fort Hays State University

Teacher-Scholar: The Journal of the State Comprehensive University Volume 9 Article 2 January 2020 Book Review: Tyranny of Metrics & Accreditation on the Edge Phillip A. Olt Fort Hays State University, paolt@fhsu.edu Follow this and additional works at: https://scholars.fhsu.edu/ts Recommended Citation Olt, Phillip A. (2020) "Book Review: Tyranny of Metrics & Accreditation on the Edge," Teacher-Scholar: The Journal of the State Comprehensive University: Vol. 9 , Article 2. Available at: https://scholars.fhsu.edu/ts/vol9/iss1/2 This Book Review is brought to you for free and open access by FHSU Scholars Repository. It has been accepted for inclusion in Teacher-Scholar: The Journal of the State Comprehensive University by an authorized editor of FHSU Scholars Repository.   Teacher-Scholar: The Journal of the State Comprehensive University, vol, 9, Fall 2018  Jerry Z. Muller. The Tyranny of Metrics. Princeton: Princeton University Press, 2018. 220 p. ISBN 9780691174952. $24.95.  Susan D. Phillips and Kevin Kinser (eds.). Accreditation on the Edge: Challenging Quality Assurance in Higher Education. Baltimore: Johns Hopkins University Press, 2018. 283 p. ISBN 9781421425443. $44.95.   The scandals in late 2018 of falsely reported data to the well-known U.S. News college rankings revealed the growing reliance in American higher education on quantitative social science for high-stakes decision making and called to mind Campbell’s Law that states, “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor” (p. 34). No other proposition could better serve as a framework to understanding Jerry Muller’s The Tyranny of Metrics or Susan Phillips and Kevin Kinser’s collection Accreditation on the Edge: Challenging Quality Assurance in Higher Education.  In The Tyranny of Metrics, Muller provides an historical exposition of the pervasiveness of this metric fixation, utilizing case studies from education, medicine, law enforcement, military, business, and philanthropy. Though the focus of the book is society as a whole, chief among the case studies is education. Metric fixation infiltrated the education sector of America in the 1990s through primary and secondary schools but quickly spread to higher education. Amidst the sea of quantitative data about higher education, such as graduation rates, professionals must still exercise their judgment to interpret such data, as quantitative data do not independently account for context or meaning. Muller’s central thesis is that such quantitative metrics are often improperly gathered, interpreted, and applied with an overconfidence in their results to describe success and provide accountability. This then produces opportunity costs among institutions by displacing goals, promoting short-termism, stealing employee time, diminishing the utility of the metrics, cascading increasingly complex rules, rewarding luck, discouraging risk-taking, innovation, and cooperation, degrading professional work, and losing overall productivity. Muller argues that metrics with goals set for minimum performance actually lower outcomes. Muller portrays quantitative decision making in near-religious terms, centered on three beliefs: standard quantitative data are better than professional judgment, transparency of quantitative metrics is the best method to assure accountability, and quantifiable measures are the best motivation for people and organizations. At times, his arguments are borderline hyperbole, reasoning for instance that, “Because belief in its efficacy seems to outlast evidence that it frequently doesn’t work, metric fixation has elements of a cult. . . . Metric fixation, which aspires to imitate science, too often resembles faith” (p. 20). Muller does not, however, call for the abolition of quantitative metrics, advocating instead for using them only when most appropriate and only as elements of broader analysis. Book Reviews  2  If metric fixation were to be manifested anywhere in American higher education, it would be in the accreditation process. In their edited collection Accreditation on the Edge: Challenging Quality Assurance in Higher Education, Phillips and Kinser provide insights into the perspectives of accreditors, institutions, policymakers, and consumers. The value of this volume arises from presenting these different perspectives as insiders who understand the complexity of accreditation and how it is perceived throughout their particular sector. The editors largely meet this goal, though with a glaring hole in the institutional perspectives section. There, the authors are described as representing “the providers of higher education” (p. 7), but of the three articles in the institutional section, the first two are written by leaders in a professional organization and a think-tank, both having clear lobbying goals noted on their organizational websites. Those authors have no notable direct experience at institutions of higher education, save one with six years working with institutional endowment investments. The third author, president of Southern New Hampshire University Paul LeBlanc, represents an actual provider of higher education, albeit a non-typical institution as SNHU is a private university with massive online enrollments. Aside from this unfortunate gap, the rest of the book is incredibly insightful in presenting the competing interests in accreditation. Common across all chapters is a recognition of the pressure on accreditation from the competing ideas of accountability and quality improvement, largely driving the prior through standardized quantitative metrics and the latter through professional judgment of peers. The authors representing accreditors somewhat balk at their quantitative regulatory role, describing many of the metrics as overly simplistic, lacking recognition of context and mission of institutions, and prone to misinterpretation. It appears there is some agreement around accreditation re-focusing on quality improvement and perhaps returning regulatory gatekeeping for student aid back to the federal government. The policymaking authors cast accreditation as a means for ensuring return on investment—that the graduates they are investing in contribute commensurately to the economy—and consumer protection. The representatives of consumers, broadly construed with focus on businesses and students, call for the increasing use of standardized metrics that focus on job-preparedness and employment outcomes for graduates.   The quantitative metric fixation in higher education that is often coordinated and enforced through accreditation poses an increasing challenge to state comprehensive universities (SCUs). As regional accreditation serves as the gatekeeper to federal financial aid, SCUs have no real option but to attempt to fulfill the increasing requirements. While struggling with stagnant state appropriations and largely declining on-campus enrollments, the production and analysis of precise data represent an added expense during a time when SCUs are commonly cutting expenditures. This creates an opportunity cost of what is sacrificed by institutions in order to fulfill the high-stakes accreditation mandates for quantitative data. Along with community colleges, SCUs seem hesitant to embrace high-stakes quantitative metrics for their outcomes, as they serve many 3  Teacher-Scholar   students from populations subject to greater adversity and less likely to be successful in the metrics used as proxy measures for institutional quality.   SCUs would be wise to step back from the busyness of survival in the current higher education landscape to consider the trend toward quantitative metrics enforced through accreditation. These two books, in tandem, serve as an excellent point to start that consideration and campus conversation. Resulting discussions may then consider questions like: • How can we balance transparency with gathering honest feedback?  • How can we develop an institutional culture of driving toward excellence rather than regressing to the mandated quantitative minimum? • How can we employ professional judgment, while utilizing quantitative data, to evaluate teaching and program effectiveness? • How can we mitigate the negative consequences of metric fixation, such as rule cascades and the degradation of work? • How can we clarify the nuances of our type of institution for consideration by policymakers in determining high-stakes metrics? • How can we most effectively demonstrate the value added to students by attending our institution? • How can we demonstrate our value to employers and the region of our state that we are committed to serving? • How can we advocate for policy reform that benefits the unique needs of our type of institution and the students we serve? • How will we fund and provide adequate staffing to meet the rising tide of high-stakes metrics for the institution while enrollments lag and appropriations do not provide extra funding for this added work? Though these questions are transferrable across sectors of higher education institutions, they are essential at SCUs. Lacking the prestige and funding of cutting-edge research, athletics recognition, or technical-skills training that benefit other sectors, SCUs are pushed to demonstrate their value in competition with such widely recognized benefits. The current trend of quantitative metric fixation bring directly regulated through the government and indirectly through accreditation skew toward data points that likely will not favor the sector. It is imperative, then, that SCUs both determine how to fund the increasing un-funded mandate of metric fixation and advocate for their sector through qualitative explanation of context, nuance, and mission fulfillment.  Reference Campbell. D. T. (1976). Assessing the impact of planned social change. Journal of MultiDisciplinary Evaluation, 7(15), 3-43.  Phillip Olt Fort Hays State University  

Book Review: Tyranny of Metrics & Accreditation on the Edge

https://scholars.fhsu.edu/cgi/viewcontent.cgi?article=1066&context=ts

Book Review: Tyranny of Metrics & Accreditation on the Edge

Abstract

Similar works

Full text

Available Versions

Fort Hays State University