Killeen, Kieran M.

English

ScholarWorks @ UVM

University of Vermont UVM ScholarWorks James M. Jeffords Center for Policy Research 2012 Educational Productivity and Opportunities to Learn: An overview of concepts and discussion points Kieran M. Killeen University of Vermont Follow this and additional works at: https://scholarworks.uvm.edu/jmjcpr  Part of the Public Affairs, Public Policy and Public Administration Commons Recommended Citation Killeen, Kieran M., "Educational Productivity and Opportunities to Learn: An overview of concepts and discussion points" (2012). James M. Jeffords Center for Policy Research. 7. https://scholarworks.uvm.edu/jmjcpr/7 This Article is brought to you for free and open access by UVM ScholarWorks. It has been accepted for inclusion in James M. Jeffords Center for Policy Research by an authorized administrator of UVM ScholarWorks. For more information, please contact donna.omalley@uvm.edu. IntroductionVermont’s Act 156 of 2012 designated a Working Group, with a mission to “review and evaluate how Vermont’s current education system allocates financial and other resources in a way that promotes high  quality, equitable educational opportunities for students throughout the state and how impediments to opportunity, such as poverty and substance abuse, may be mitigated.”  This Brief presents a summary of educational research and theory in support of this mission.  There are three sections to the Brief, each concluding with a series of guiding questions to assist the deliberations of the Working Group. These deliberations will center around how best to support a new and more comprehensive system for monitoring Vermont’s education productivity.A Study of Educational ProductivityResearchers have examined many aspects of education, mainly by trying to decompose individual parts of an increasingly complicated system and examining their functioning through research. Much research has focused on targeted audiences like children and young adolescents, but also important actors within the education system (e.g. teachers, administrators, parents and policy makers). Other research focuses on structures that appear to mediate the educational system (e.g. schools, districts or supervisory unions) and the resources expended in these systems.  Simple questions about how children learn often are broken up by researchers into studies of how the variation in teaching techniques influences student learning, or how curricular standards alter teacher practices. In the field of educational finance and policy studies, efforts to isolate important school and non-school factors that influence student achievement fall under models termed education production functions. Rice and Schwartz (2008) define education production functions as those that link “school inputs to educational outcomes and identifies the impact of changes in inputs (e.g. teachers) on student outcomes (e.g. achievement as measured by test scores).” These models, the authors go on to note, grow increasingly complex as the productivity of education is reduced to a narrowly measured set of inputs and outcomes. Figure 1 below depicts a typical presentation of a production function model:Figure 1.    A Basic Education Production FunctionThese models and approaches have been helpful, but also struggle to pinpoint which key inputs constitute key measures of education resources, as well as which outcomes are appropriate to measure productivity. For these reasons, as well as in response to calls for more refined approaches to the study of educational productivity, researchers have turned to other approaches.Increasingly, critics have argued that more progress in understanding childhood learning and educational systems may be had through alternative and non-traditional approaches to understanding what works in education. One dominant effort in the education field involves the broader incorporation of randomized field trials or experimental designs into the study of education. Randomized field trials are a specific method designed to isolate the effect of a program on human behavior (e.g. learning; National Research Council, 2004). Randomized field trials include treatment and control  groups, an approach more commonly used in the health sciences, to limit known sources of error and bias.  The incorporation of randomized field trials proliferated under the Bush era of education reform, spirited mainly by the US Department of Education.Efforts to examine what experimental designs will offer the field of education finance and policy is the subject of a recent stand alone issue of the journal Education Finance and Policy. In this recent issue, four articles are presented that show how experimental designs may be used to examine local, state and large program level effectiveness in the study of education. For example Glazerman (2012) describes a large scale evaluation of the Teach For America (TFA) program, a study that involved the random assignment of students to TFA and non-TFA led Educational Productivity and Opportunities to Learn:An overview of concepts and discussion pointsA PUBLICATION OF THE UNIVERSITY OF VERMONT JAMES M. JEFFORDS CENTERIssue Brief Vol 2, No 3                 September 2012by Kieran M. Killeenclassrooms. This article and others in the special issue offer many caveats and cautions in the proliferation of experimental designs in education. (See Schanzenbach, 2012).Complexity Theory as a Context for the Working GroupOther new techniques are being brought to bear on the study of education.  For example, complexity theory can be used to appreciate the interactions between micro-level learning processes among students, their classrooms, schools, communities and families.  Johnson (2008) states that “the idea of a school as a complex system in which developmental processes and outcomes emerge from a complex interaction among systemic layers is consistent with what seems to be intuitively known by many educators – that the rate of academic achievement is not a simple monotonic function that increases toward absolute proficiency. Instead, any mathematical model for expected rate of change in academic achievement of a particular school must build the ecological systems of the school into the equation as parameters.” Complexity theory and their associated models attempt to do just this by employing analytical techniques that differ substantially from more traditional linear approximations of education productivity. Prior to the first official meeting of the Working Group, a small group of Vermont legislators and state officials who had been instrumental in the writing of the legislation that created the Working Group assembled in the Governor’s Office.   In response to questions from members of that group a presentation was made on the relationship of Complexity Theory to the problem of understanding the relationships among inputs and outcomes of the educational system.  This presentation was led by Dr. Chris Koliba of the University of Vermont. In Figure 2 below, Koliba depicts student learning not as a linear outgrowth of key resource inputs that may or may not generate opportunities to learn, but rather views that student learning as a dynamic outcome dependent on many overlapping and continuous systems. Researchers like Maroulis et al. (2010) have argued similarly that complexity theory could be used to examine education policy through adaptive approaches to policy studies, including agent based models. A variety of authors have argued that complex systems theory offers new insights into understanding basic learning systems (e.g. Jacobson and Wiliensky et al 2008). Others have utilized specific complex system techniques for the study of important education policy interventions. For example, Mauralis (2010) examined the movement of students across districts using agent based modeling to evaluate school choice programs.   (C. Koliba, 2012)Figure 2. A Complex Systems Perspective of Education ProductionMore specifically, the study of educational outcomes, including education spending, has been approached through neural network analytical techniques, another common class of techniques utilized by complex systems thinkers. Baker and Richards (1999) compared and contrasted the predictive validity of traditional econometric or regression based techniques versus three specific neural network techniques. They found some advantage to using neural network techniques given large panels of spending data over time.  The notion that aspects of education ought to be examined using analytical techniques capable of handling the rich and multi-layered dimensions that influence student learning is not lost on researchers. The tension comes not in knowing how to analyze but rather the frustrating position of knowing how best to examine the health of a system, with rather limited data, and how best to communicate the results to diverse audiences. Equal Educational Opportunity in the Vermont ContextLegislative Working Group Chairperson Senator Kevin Mullin stated that “The contract with the Jeffords Center [UVM] needs to focus on the topic and charge at hand which is, are students in Vermont receiving equal educational opportunities and what type of policies can ensure that they are?” (K. Mullin, personal communication, July 2012).This is a particularly important and framing question that will organize the effort of the Legislative Working Group. It is important for several reasons, mainly in the ways that it articulates new foci in the study of equal educational opportunities. Page 2      A PUBLICATION OF THE UNIVERSITY OF VERMONT JAMES M. JEFFORDS CENTERDiscussion Questions1. What does educational productivity mean in the Vermont context? How have productivity ques-tions been asked and answered?2. What are the main indicators of education pro-ductivity that warrant study in Vermont?3. Are the economics of education productivity an interest? That is, is there interest in understand-ing both if programs are effective, but also if they are cost-effective?Figure 3 below presents a systems perspective of education in that outcomes are presented as a function of transformed educational inputs. Inputs, much like a factory model of manufacturing, are manipulated by processes towards the creation of defined outcomes. This model is important as it depicts various places where notions of equality may be examined. Figure 3.  Equal Educational OpportunityVermont’s new baseline in education finance history came about in the late 1990s with the passage of the Act 60 school finance legislation. Act 60 defined equal educational opportunity at the input side of education. Equality was defined at the outset, namely by how much resources were available at the district level to expend on education. Equality was operationalized by this piece of legislation as the even distribution of fiscal resources across districts. Low spending districts were granted additional resources by the State through a new school finance system distribution system; high spending districts were given incentives to cap their spending. Act 60 and also through subsequent revisions to the Act, made it more costly for wealthy localities to raise property tax revenue to spend locally. Various researchers and consultants have examined whether Act 60 narrowed the variation in education spending across school districts and determined that the goals have been met (See Downes, 2004; Schmidt and Scott, 2006; Sass, 2006).Equality in the production of education may, however, be conceptualized in more than this one way. There are three common ways that the field of education research has explored equality. Defined above, equality as a function of the even distribution of key inputs like education resources may be defined as a form of horizontal equity. Horizontal equity means the equal treatment of equals, in this example, no one district shall be treated differently than another in a system. A second form of equality is often termed vertical equity. In a vertically equitable education system fairness is observable when unequals are treated unequally. In a vertically equitable system more resources are devoted to students that necessitate them. For example, special education students require more intensive contact time with educators to meet standards that on average are met by regular education students with much less educator contact time. Applying a similar or identical amount of contact time for these two types of students (unequals) would be viewed as unfair. In this example, the notion of horizontal and vertical equity are in competition. A third model of education quality focuses upon the distribution and stratification of outcomes of the educational system. Some questions will help frame this focus on educational opportunities. Do children demonstrate the same or similar outcomes (e.g. college attendance, test scores, civic engagement) upon leaving an educational system? If outcomes differ, do these outcomes differ by recognizable ways? Do we find that students of particular racial or ethnic groups have relatively even or uneven access to post-secondary opportunities compared with their peers, or graduates of their same high schools?Educational production systems may and often do wrestle with multiple conceptions of equality simultaneously. Education finance systems routinely try to adjust how resources are distributed through policies that favor targets of horizontal or Page 3      A PUBLICATION OF THE UNIVERSITY OF VERMONT JAMES M. JEFFORDS CENTERDiscussion Questions1. Does the analytical approach to evaluating and understanding Vermont’s education system matter?2. From the perspective of a Legislator, is there a downside to using certain types of analysis to understand Vermont’s education system?3. What are some possible uses of randomized experiments to study Vermont’s education sys-tem? How have randomized experiments been used in other sectors of Vermont, and can they be applied to the study of education?Discussion Questions1. What does this Committee mean by equal edu-cational opportunity?  Are you focused on the input, process or outcomes of the system?2. Is the Committee interested in exploring the ad-vancement of equity adjustments to Vermont’s school finance system based on student need?3. Are you interested in learning more about what educational questions may be approached through the development of new data systems? For example:a. What findings might result from accounting for resource expenditure at the school versus district level?b. What findings are possible if individual student records were linked to physical property addresses?c. What findings are possible if teacher personnel records were linked with job application history or students?vertical equity. In Vermont, very little has been done to specifically target the amount of resources that flow to particular students or districts. The system favors input based equality over other adjustments. Specific aid programs, for example, to small schools or schools with particularly high poverty levels or other forms of challenging student environments are not the target of specific policies. Such policies are often created to direct resources in a way that may ameliorate or even compensate education agencies for extraordinary costs associated with challenging student body populations. However Vermont’s experience with vertical aid adjustments is more limited, particularly in comparison with other states. ConclusionThis brief was tailored to help participants in the Working Group consider ways that educational systems have been studied both traditionally, and through more innovative approaches. Three major sections are offered to help Working group members think about research methods in the examination of educational processes and outcomes. In the final section Working Group members are encouraged to think more broadly about the concept of equal educational opportunity, not just as a study of outcomes in education, but as an approach to understanding the processes of education too.Acknowledgements:The author is very grateful for the important contributions made by:  Laurie Eddy, Bud Meyers and John RogersReferences:Baker, B., & Richards, C. (1999).  A comparison of conventional linear regression methods and neural networks for forecasting educational spending. Economics of Education Review, 18(4).Downes, Thomas A.  [2004] “School Finance Reform and School Quality: Lessons from Vermont,” in John Yinger, editor, Helping Children Left Behind (Cambridge, MA: MIT Press).Glazerman, S. (2012). Random assignment within schools: Lessons learned from the Teach for America Experiment. Education Finance and Policy, 2(2), 124-142.Johnson, E.S. (2008). Ecological Systems and Complexity Theory: Toward an Alternative Model of Accountability in Education Complicity: An International Journal of Complexity and Education, 5 (1), 1-10.Maroulis, S., Guimerà, R., Petry, H., Stringer, M.J., Gomes, L.M., Amaral, L.A.N., Wilensky, U. (2010). Complex Systems View of Educational Policy Research. Science, 1 October 2010: 330 (6000), 38-39.National Research Council. (2004). Implementing Randomized Field Trials in Education: Report of a Workshop. Committee on Research in Education. L. Towne and M. Hilton, Eds. Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.Page 4      A PUBLICATION OF THE UNIVERSITY OF VERMONT JAMES M. JEFFORDS CENTERRice, J.K. and Schwartz, E. (2008). Toward an understanding of productivity in education. Handbook of Research in Education Finance and Policy, Helen Ladd and Ted Fiske (eds.). Routledge Press: New York, NY.Schanzenbach, D.W. (2012). Limitations of experiments in education research, Education Finance and Policy, Vol 2: 2, 219-232.Sass, Darcy Rollins. (2006). School Finance in Vermont: Balancing Equal Education and Fair Tax Burdens. State Tax Notes, April 2, 33–42.Schmidt, Stephen J. and Scott, K. (2006).  Reforming Reforms: Incentive Effects in Education Finance in Vermont.  Education Finance and Policy 1(4): 441-464.AuthorKieran M. Killeen is a nationally recognized expert in the field of education finance and policy studies. Dr. Killeen is a former special education teacher from Louisiana and participant in the Teach for America national teacher corps.  He is currently an Associate professor of Educational Leadership and Policy Studies at the Univerity of Vermont and has taught graduate courses in education finance and policy, organizational studies, and survey research methods.  Dr. Killeen holds a Ph.D. in the Social and Philosophical Foundations of Education and a M.R.P. in City and Regional Planning from Cornell University.James M. Jeffords Center102 Farrell Hall, 210 Colchester AvenueBurlington, VT 05405Email:JeffordsCenter@uvm.eduWebsite:  www.uvm.edu/~jeffords

Educational Productivity and Opportunities to Learn: An overview of concepts and discussion points

https://scholarworks.uvm.edu/cgi/viewcontent.cgi?article=1006&context=jmjcpr

Educational Productivity and Opportunities to Learn: An overview of concepts and discussion points

Abstract

Similar works

Full text

Available Versions

ScholarWorks @ UVM