Croxford, Linda

Edinburgh Research Explorer

     Edinburgh Research Explorer                                      Measuring Performance - Tackling Inequalities?Citation for published version:Croxford, L 2003, Measuring Performance - Tackling Inequalities? CES Briefing, vol. 26, Centre forEducational Sociology.Link:Link to publication record in Edinburgh Research ExplorerDocument Version:Publisher's PDF, also known as Version of recordGeneral rightsCopyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)and / or other copyright owners and it is a condition of accessing these publications that users recognise andabide by the legal requirements associated with these rights.Take down policyThe University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorercontent complies with UK legislation. If you believe that the public display of this file breaches copyright pleasecontact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately andinvestigate your claim.Download date: 05. Apr. 2019 Measuring Performance – Tackling Inequalities? by Linda Croxford No. 26, February 2003   The National Priorities in Education set by the Scottish Parliament include priorities relating to ‘Achievement and Attainment’ and ‘Inclusion and Equality’. The requirement for schools and local authorities to report their progress towards these priorities has focused attention on how best to measure pupil performance and attainment. This Briefing discusses some of the implications of using quantitative measures to do so, including their potential role in identifying and tackling inequalities and underachievement in Scottish education.   }  The measurement of inequalities in pupils’ attainment is an important tool for tackling inequalities. It can identify and focus attention on the aspects of school provision and processes that may be reinforcing inequalities. }  Scottish education currently does not have a system that measures inequalities in pupils’ attainment; a systematic approach needs to be developed including more appropriate methods of assessment. }  The education community tends to be resistant to the measurement of performance, being concerned that the results may be used in a judgemental rather than enabling manner. Some government policies, such as ‘league tables’, create obstacles to more positive approaches to performance monitoring.  }  To measure inequalities in pupils’ attainment, schools and local authorities need to collect information on the range of factors that are associated with inequalities and incorporate them in performance-monitoring systems. }  Analysis of pupil performance should always be based on ‘value-added’ approaches that take account of prior attainment and the various factors associated with inequalities. }  Measurement of differences in pupils’ attainment should begin in the earliest stages of schooling if early intervention strategies are to be successfully implemented. Inequalities are found among pupils entering Primary 1, and at every school stage.  }  Tackling inequalities in educational opportunity requires focused interventions and targeted support. This is likely to require changes in policy and practice at every stage of the education system. Background Since the 1990s, the increasing emphasis by policy makers on raising standards of attainment has led to publication of quantitative measures of school performance and targets for improvement. In Scotland, schools are expected to use performance indicators, including quantitative information, to evaluate their own performance and develop improvement plans (CES Briefing No. 14). Similarly, policy makers in the field of social justice and inclusion have also adopted an approach in which milestones and indicators of progress are monitored. Now, the Scottish Parliament has set National Priorities In Education– defined under the headings: Achievement and Attainment; Framework for Learning; Inclusion and Equality; Values and Citizenship; Learning for Life. Schools and local authorities have been given new responsibilities to report their progress towards achieving these national priorities, and Learning and Teaching Scotland has developed a number of performance measures to support schools and local authorities in monitoring their progress.  Measuring (in)equalities The establishment of Inclusion and Equality as a national priority in Scottish education focuses attention on the need to promote equality in the outcomes of education, including reducing inequalities in attainment. It also creates a new requirement for schools and local authorities to monitor (in)equalities in performance. The measurement of inequalities is important because it can focus attention on the need to tackle aspects of school provision and processes that reinforce inequalities. An example of the productive use of such information is the study of gender differences in performance published in 1998, which caused many schools to examine the causes of perceived underachievement by boys (Tinklin et al., 2001). However, Scottish education currently does not have a system that measures inequalities in pupils’ attainment. In order to measure inequalities in pupils’ performance, information about factors associated with inequalities needs to be collected and incorporated in performance-monitoring systems. Such factors include gender, age, special educational needs, ethnicity, poverty, family structure such as lone-parent families and looked-after status. However, although information about gender, age and special needs is commonly collected, little information about other inequalities is collected or analysed by schools and local authorities. The recent ScotXed initiative aims to collect more consistent data on ethnicity as a result of UK concerns about institutional racism. It would be beneficial if a similar approach could be taken to the measurement of poverty and family background since the measures commonly used as proxies for poverty – entitlement to a free school meal or clothing grant – are very unreliable indicators. The collection of such data can be sensitive, and it is important that it is collected and used with care, in order to challenge inequalities, not justify or reinforce them. We should be clear that it is the effects of poverty or racism that are the ‘targets’ here, not the individual parents and pupils who suffer these problems. If the measurement of inequalities is to be used as a means of identifying and meeting need, then these data must be collected and used in a climate of trust.  Assessing attainment A fundamental requirement for measuring inequalities in pupils’ attainment is a system of assessing pupils on a comparable basis. At present, the most systematic assessments are provided by national examinations at Standard Grade at age 15-16 - this is a very late stage to take action on inequalities. Inequalities have been identified among pupils entering Primary 1, and throughout primary and secondary stages (Croxford, 2001b), and therefore comparable assessments of performance should begin in the earliest stages of schooling. Unfortunately, current assessments in the 5-14 curriculum do not provide appropriate measures for comparing pupils’ attainments. This is because 5-14 national tests were designed to confirm a teacher’s judgement as to whether or not a pupil has attained a specific level, not to measure and compare a range of pupils’ attainments. Learning and Teaching Scotland has recently introduced an initiative to improve assessment practice, including the creation of new assessment materials, but it is not clear whether this will lead to more comparable measures of pupils’ performance.  There has been strong opposition to the use of systematic forms of assessment in primary and early secondary stages in Scotland, because of fears that they may be used in a judgemental rather than enabling manner by government.  Evidence of the impact of the publication of league tables based on Key Stage testing in England and Wales, and their use in appraisal systems, suggests that such fears are well founded. A strategy adopted in a number of Education Authorities in Scotland is to introduce appropriate systems of assessment in all of their schools, but to keep local analysis of differences in performance between pupils, and between schools confidential.  Analysis of differences in pupils’ progress Policy makers tend to use performance indicators that are based on average performance. For example, performance measures for the national priorities are defined in terms of the percentage of pupils achieving five or more awards of the Scottish Credit and Qualifications Framework (SCQF). However, research has shown that the use of averages is misleading, and analysis of performance should be based on pupil-level data using ‘value-added’ approaches that take account of prior attainment (Fitz-Gibbon, 1996). In the value added approach, statistical methods have been developed that model simultaneously the differences between schools and between pupils, and thus take account of the real world situation in which each pupil’s attainment is influenced by the contextual effect of the school intake as well as her/his own prior attainment and characteristics. Most research studies using ‘value-added’ methodology have focused on differences in school effectiveness rather than inequalitie s in pupil achievement but analysis using these statistical methods and based on pupil-level data can provide a number of insights into inequalities. Firstly, it can give an overall picture of inequalities in pupils’ attainment at specific time-points, such as the attainment gap between pupils living in poverty and their more affluent peers, or the gap in attainment between black and white pupils. Secondly, a ‘value-added’ analysis can reveal the extent to which the initial gap in pupils’ attainment gets wider as they progress at different rates. The identification of a growing gap may indicate that aspects of the structures and processes of schooling are exacerbating initial differences in attainment and contributing to increasing levels of inequality. Thirdly, the analysis can identify the effects of social segregation between schools arising from differences in school intake patterns; for example the effects of attending a school in which the majority of pupils live in areas of deprivation compared with a school serving more affluent areas. Finally, if schools record information about interventions aimed at tackling inequalities and underachievement, a value-added analysis can be used to evaluate the impact of the intervention.  An example from the Primary 1 stage An example of a ‘value-added’ analysis of baseline assessment data may help to clarify the methodology (Croxford, 2001a). Primary schools in a number of Scottish local authorities carry out baseline assessment when pupils first enter Primary 1, and repeat the assessment at the end of the Primary 1 stage. The baseline assessments are carried out on a one-to-one basis, and measure aspects of reading-readiness and mathematical concepts. The assessments are used by schools to identify pupils in need of learning support.  In Aberdeen City Council, analysis of the data at the start of Primary 1 showed that pupils had lower baseline scores on average if they were younger than average, received a clothing grant (a measure of poverty), or had English as a second language.  At the end of Primary 1 the relative progress of pupils was analysed using a value-added analysis. It confirmed that attainment at the end of Primary 1 was strongly influenced by baseline attainment at the start of Primary 1, and that the gap between low and high attaining pupils widened. It showed that pupils in receipt of clothing grants made less progress in reading than their peers with the same baseline score who did not receive clothing grants. Girls made more progress in reading but less progress in mathematics than boys.  Differences in school intake characteristics had significant effects on pupils’ baseline attainment and progress. Pupils attending schools that served areas of deprivation tended to have lower baseline scores, and make less progress in reading in the course of Primary 1 than pupils attending schools in more affluent areas. The impact of the Early Intervention Programme on pupils’ progress in reading and mathematics was evaluated over a three-year period. The analysis showed that the programme had been very successful in raising overall reading levels, but less successful in reducing inequality among pupils from different social backgrounds. Similar conclusions for elsewhere in Scotland were reached by the national evaluation of the Early Intervention Programme   Tackling inequalities The analysis at the start of Primary 1 demonstrates that some pupils start school relatively disadvantaged in terms of their early reading and mathematics skills. We might expect that their experience in Primary 1 should assist them to catch up, but in fact the value-added analysis shows that the gap widens in the course of Primary 1. These findings demonstrate the need for additional compensatory teaching for pupils who start primary school with relatively low attainment. Research elsewhere has demonstrated that interventions such as ‘reading recovery’ that give sustained one-to-one teaching to low-attaining pupils in early primary stages provide long-term benefits to the progress of disadvantaged pupils. The researchers concluded that “targeting intervention directly at poor children offers a viable strategy for tackling illiteracy among children from the most deprived social groups” (Hurry & Sylva, 1997).  Policy issues The measurement of performance tends to be viewed with suspicion by the education community because of fears about how the results will be used. The continued publication of school league tables, with media focus on ‘naming and shaming’, makes it difficult for practitioners to consider measurement of performance as a useful tool for self-evaluation.   Nevertheless, without measurement of inequalities in performance it is difficult to focus on aspects of schooling that have detrimental effects on some groups of pupils, and difficult to develop policies to address these problems. For example, evidence from England shows that some ethnic minority pupils fare less well in schools than their white peers. Is this the case in Scotland? At present we do not know because in the Scottish system there is no measurement of inequalities in performance. Consequently, we do not know whether there is a need for policy intervention.  The absence of evidence about inequalities in performance also means that it is difficult to evaluate the impact of policies. For example, there has been concern expressed in recent years about perceived low levels of progress in the S1-S2 stages. Many schools have been encouraged to introduce setting by ability in S1-S2 to increase rates of progress among high ability pupils. Unfortunately, there has been little or no measurement of progress in S1-S2 either before or after these developments, and no measurement of the impact of changes in class organisation on inequality. However, research elsewhere shows that the use of ability grouping, such as setting or streaming, is likely to increase social inequality (CES Briefing No 25). Therefore, in spite of the National Priority of Inclusion and Equality  the increased use of setting in Scottish secondary schools is likely to increase inequality.  If Scotland is serious about creating an education system “in which everyone matters”, and serious about giving priority to inclusion and equality, the measurement and analysis of inequalities in performance are essential. We are currently a long way from being able to monitor inequalities in schooling, and we need to start putting essential measures in place.  Further reading  CROXFORD, L. (2001a) The Impact of the Early Intervention Programme in Primary 1, Education in the North, 9, pp. 26-32. CROXFORD, L. (2001b) Socio-economic Inequalities and Educational Attainment in Scotland, in: Social Justice...a Scotland where everyone matters: Annual Report 2001 (Edinburgh, Scottish Executive). FITZ-GIBBON, C. T. (1996) Monitoring Education: Indicators, Quality and Effectiveness (London, Cassell). HURRY, J. & SYLVA, K. (1997) Evaluation of Interventions for children with reading difficulties: summary of five-year study. (London, London University, Institute of Education.). TINKLIN, T., CROXFORD, L., DUCKLIN, A. & FRAME, B. (2001) Gender and Pupil Performance in Scotland’s Schools  (Report to the Scottish Executive Education Department) (Edinburgh, University of Edinburgh, CES).  Further information For further details contact Linda Croxford, CES. Email L.Croxford@ed.ac.uk. The views expressed are those of the author.   About the study This Briefing is part of the Knowledge Transfer initiative of the University of Edinburgh. The research on which it is based has been carried out with Aberdeen City Council, and the City of Edinburgh Council as part of the evaluation of the Early Intervention Programme.  Related CES Briefings No. 14 League Tables – Who Needs Them? by L.Croxford No. 16 Inequality in the First Year of Primary School by L.Croxford No. 19 Inequality in Attainment at Age 16: A ‘Home International’ Comparison by L.Croxford No. 25 Standards, Inequality and Ability Grouping by A.Gamoran All Briefings can be downloaded from our website, free of charge. If hard copy or multiple copies are required please contact Carolyn Newton at the address below. The CES Briefings series is edited by Cathy Howieson. All comments on the Briefings are welcome (c.howieson@ed.ac.uk).     

Measuring Performance - Tackling Inequalities?

https://www.pure.ed.ac.uk/ws/files/25056551/Brief026.pdf

Measuring Performance - Tackling Inequalities?

Abstract

Similar works

Full text

Available Versions

Edinburgh Research Explorer