Sibieta, Luke

Digital Education Resource Archive

1         Comparing GCSE performance in England and Wales – equivalent or not?    Luke Sibieta, Research Fellow, Education Policy Institute January 2019 ______________________________________________________________________________________ International comparisons are popular in education policy debate as they illustrate the differences in the skills young people have acquired. However, it can be difficult to draw reliable conclusions about the effectiveness of different policy approaches from international comparisons as there are also vast differences in education systems, cultures, social structures and economies.  Whilst international comparisons feature heavily in education debates, there is less focus on cross-national comparisons within the UK. Different exams in Scotland and a mostly segregated school system in Northern Ireland make such comparisons difficult. Apart from the bilingual system in Wales, England and Wales shared a common set of school structures and exams up to the point of devolution, and have gradually been diverging ever since.  What can we learn from existing data comparing England and Wales, and how could we learn more?    England-Wales differences are increasingly shaped by GCSE-equivalent vocational qualifications  The proportion of children in England and Wales achieving 5+ GCSEs or equivalent at A*-C was very similar between 1995 and 2002. From the mid-2000s, performance on this measure increased substantially in both countries, though the rate of increase was much faster in England. By 2011-12, 82 per cent of children in England achieved this threshold compared with 73 per cent in Wales. This compares with around 50 per cent in both countries just a decade earlier.  2  Figure 1 – Proportion of pupils achieving 5+ GCSEs or equivalent at A*-C in Wales and England over time   Source: Stats Wales, Key Stage 4 examination results (https://statswales.gov.wales/Catalogue/Education-and-Skills/Schools-and-Teachers/Examinations-and-Assessments/Key-Stage-4); GCSE and Equivalent Results, 2014-15 and 2015-16 (https://www.gov.uk/government/collections/statistics-gcses-key-stage-4) Most of these substantial increases in performance occur after equivalent qualifications were included in headline measures of performance in the mid-2000s in England and Wales. The 2011 Wolf Review of Vocational Education concluded that a rapid uptake of vocational qualifications in England could account for a large part of the rise in headline performance over time. It also concluded that these vocational qualifications were likely to be of less long-term benefit to pupils than standard GCSEs. Concerns have also been raised about the effects of early and repeated entries in both England and Wales.    In response, GCSE performance measures were substantially reformed in England from 2013-14 to restrict the contribution vocational qualifications could make (e.g. only from an approved list and not counting for more than one GCSE) and to only count pupils’ first attempt at GCSEs. The large downward revision to results in 2013-14 in England was a direct result of these changes. New GCSE specifications and a 9-1 numerical grading structure also began to be phased in from 2016-17.  Following a similar review of qualifications, Wales introduced limits on the contribution vocational qualifications could make in two steps in 2015-16 and 2016-17 (e.g. counting for no more than two GCSEs). From 2016-17, Welsh results also started to reflect new GCSE specifications (different to those introduced in England), though the grading structure was unchanged.  As can be seen, the overall effect on results appears to be largest for 2016-17. Welsh statistics will still, however, reflect early and repeat entries until 2019. Producing comparisons across England and Wales that account for the effect of GCSE equivalent qualifications is not easy. Existing work already shows that the difference in results across the two countries is near zero if you just look at GCSEs. However, this only looks at pupils who took GCSEs. We don’t know how pupils who took vocational qualifications would have performed in GCSEs. 0102030405060708090Percent achieving 5+ GCSEs or equiv. A*-CWales2013/14: GCSE & equiavlent  rule changes in England2015-16 and 2016-17: GCSE and equiavlent rule changes in WalesEngland 2016-17:New GCSEs in England ad Wales3  Lower performance in maths and English in Wales  What we do see is that pupils in Wales show lower levels of performance in maths and English GCSEs alone, which almost all pupils would have taken across both countries. Around 10 per cent fewer children achieved a grade C or higher in English and maths in Wales than in England in 2012-13 (we focus on English only here as comparable data for English or Welsh is only available for a short amount of time and shows similar results).    These differences reduce following rule changes in England in 2013-14 that only count a pupil’s first attempt. However, performance remained lower in Wales even though reforms to early and repeat entries aren’t going to made until 2019. New GCSE specifications for both countries in 2016-17 further reduce comparability across country and over time.  Figure 2 – Proportion of pupils achieving C or higher in maths and English in Wales and England over time   Source: Stats Wales, Key Stage 4 examination results (https://statswales.gov.wales/Catalogue/Education-and-Skills/Schools-and-Teachers/Examinations-and-Assessments/Key-Stage-4); GCSE and Equivalent Results, 2014-15 and 2015-16 (https://www.gov.uk/government/collections/statistics-gcses-key-stage-4) Despite issues of declining comparability, lower levels of performance in maths and English in Wales largely match differences in international PISA rankings.  Reading and maths scores in England have remained around the OECD average in recent years. In contrast, performance in Wales has been consistently below the OECD average (despite some catch up in 2015). Numeracy results in Wales in 2015 were about 0.15 standard deviations in average skill levels behind England and over 0.2 standard deviations behind in reading, both sizeable differences.    League tables, tests and demographics There are a range of possible explanations for lower levels of performance in Wales. Previous work has argued that the removal of league tables in Wales in the early 2000s reduced GCSE performance as compared with England by just less than 0.1 standard deviations. Some of this effect seems likely to 01020304050607080Percent achieving grade C or aboveWales - English England - EnglishWales - Maths England - Maths4  reflect a greater incentive to make use of GCSE equivalents in England to boost league table position. Whatever the true effect, Welsh policymakers have made some steps towards making more school-to-school comparisons by introducing a system of school banding in 2011.  Another potential explanation is the removal of external tests at age 11, which may have reduced accountability and/or given pupils in Wales less experience of high stakes tests. By definition, it is quite hard to test this theory given the lack of data. However, the re-introduction of numeracy and reading tests in 2013 for all pupils aged 7-14 in Wales does show that the Welsh government seems to see a greater role for testing again.  Another potentially important explanation is the role of differences in demographics across the two countries. This is currently hard to test using publicly available data. In the figure below, we show the GCSE performance of pupils eligible and not eligible for free school meals across Wales and England over time (the measure used here reflects the per cent of children achieving 5+ GCSEs or equivalent at A*-C, including English/Welsh and maths). This shows a lower level of performance for both disadvantaged and non-disadvantaged pupils in Wales than in England, with the biggest gap for disadvantaged pupils. This lower level of performance remains for disadvantaged pupils in Wales even after the GCSE rule changes in England in 2014.  Performance for disadvantaged and non-disadvantaged pupils in Wales continues to grow after this point, but the different rules for GCSE performance measures makes it hard to interpret these changes relative to England.  Figure 3 – Proportion of pupils eligible and not eligible for free school meals achieving 5+ A*-C in GCSEs or equivalent (inc. English/Welsh and maths) over time   Source: Stats Wales, Key Stage 4 examination results (https://statswales.gov.wales/Catalogue/Education-and-Skills/Schools-and-Teachers/Examinations-and-Assessments/Key-Stage-4); GCSE and Equivalent Results, 2010, 2014 and 2016 (https://www.gov.uk/government/collections/statistics-gcses-key-stage-4) This simple comparison suggests that differences in deprivation are unlikely to explain lower levels of performance in Wales on their own. There are, however, several other sources which could account for 01020304050607080Percent achieving 5+ GCSEs or equiv at A*-C including English and MathsWales FSM England FSM Wales Non-FSM England Non-FSM5  performance differences. There is evidence of a greater level of persistent child poverty in Wales than in England. A much greater share of pupils in England are from minority ethnic groups (about 30 per cent) as compared with Wales (about 12 per cent), and minority ethnic groups tend to perform better in GCSEs. We hope to do further work using administrative for Wales and England that allow us to account for this multitude of differences.  Smaller differences at earlier ages Another limitation of existing comparisons is that they focus on differences at the end of compulsory schooling at age 15 and 16. This misses out seeing how the differences emerge over the course of primary and secondary schooling. Undertaking comparisons before age 15 is problematic, however, due to a lack of comparable test data over time.  The Millennium Cohort Study (MCS) provides one potential source of data for comparison by tracking the lives of a sample of children across the UK born around the turn of the millennium.  The figure below shows children’s average vocabulary scores in England, Wales, Scotland and Northern Ireland at ages 3, 5, 7, 11 and 14 (all figures are standardised and shown relative to the UK average at each age). England remains close to the UK average at each age, which is unsurprising given that it makes up more than 80 per cent of the UK population. Wales starts off close to the UK average at ages 3 and 5, but is noticeably behind at ages 7 and 14 (by about 0.1 standard deviations).  This evidence corroborates GCSE and PISA results, though the difference is slightly less than the 0.2 standard deviations seen in the PISA reading measure in 2015. Furthermore, Wales is actually slightly above the UK average at age 11.  Figure 4 – Average British Ability Scale Vocabulary Scores in England, Wales, Scotland and Northern Ireland across ages (MCS)  Source: Author’s calculations using MCS User Guide to Initial Findings Ages 3, 5, 7 and 11 (https://cls.ucl.ac.uk/cls-studies/millennium-cohort-study/) and Sullivan, Moulton and Fitzsimons (2017), https://cls.ucl.ac.uk/wp-content/uploads/2017/11/CLS-WP-201714-The-intergenerational-transmission-of-vocabulary.pdf -0.2-0.15-0.1-0.0500.050.10.150.2Age 3 Age 5 Age 7 Age 11 Age 14Average vocabulary scores relative to UK (national standard deviations)England Wales Scotland Northern Ireland6  We hope to explore these differences in more detail to examine the role played by richer measures of family background and behaviours available in the MCS, as well as the relative role played by primary and secondary schools in explaining differences in educational outcome across England and Wales.  New GCSEs will reduce comparability over time Wales introduced new GCSE English, maths and Welsh specifications in 2016-17, and fewer children achieved a grade C in these new specifications than did in previous years (see Figure 2 above). England also introduced new GCSEs in English and maths in the same year and moved to a numerical grading structure. As these new specifications and grading structures are rolled out across more subjects over time, it will become increasingly hard to compare GCSE results across the two countries. Qualifications regulators across the UK have sought to enable broad comparisons across the different qualifications and have committed to retaining their value over time. However, in practice, there are still likely to be significant challenges for universities and employers seeking to make comparisons in applicants’ GCSEs taken in England and Wales, and across different years. This is an inevitable result of the two countries charting different courses in GCSE reform.   Summary There seems to be clear evidence from a range of sources to suggest that teenagers in Wales display worse educational outcomes than children in England. This is seen across a range of sources – GCSE results, PISA tests, cognitive tests in surveys – and is common to disadvantaged and less disadvantaged pupils alike. However, children in Wales also seem to display similar levels of literacy at ages 3, 5 and 11 to their peers in England. We plan to explore these differences in more detail in future work to uncover the relative importance of family background and schools in explaining these cross-country differences, as well as the specific role played by changing qualification and assessment regimes. Wales and England are now embarking on quite different reforms to GCSEs. It is therefore increasingly important to understand the differences that have led up to this point.   

Comparing GCSE performance in England and Wales: equivalent or not?

https://dera.ioe.ac.uk//32758/1/Pupil-Performance-England-vs.-Wales_EPI.pdf

Comparing GCSE performance in England and Wales: equivalent or not?

Abstract

Similar works

Full text

Available Versions

Digital Education Resource Archive