Is the government's National Literacy Strategy effective? Stephen Machin and Sandra McNally look at the evidence from the pilot project.

Machin, Stephen

McNally, Sandra

English

Research Papers in Economics

CentrePiece Spring 2004 3
by Stephen Machin and Sandra McNally
I
n 1999 the Moser report identified one in five adults in the
UK as being functionally illiterate. How do we ensure 
that the next generation of adults does not suffer the
same fate? 
The National Literacy Strategy, introduced in September
1998, is a major initiative aimed at tackling these problems
at primary school. It involves a daily “literacy hour”, with a
practical structure for time and class management and
teaching objectives for each term. But does it work? The
government has been criticised for failing to meet its own
targets for tests at the end of Key Stage 2 (i.e. the 7 to 11
phase of education). Although economists have had much
to say about the effect of increasing resources on pupil
attainment, they have generally not considered the effect 
of changing the content and structure of how a subject 
is taught. 
We present evidence that the literacy hour works. It comes
from the National Literacy Project (NLP), which was an
immediate forerunner of the National Literacy Strategy. This
introduced the literacy hour into a sub-set of schools within
a number of Local Education Authorities. We show that the
policy not only led to a substantial improvement in attain-
ment, but did so at a low cost. It also had a marked impact
on the well-known “gender gap” (favouring girls) as it had a
larger differential impact on boys.
The NLP was aimed in particular at improving the low levels
of reading and writing skills in many badly performing inner
city schools. An OFSTED report at the time was critical of
the teaching practices in such schools, which included
problems like free reading with little or no intervention by the
teacher and too much time hearing individual pupils read. In
the same way as the National Literacy Strategy, the NLP
changed the content and structure of how literacy was
taught. This new approach was based on educational
research and on international experience of similar
schemes, especially in the US. Since the NLP was intro-
duced in only a sub-set of schools two years prior to the
National Literacy Strategy (which affected all schools), we
have an opportunity to evaluate the effects of the literacy
hour by comparing the educational attainment of children in
NLP schools with that in similar schools where the NLP was
not introduced. 
The National Literacy Project was introduced in some 400
junior schools during the school years 1996/97 and
Large benefits,
low cost
Is the government’s National Literacy Strategy effective? 
Stephen Machin and Sandra McNally look at the evidence 
from the pilot project. CentrePiece Spring 2004 5
schools, which only had one year of exposure to the literacy
hour. Again, the changes are larger in the NLP schools, with
the figures going up by 3.4 percentage points as compared
with 1.1 percentage points in the control schools.
Our analysis, of course, needs to allow for differences in
the characteristics of schools and the data allow us to
control for a large number of factors. These include infor-
mation on outcomes (e.g. results, absences), inputs (e.g.
pupil-teacher ratios), social disadvantage (e.g. percentage
of students eligible for free school meals or with special
educational needs) and type of school (e.g. single sex,
grammar). A full account of the methodology will be found
in our forthcoming CEE Discussion Paper. 
Taken as a whole, the results of our regression analysis
strongly corroborate the view that the literacy hour under
the NLP significantly raised pupil performance in the
primary schools that were exposed it. Furthermore, for the
first cohort of children exposed to the literacy hour, there is
a positive and statistically significant effect of the policy on
GCSE results in English at age 16. 
We were also interested to see whether the literacy hour
had a differential impact on boys and girls. Boys have tradi-
tionally performed worse than girls in literacy-related activi-
ties. For example, in 1996 (the year before the NLP was
introduced) only 49% of boys achieved Level 4 or above in
KS2 English, compared with 64% of girls (see Table 2). If
it is correct that boys have a greater problem than girls with
concentration and focus, it might be expected that the NLP
would benefit them more.
We do indeed find such a gender difference in the NLP’s
impact at primary school. For reading, the literacy hour
raised boys’ mean percentile scores by somewhere
between 2.5 and 3.4 percentile points. The probability of
achieving Level 4 or above in KS2 English was up by
between 2.7 and 4.2% for boys. Effects for girls were
considerably smaller and not always statistically significant.
Hence the NLP had a large impact on the oft-cited gender
gap in literacy. 
It is interesting to put this finding for the 1997 cohort
against the national figures for attainment in English given in
Table 2. It is evident that the gender gap in primary school
reading and English has reduced in recent years. Our
findings are entirely consistent with the literacy hour having
continued to play an important role since the National
Literacy Strategy was introduced.
The question remains as to whether the policy was cost
The data
The empirical analysis is based on
administrative records of pupil-level
attainment and on school-level data.
For pupils, the data consists of
detailed information on educational
attainment from when they were of
age 11 and age 16. At age 11, all
pupils in England are tested at the
end of “Key Stage 2”. At age 16,
exams at the end of “Key Stage 4”
(i.e. GCSE or GNVQ) mark the end
of a pupil’s compulsory education.
The first available year of national Key
Stage 2 data for pupils is 1996, the
school year before the National
Literacy Project was introduced (we
refer to school years according to
when pupils took the exam – so
“1996” refers to the 1995/96 school
year). The NLP was introduced for
two cohorts, in 1997 and 1998.
Pupils within the first cohort finished
their compulsory education in 2002.
Hence, to evaluate the impact of the
NLP on attainment at secondary
school, we matched pupil records
from 1996 (i.e. pre-NLP) and 1997
with GCSE/GNVQ attainment data in
2001 and 2002 respectively. (At the
time of writing, we do not have data
on 2003 GCSE results and, there-
fore, cannot perform the secondary
school analysis for the second cohort
of children affected by the NLP.) 
The pupil-level files have detailed
information on attainment, gender and
codes for the schools attended,
which allows us to match national
school-level data from the School
Performance Tables and files from the
LEA and School Information Service
(LEASIS). We concentrate on two
outcome measures at the end of
primary school: the percentile reading
score and the percentage of students
attaining Level 4 or above in Key
Stage 2 English. The second measure
is a key policy indicator and is the
standard deemed to be appropriate 
at age 11. 
The NLP was introduced in some 400
schools, of which 80% were in inner
cities – several LEAs in London and
also in Sandwell, Liverpool,
Manchester, Sheffield, Newcastle and
Bristol. NLP schools represented in
total about 40% of all primary schools
within these LEAs. The remaining
NLP schools were run by three
county councils (Hampshire, Essex
and Norfolk), where they represented
only about 7 % of all primary schools. 
In order to establish a control group
against which to measure the
performance of NLP schools, we
identified geographically adjacent
LEAs not involved in the NLP. (If there
was more than one, we chose that
with the closest pre-policy perform-
ance profile.) Where we could find no
close control comparison for an NLP
authority, it was dropped from our
sample. This affected the county
councils and Bristol, where the city is
completely surrounded by semi-rural
areas. However, our sample
comprises 72% of all NLP schools 
in England.
As a robustness check, we have also
estimated regressions for a control
group consisting of all other
maintained schools in England.
CentrePiece Spring 2004 4
1997/98. (It was also launched in 112 infant schools, but
this part is not relevant here.) About 80% of NLP schools
were located in inner cities, where the most disadvantaged
in England are concentrated. Most schools entered the
project because they had weaknesses in reading. The
planned cost of the NLP was £12.5 million over five years.
The descriptive part of our overall findings is summarised in
Table 1. The upper part of the Table shows two primary
school attainment measures for three consecutive years: a)
the mean percentile reading score; and b) the percentage
reaching Level 4 or above in Key Stage English. Although
average performance levels are lower in NLP schools at
each point in time (which is taken account of in the regres-
sion approach) an interesting pattern emerges. In “before
and after” terms, there is clear evidence of improvement in
the NLP schools, compared with the control group. 
For reading, the mean score goes up 2.1 percentile points
in the NLP schools and falls by 1.1 in the control group. The
same relative pattern of improvement is seen for KS2
English, where the percentage of pupils attaining Level 4 or
above rises by more in NLP schools (by 12.2 percentage
points compared with 8.8 in the control group).
The lower part of the Table shows statistics on secondary
school performance in GCSE English five years later.
Currently, this can only be done for the 1997 cohort of NLP
Table 1.  Descriptive statistics
NLP Schools Control LEA schools
A. Primary school attainment
1996 1997 1998 change  1996 1997 1998 change
1996-98 1996-98
Percentile reading score 44.4 45.3 46.5 2.1 53.9 53.2 52.8 -1.1
(.8) (.5)
% reaching Level 4 or 
above in KS2 English 37.3 47.6 49.5 12.2 50.5 57.8 59.3 8.8
(.9) (.6)
No. of pupils 12645 12586 12814 22461 21976 22172
No. of schools 283 284 283 553 551 547
B. Secondary school attainment
2001 2002 change 2001 2002 change 
2001-02 2001-02
Percentage with 
GCSE Grade 
A*-C in English 38.7 42.1 3.4 47.3 48.4 1.1
(1.1) (.6)
No. of pupils 5142 5167 18956 19553
No. of schools 147 145 549 551
Notes: Panel A covers cohorts 1 and 2 of NLP, while Panel B only considers cohort 
1 (due to lack of GCSE data for 2003). Standard errors in parentheses.
The data allow us to control for differences in schools There is clear evidence of improvement in NLP schoolsCentrePiece Spring 2004 7 CentrePiece Spring 2004 6
assuming that the pupil goes on to work from age 20 to 65
and using a discount rate of 3%, we estimate under a
number of realistic assumptions the present discounted
value of the cumulative effect of the literacy hour to be
somewhere between £2,000 and £5,500.
Whichever way one looks at it, the benefits of the literacy
hour seem to be large and the costs small. These findings
are of considerable significance to the wider debate about
what works best and most cost effectively for improving
pupil performance. 
Stephen Machin is Director of Research at the CEP, Director of
the DfES Centre for the Economics of Education and Professor of
Economics at University College London.
Sandra McNally is a member of the Centre for the Economics of
Education at the CEP.
This article is based on research that will be published in a
forthcoming CEE Discussion Paper.
References & further reading
Beard, R. (2000) Research and the National Literacy Strategy,
Oxford Review of Education, 26, (Nos. 3 & 4), 421-36.
Department for Education and Employment (1999) A Fresh Start:
Improving Literacy and Numeracy, DfEE: London.
Fisher, R. and M. Lewis (1999) ￿Anticipation or Trepidation?
Teachers’ Views on the Literacy Hour￿, Reading, 33, 23-28.
Jacob, B. (2002) Where the Boys Aren￿t: Non-cognitive Skills,
Returns to Schooling and the Gender Gap in Higher Education,
National Bureau of Economic Research Working 
Paper 8964.
Literary Task Force (1997) The Implementation of the National
Literacy Strategy, London: Department for Education and
Employment.
Office for Standards in Education (1998), The National Literacy
Project: An HMI Evaluation, London: Ofsted.
Office for Standards in Education (1999) Primary Education 1994-
1998: A Review of Primary Schools in England, London: Ofsted.
Sainsbury, M. (1998), Evaluation of the National Literacy Strategy:
Summary Report, Slough: NFER. 
West, A. and H. Pennell (2003) Underachievement in Schools,
RoutledgeFalmer: London and New York.
effective. We try to answer this by comparing the per pupil
costs of the policy with the economic benefits, as reflected
in predicted labour market earnings.
The main costs of the NLP were 14 local centres (each
about £25,000 a year) and literacy consultants in each
participating Local Education Authority (about £27,000 a
year for each consultant). Schools also received some
funding for teacher training and resources, which was
broadly the same for each school (though some account
was taken of the pupil-teacher ratio). However, since the
National Literacy Strategy was introduced two years after
the NLP, only the first two years of the original £12.5 million
five-year programme are relevant. The total cost per annum
was thus £2.5 million (or about £2.8 million in 2001 prices).
The cost per pupil involved was just over £25 a year. 
It might be argued that the literacy hour takes teaching
effort and resources away from other subjects and that this
indirect cost effect (via substitution) should be taken
account of in a cost-benefit calculation. However, literacy
was being taught in some form before the policy for the
same kind of time. Therefore, the literacy hour represents a
change in how reading and writing are taught, rather than
an increase in the time devoted to the subject. There are
also likely to be positive spillovers between pupil subject
areas and associated teacher practice. 
First, since the ability to read and write are important
generic skills, an improvement in how these skills are taught
might lead to improved performance in other subjects.
Secondly, the literacy hour is likely to have caused teachers
to re-evaluate their teaching methods in other subjects. This
is important in English primary schools because, generally,
pupils within a particular year group are taught every
subject by the same teacher. We do indeed find some
evidence linking the literacy hour to higher levels of achieve-
ment in Mathematics. Thus, if anything, the effects of the
NLP are likely to be underestimated in our approach.
To estimate benefits of the policy we investigated the impact
of reading scores on future labour market earnings, using
the British Cohort Study. This is a panel survey of all those
living in Great Britain born between 5 and 11 April 1970.
We regressed the log of labour market earnings (at age 30,
in 2000) on age 10 percentile reading scores (from 1980).
We then include controls for various factors, like gender,
region, family background and highest educational qualifi-
cation achieved by age 30. Since the educational qualifica-
tion variable is likely to partly capture the effect of the
reading score, the effect of reading on labour market
earnings is likely to be an underestimate when this variable
is included. 
These estimates are inevitably somewhat broad brush. But,
Table 2.  Primary and secondary school English attainment 1966-2002
A. Primary school attainment
% Pupils achieving Level 4 and above in Key Stage 2 English
1996 1997 1998 1999 2000 2001 2002 Change % points
% % % % % % % 1996-2002
All  57 63 65 71 75 75 75 18
Boys 50 57 57 65 70 70 70 20
Girls 65 70 73 76 79 80 79 14
% Pupils achieving Level 4 and above in Key Stage 2 Reading
1996 1997 1998 1999 2000 2001 2002 Change % points
1997-2002
All  n/a 67 71 78 83 82 80 13
Boys n/a 63 64 75 80 78 77 14
Girls n/a 71 79 82 86 85 83 12
B. Secondary school attainment
% Pupils achieving Grade C or above GCSE English
1996 1997 1998 1999 2000 2001 2002 Change % points
1996-2002
All  49 52 51 53 54 59 56 7
Boys 40 43 42 45 46 51 48 8
Girls 58 61 59 61 62 66 64 6
Note: Data from DfES national statistics.
Cost per pupil was just over £25 a year Scores for boys improved more than for girls

Large benefits, low cost.

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics