Economists have typically been sceptical that computers improve educational outcomes. But research by Stephen Machin, Sandra McNally and Olmo Silva finds evidence that new technology can have a positive effect on pupils' performance.

Machin, Stephen

McNally, Sandra

Silva, Olmo

English

Research Papers in Economics

CentrePiece Summer 2006
10
T
he view that information and
communication technology
(ICT) is a useful tool for raising
educational standards dates
back to the 1950s and the findings of
Harvard psychologist BF Skinner. More
recently, support for the effectiveness of ICT
as a teaching and learning device has come
from the educational and psychological
literature (recently reviewed by Heather
Kirkpatrick and Larry Cuban), which tends
to make enthusiastic claims for the value of
new technology in schools.
Economists have generally been much
more cautious, in particular raising concerns
about the methodological validity of much
of this research. Most of the positive
findings are inferred from simple
correlations between ICT and test scores.
These cannot take account of unobserved
school characteristics – such as more
motivated teachers – that may lead to both
ICT adoption and better attainments.
Indeed, starting with work by Joshua
Angrist and Victor Lavy, a small number of
economic studies, applying more rigorous
methods of analysis, have found it hard to
uncover any evidence of a positive causal
relationship between computers (and/or
computer software) and pupil performance.
This has not stopped the UK
government seeing ICT investment in
schools as ‘crucial to our drive to raise
standards’ (former secretary of state for
education Ruth Kelly) and envisaging ICT
being widely used across the whole
curriculum in all state schools (see
Department for Education and Skills (DfES),
2003, and Office for Standards in
Education (Ofsted), 2001).
The positive rhetoric has been backed
up by considerable public investment.
Between 1998 and 2002, ICT expenditure
almost doubled in English secondary
schools – from an average of £40,100 to
just under £75,300 per school, or 3% of
overall expenditure – and more than trebled
in primary schools – from £3,600 in 1998
New technology in schools:
is there a payoff?
Economists have typically been sceptical that computers improve
educational outcomes. But research by Stephen Machin, Sandra
McNally and Olmo Silva finds evidence that new technology can
have a positive effect on pupils’ performance.
Table 1:
How was ICT money spent in schools?
Percentage devoted to different items
1999/00 2001/02 Percentage 
Primary schools change
Hardware 63 53 -16
Software 10 10 0
Internet 8 7 -12
Training 7 11 +57
Technical support 9 13 +44
Administration + other 3 6 +100
Total expenditure per school £10,000 £14,100 +41
Secondary schools
Hardware 57 55 -3.5
Software 9 9 0
Internet 4 3 -25
Training 4 6 +50
Technical support 14 17 +21
Administration + other 12 10 -17
Total expenditure per school £56,500 £76,000 +34.5
Source: Authors’ calculations from ICT Survey of Schools in England (DfES)
Large increases in ICT funding 
have improved educational
performance in primary schoolsCentrePiece Summer 2006
11
to £12,900 in 2002, or 2% of overall
expenditure. Most of this dramatic
acceleration took place from 2000 and the
upward trend continued in 2003 and 2004.
Table 1 shows how primary and
secondary schools used these resources in
2000 and 2002. Between these years, the
share of ICT expenditure devoted to
hardware and software actually decreased –
by 16% in primary schools and 3.5% in
secondary schools – but as the total
amount of ICT funding was increasing,
overall expenditure in these two categories
grew steadily.
Interestingly, the share of resources
devoted to teacher training rose by 57%
for primary schools and 50% for secondary
schools between 2000 and 2002. This
suggests that the extra funding may have
improved the quality of ICT use in schools,
not simply increasing the quantity of 
ICT equipment. 
Table 2 provides a more complete
picture of schools’ ICT use, showing how
new technology and training grew in
response to the extra resources. Although
schools were already well resourced, there
were quite sizeable changes between 2000
and 2003. For example, the pupils to
computer ratio in 2000 was 10 to 1 in
primary schools and 7 to 1 in secondary
schools; by 2003, the respective ratios were
6 to 1 and 4 to 1.
The indicators of ICT use in the
classroom also show fairly high percentage
increases over this short time period. What’s
more, the fraction of teachers trained to
use ICT grew substantially. This confirms
our intuition that rather than just increasing
the quantity of ICT equipment, schools also
invested in the quality of ICT use.
Although secondary schools were
better equipped with ICT in 2000, the
greatest relative increase over time was
experienced in primary schools. It is also
notable that ICT is used regularly for
teaching purposes in a much higher
percentage of primary schools than
secondary schools. Any effect of ICT on
educational performance is therefore likely
to be more evident in primary schools than
secondary schools.
We also have evidence that ICT is
widely used in primary schools to teach
English: 65% of primary schools report that
they ‘substantially use’ ICT for teaching this
subject. The next most important ‘ICT user’
is Mathematics, where ICT is ‘substantially
used’ in about 56% of primary schools,
followed by Science (35%).
So has the big increase in ICT
investment made a difference to
educational standards? Our research
evaluates whether changes in ICT
investment had any causal impact on
changes in educational outcomes in English
schools between 1999 and 2003. To do
this, we rely mainly on administrative data
at the level of local education authorities
(LEAs), focusing on average achievements
at the end of primary education in English,
Mathematics and Science. Following DfES
targets, we look at the proportion of pupils
achieving level 4 or above in the three
subjects at age 11, the end of Key Stage 2.
Inferring a causal relationship between
ICT investments and pupil achievements
from simple correlations can be misleading.
For example, we could imagine that schools
or LEAs with more motivated teachers and
head teachers are both more likely to adopt
ICT and to produce better attainments:
then, if we did not control for motivations,
and just related ICT use to pupil tests, a
positive relationship may emerge just
because of this unobserved common factor
(motivation) driving both observed
outcomes.
To overcome this problem and identify
the causal impact of ICT use on pupil
achievement, we exploit a 2001 policy
change that modified the rules for ICT
investment in different regions of England.
Table 2:
Trends in ICT expenditure and use of ICT resources 
Primary schools Secondary schools
1999/00 2002/03 Percentage 1999/00 2002/03 Percentage 
change change
Computers per pupil 0.10 0.16 +60 0.15 0.23 +53
Percentage of teachers using ICT regularly 75 92 +23 38 55 +45
Percentage of teachers trained to use ICT  81 95 +17 75 83 +11
Percentage of teachers with recently updated training 57 85 +49 48 69 +44
Percentage of schools connected to the internet 86 100 +16 99 100 +1
Source: Authors’ calculations from ICT Survey of Schools in England (DfES)
The positive impact of
ICT investment is most
evident in the teaching
of English CentrePiece Summer 2006
Before 2001, funding was allocated from
central government to LEAs through a
bidding process, aiming to direct money
towards LEAs with innovative and
interesting proposals for the use of 
ICT funds.
From 2001 onwards, allocations were
instead made according to a formula based
on school and pupil numbers in LEAs with
an adjustment for population density. The
change in the allocation mechanism
created ‘winners’ and ‘losers’ among LEAs:
areas that had benefited a lot under the old
system stood to lose from the transition to
a formula-based system, and vice versa. 
In our analysis, we argue (and provide
evidence) that money was reshuffled across
LEAs in a ‘random’ way, that is, in a way
unrelated to unobservable LEA
characteristics that may give rise to a
spurious relationship between ICT funding
and test scores. We then use the changes
in the ICT funding accruing to LEAs to
estimate the effects of ICT expenditure on
educational standards. Our approach
identifies the effect of being a winner or a
loser in the new system of ICT allocation. 
We estimate the effect of changes in
ICT funding per pupil on changes in
achievements in English, Mathematics and
Science at the end of primary education.
We find a positive relationship between ICT
funding per pupil and performance in
English: a doubling of ICT funding per pupil
in schools leads to a 2 percentage point
increase in the proportion of pupils
achieving level 4 or above at age 11.
Changes in ICT funding of this
magnitude really did happen for primary
schools over this period, and the impact on
performance in English is notable given that
the average growth rate of pupils' scores in
this subject was around 7% between 1999
and 2003. But it is important to note that
this causal effect of ICT is not an average
effect for all schools in England. Rather, it is
the causal effect of large changes in ICT
investment for LEAs that were substantially
affected by the rule change – the winners.
For Mathematics, the impact of ICT on
test scores is very close to zero. But there is
a positive relationship between ICT and
achievements in Science: in this case, 
a doubling of ICT funding per pupil leads 
to an increase of 1.6 percentage points in
the proportion of pupils achieving level 4 
or above.
So, unlike previous economic studies,
we find evidence for a positive causal
impact of ICT investment on educational
performance in primary schools. This is
most evident in the teaching of English,
where we also find high use of ICT for
teaching purposes. We also observe a
positive impact for Science, though not for
Mathematics. How can we reconcile our
evidence with previous research that finds
no effect? 
Our estimates identify the impact of
being a winner or a loser under the new
system. After the policy change, the
average growth rate of ICT funds among
LEAs mostly benefiting from the reform
was roughly 60%. This contrasts with a
much smaller change of 20% for LEAs that
lost more from the introduction of the
formula-based system. Intuitively, it is the
comparison between these two groups that
drives our identification of the impact of
ICT on educational outcomes: our strategy
mainly captures the impact of large
changes in ICT investment on primary
school performance. 
LEAs benefiting most from the policy
change were LEAs with lower overall
expenditure per pupil but better
educational standards (as measured by
exam pass rates and truancy rates). This
suggests that resources were redirected to
areas that were in a better position to use
them efficiently. Furthermore, new
technology was already in place in English
schools since the mid-1990s, and money
redirected after the policy change was
mainly spent in updating resources and
teachers’ skills.
So it appears to be the joint effect of
large increases in ICT funding – and a fertile
background for making efficient use of it –
that led to the positive effects of ICT
expenditure on educational performance.
This article summarises ‘New Technology in
Schools: Is There a Payoff?’ by Stephen
Machin, Sandra McNally and Olmo Silva,
Discussion Paper No. 55 from the Centre for
the Economics of Education at CEP
(http://cee.lse.ac.uk/cee%20dps/ceedp55.pdf).
The authors are all CEE researchers and
active members of CEP’s wider research
programme on education and skills.
Further reading
Joshua Angrist and Victor Lavy (2002),‘New
Evidence on Classroom Computers and Pupil
Learning’, Economic Journal 112, 735-65.
Department for Education and Skills (2003),
Fulfilling the Potential: Transforming Teaching
and Learning through ICT in Schools.
Heather Kirkpatrick and Larry Cuban (1998),
‘Computers Make Kids Smarter – Right?’,
TECHNOS Quarterly for Education and
Technology 7(2), 1-11.
Office for Standards in Education (2001), ICT
in Schools: The Impact of Government
Initiatives: An Interim Report.
BF Skinner (1954),‘The Science of Learning
and the Art of Teaching’, Harvard Educational
Review 24, 86-97.
BF Skinner (1958),‘Teaching Machines’,
Science 128, 969-77.
Schools have invested not just in
more ICT equipment but also,
through teacher training, in better
quality ICT use

New technology in schools: is there a payoff?.

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics