Adapting Paper-Based Tests for Computer Administration: Lessons Learned from 30 Years of Mode Effects Studies in Education

In today’s digital age, tests are increasingly being delivered on computers. Many of these computer-based tests (CBTs) have been adapted from paper-based tests (PBTs). However, this change in mode of test administration has the potential to introduce construct-irrelevant variance, affecting the validity of score interpretations. Because of this, when scores from a CBT are to be interpreted in the same way as a PBT, evidence is needed to support the reliability and validity these scores (AERA et al. 2014). Numerous studies have investigated the impact of changing the mode of test delivery from paper to computer, not only in terms of their psychometric properties, but also with regard to possible sources of construct-irrelevant variance. This article summarizes the main lessons learned from mode effects studies in education over the past 30 years and discusses some of the questions remaining

Evaluation of the computers for pupils initiative: final report

The Computers for Pupils (CfP) initiative aimed to help overcome the digital divide, which can prevent young people from disadvantaged backgrounds from enjoying the benefits of access to information and communications technology. The initiative provided funding for schools in deprived areas to invest in home access to ICT for their neediest pupils in order to: • give eligible learners the same opportunities as their peers • contribute to raising educational achievement • support personalised learning • encourage the development of ICT skills among learners and families. In December 2006, Becta commissioned the National Foundation for Educational Research (NFER) to undertake a national evaluation of the CfP initiative. The main aim of the two-year evaluation was to assess the impact of CfP on learners and their families and to explore how schools and teachers had developed their pedagogic practices in order to support and capitalise on the new educational opportunities afforded by the technology. The evaluation involved distinct though interrelated strands of quantitative and qualitative research: • Questionnaire surveys of teachers in CfP schools, learners selected for CfP, and learners’ parents, which were conducted twice (autumn 2007 and 2008) in order to explore changes in general access to and use of computers and ICT, and assess the impact of the CfP initiative • In-depth case-study research across 13 schools within eight local authorities (LAs) in the spring and summer terms 2008 and again in the autumn term 2008 (including the same LAs and schools at all time points), in order to explore through detailed interviews perceptions of the implementation and impact of CfP. Key findings in relation to the aims of CfP