Estimating the reliability of composite scores

In situations where multiple tests are administered (such as the GCSE subjects), scores from individual tests are frequently combined to produce a composite score. As part of the Ofqual reliability programme, this study, through a review of literature, attempts to: look at the different approaches that are employed to form composite scores from component or unit scores; investigate the implications of the use of the different approaches for the psychometric properties, particularly the reliability, of the composite scores; and identify procedures that are commonly used to estimate the reliability measure of composite scores

Public perceptions of unreliability in examination results in England: a new perspective

Presented at the 36th International Association for Educational Assessment (IAEA) Annual Conference in Bangkok, Thailand, 22-27 August 2010. This report is about a quantitative study on public perceptions of unreliability in examination results, using an online questionnaire survey

The development of a computer assisted design, analysis and testing system for analysing students’ performance

Recent years have seen increased application of Computer Assisted Assessment (CAA) in education at various levels, and a variety of computer software systems have been developed for use in computer-based testing and analysis. However, many existing system are primarily designed to provide objective assessment of students and analysis of test items. The present study presents the development of a Computer Assisted Design, Analysis and Testing System (CADATS) that can be used by primary and secondary schools and other test organisations to undertake computer assisted assessment. The system incorporates an Item Response Theory (IRT) model – the Rasch model to facilitate the administration of IRT – based tests on computers and the analysis of test items and students’ performance using modern test theories. Specifically, the system has been created to design and undertake computer-based tests (CBTs), including the Computer Adaptive Tests (CATs), and to undertake diagnostic analysis on students’ performance at both individual and school levels in order to identify curriculum areas where students are under performing

The development of a fraction testing and tutoring system

One of the major areas in Computer Aided Learning (CAL) research has been the development of Intelligent Tutoring Systems (ITSs) for tutoring individual subjects through the application of cognitive tutor technology. In parallel to the use of computer aided learning (CAL), Computer Based Testing (CBT) has also been widely used for a variety of purposes. Studies have shown that fractions are one of the most difficult subjects to learn due to the complexity of the concepts and skills involved. Although there are various fraction tutoring systems available, most of them lack user interactivity and do not provide intuitive insight into the many concepts and skills associated with fractions. Furthermore, these systems are generally intended for longer-term learning use (e.g. weeks or even longer) and the effectiveness of using such systems is therefore usually difficult to assess. This paper presents the development of an interactive computerised Fraction Testing and Tutoring System (FTTS) delivered through the World Wide Web that incorporates learning capabilities for the purpose of both tutoring and testing fractions for school students aged 10-12. The system implements the cognitive learning model proposed by Derry in 1990. This system can perform fraction tutoring and at the same time undertake fraction testing. Results obtained from the use of this system can therefore be used to assess the effect of tutoring on students’ performance