Linking tests of English for academic purposes to the CEFR: the score user’s perspective

The Common European Framework of Reference for Languages (CEFR) is widely used in setting language proficiency requirements, including for international students seeking access to university courses taught in English. When different language examinations have been related to the CEFR, the process is claimed to help score users, such as university admissions staff, to compare and evaluate these examinations as tools for selecting qualified applicants. This study analyses the linking claims made for four internationally recognised tests of English widely used in university admissions. It uses the Council of Europe’s (2009) suggested stages of specification, standard setting, and empirical validation to frame an evaluation of the extent to which, in this context, the CEFR has fulfilled its potential to “facilitate comparisons between different systems of qualifications.” Findings show that testing agencies make little use of CEFR categories to explain test content; represent the relationships between their tests and the framework in different terms; and arrive at conflicting conclusions about the correspondences between test scores and CEFR levels. This raises questions about the capacity of the CEFR to communicate competing views of a test construct within a coherent overarching structure

Sequencing as an item type.

A text’s coherence clearly depends upon the way ideas are related within that text, both in terms of their logical relations, as well as the cohesive devices that show, or create, the links between ideas, across paragraphs and sentences. Thus, it would appear that part of the ability of a competent reader is to recognize the appropriate order of ideas in text, to identify cohesion and coherence in text in order to relate the ideas to each other, and to understand authorial intention with respect to the sequence of ideas. It follows from this that a potentially useful test method that might tap such abilities is to require candidates to inspect text in which the elements are out of order, and to reconstruct the original order. This, it might be supposed, would require candidates to detect the relationship among ideas, to identify cohesive devices and their interrelationships. Such test methods are, indeed, increasingly common in so-called reading tests. However, we know of no reports of research into, or even descriptions of the use of, this promising task type. In this article we report on potential problems in scoring responses to sequencing tests, the development of a computer program to overcome these difficulties, and an exploration of the value of various scoring procedures