State of the art review : language testing and assessment (part two).

In Part 1 of this two-part review article (Alderson & Banerjee, 2001), we first addressed issues of washback, ethics, politics and standards. After a discussion of trends in testing on a national level and in testing for specific purposes, we surveyed developments in computer-based testing and then finally examined self-assessment, alternative assessment and the assessment of young learners. In this second part, we begin by discussing recent theories of construct validity and the theories of language use that help define the constructs that we wish to measure through language tests. The main sections of the second part concentrate on summarising recent research into the constructs themselves, in turn addressing reading, listening, grammatical and lexical abilities, speaking and writing. Finally we discuss a number of outstanding issues in the field

The effect on the doze test of changes in deletion frequency

Although the pseudo-random cloze procedure has been in use for some twenty-five years as a measure of readability and reading comprehension, little research has been carried out into the effect of deleting words from text more or less frequently. This paper reports on an experiment in which the deletion frequency variable was systematically studied. Every 6th, 8th, 10th and 12th word was removed from three texts of differing difficulty, and the effect studied. Significant differences among cloze tests resulted, but the differences were unpredictable. Deleting every 12th word did not necessarily result in an easier test than deleting every 6th 8th or 10th word. However, when only items identical to both cloze tests under consideration were compared, no significant differences were found. It appears that cloze items are, on the whole, unaffected by context greater than five words. Testers are warned that changing deletion frequency may result in a different measure of readability or comprehension.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/71392/1/j.1467-9817.1979.tb00198.x.pd

A study of the cloze procedure with native and non-native speakers of English

This study examined. various aspects of the methodology of the close procedure to determine their effect on the nature of close tests. It was hypothesised that changes in the frequency of word deletion, in the difficulty of the original text and in the procedure used to judge acceptable restorations of the deleted word would produce significantly different close tests and would result in varying correlations with measures of English proficiency. Three texts were selected and each was subjected to the deletion of every sixth, eighth, tenth and. twelfth word, to give twelve close tests. Five procedures were developed to score the responses to these tests for the degree of similarity they showed to the deleted. words. The tests were administered to 360 adolescent native speakers of English and. 360 adult non-native speakers of English wtho were pursuing further studies in Britain. It was found that significant differences existed among close tests when deletion frequency was changed, but that se scoring procedures reduced. this effect, The change in deletion frequency had. no effect on the measurement of text difficulty, but significant interactions were observed, among the three experimental variables. Different cloze tests gave unpredictably different measures of English proficiency. A study of identical deletions showed. that no increase in the predictability of deleted word was gained. by increasing context from five words to eleven words. Since the quantity of context had no effect on predictability, it was suggested that close is essentially sentence-bound. The nature of the correlations of cloze with measures of English proficiency and the results of factor analyses suggested that cloze is a better test of syntax and lexis than of higher-order reading abilities. Implications for future use of the doze procedure are presented and suggestions made for further research

Exploring the role of phraseological knowledge in foreign language reading

Foreign language (FL) knowledge has been shown to contribute significantly to FL reading performance. Studies have contrasted the contribution of FL vocabulary and syntactic knowledge, following a dichotomous view of these components, producing mixed results. Despite the increasingly recognized formulaic nature of language, the contribution made by phraseological knowledge to reading ability has not been investigated systematically. This study examines the impact of a broader construct definition of linguistic knowledge – which includes a phraseological component – in explaining variance in reading performances. Test scores of 418 learners of English as a foreign language (EFL) were modeled in a structural equation model, showing that a phraseological knowledge measure outperformed traditional syntactic and vocabulary measures in predicting reading comprehension variance. Additional insights into the role of phraseological knowledge were gained through verbal protocol analysis of 15 EFL learners answering reading comprehension items that targeted the understanding of phrasal expressions within written context. The findings hint at an underestimated, but critical, role of phraseological knowledge in FL reading, and are relevant to both the assessment and the teaching of EFL ability

Diagnostic assessment of reading and listening in a second or foreign language:elaborating on diagnostic principles

Alderson, Brunfaut and Harding (2014) recently investigated how diagnosis is practised across a range of professions in order to develop a tentative framework for a theory of diagnosis in second or foreign language (SFL) assessment. In articulating this framework, a set of five broad principles were proposed, encompassing the entire enterprise of diagnostic assessment. However, there remain questions about how best to implement these principles in practice, particularly in identifying learners’ strengths and weaknesses in the less well-documented areas of SFL reading and listening. In this paper, we elaborate on the set of principles by first outlining the stages of a diagnostic process built on these principles, and then discussing the implications of this process for the diagnostic assessment of reading and listening. In doing so, we will not only elaborate on the theory of diagnosis with respect to its application in the assessment of these skills, but also discuss the ways in which each construct might be defined and operationalized for diagnostic purposes

Bridging assessment and learning:a view from second and foreign language assessment

This paper considers issues around the relationship between assessment and learning, as put forward by Baird, Andrich, Hopfenbeck and Stobart (2017), from the perspective of the field of second and foreign language assessment. In our response, we describe shared observations on the nature of research and practice in general educational assessment and in language assessment (including with respect to linking assessment with theories of learning, managing impact, and enhancing assessment literacy). At the same time, we also identify areas where language assessment seems to diverge from current research and practice in general educational assessment (for example in the areas of assessment purposes, construct definitions, and validation theory and practice). As a consequence, we believe that close monitoring of advances in both fields is likely to be mutually beneficial

Bridging assessment and learning:a view from second and foreign language assessment

This paper considers issues around the relationship between assessment and learning, as put forward by Baird, Andrich, Hopfenbeck and Stobart (2017), from the perspective of the field of second and foreign language assessment. In our response, we describe shared observations on the nature of research and practice in general educational assessment and in language assessment (including with respect to linking assessment with theories of learning, managing impact, and enhancing assessment literacy). At the same time, we also identify areas where language assessment seems to diverge from current research and practice in general educational assessment (for example in the areas of assessment purposes, construct definitions, and validation theory and practice). As a consequence, we believe that close monitoring of advances in both fields is likely to be mutually beneficial

Exploring the Ecological Validity of Thinking on Demand: Neural Correlates of Elicited vs. Spontaneously Occurring Inner Speech

Psychology and cognitive neuroscience often use standardized tasks to elicit particular experiences. We explore whether elicited experiences are similar to spontaneous experiences. In an MRI scanner, five participants performed tasks designed to elicit inner speech (covertly repeating experimenter-supplied words), inner seeing, inner hearing, feeling, and sensing. Then, in their natural environments, participants were trained in four days of random-beep-triggered Descriptive Experience Sampling (DES). They subsequently returned to the scanner for nine 25-min resting-state sessions; during each they received four DES beeps and described those moments (9 × 4 = 36 moments per participant) of spontaneously occurring experience. Enough of those moments included spontaneous inner speech to allow us to compare brain activation during spontaneous inner speech with what we had found in task-elicited inner speech. ROI analysis was used to compare activation in two relevant areas (Heschl’s gyrus and left inferior frontal gyrus). Task-elicited inner speech was associated with decreased activation in Heschl’s gyrus and increased activation in left inferior frontal gyrus. However, spontaneous inner speech had the opposite effect in Heschl’s gyrus and no significant effect in left inferior frontal gyrus. This study demonstrates how spontaneous phenomena can be investigated in MRI and calls into question the assumption that task-created phenomena are often neurophysiologically and psychologically similar to spontaneously occurring phenomena