Finding the right pace: How speech rate influences item difficulty of a listening comprehension test.

Abstract

To comprehend spoken language is crucial, especially in school since teachers’ verbal explanations form the basis of learning in almost every subject. Given the rising number of immigrant students, valid and reliable listening comprehension (LC) tests become increasingly important. They help detecting struggling students and identifying specific competencies that need fostering. Unfortunately, most components of LC tests and their effect on item difficulty are still poorly understood, among them the speech rate which is usually taken as given characteristic of the used stimulus. But especially the speed of presenting information could easily be altered by teachers which makes understanding its impact on item difficulty an essential research endeavour. Based on two consecutive studies including n=419 Luxembourgish 3rd graders, the present paper discusses how manipulating the speech rate of informative, as well as narrative texts impacts the psychometric difficulty of a LC test in German. Four different texts were presented in two versions of varying speed: A slower version, using the mean speech rate of available LC units; a second version with higher speech rate, being oriented at authentic German radio programs. The different versions were anchored via additional, non-manipulated units and item difficulty estimation was done within IRT framework. In line with previous studies, higher speech rate increased item difficulty (Cohen’s d ranging from 0.4 to 1.08) but only for narrative texts. Surprisingly, the opposite effect was found for informative texts (d ranging from -1.07 to -0.46), showing that the slower version was more difficult to comprehend. We will discuss potential explanations for these findings and concrete take-aways for future LC test construction