All good readers are the same, but every low-skilled reader is different: an eye-tracking study using PISA data

Abstract

PISA results show that a considerable number of fifteen-years old pupils after 8-10 years of schooling have a low level of functional reading literacy, as defined in the PISA framework. While PISA results help identify level of reading competency, they do not reveal what might be the reasons why some students fail to solve the tasks. One way to explore the difficulties pupils encounter while solving PISA reading tasks is to track their eye-movements during reading. The main aim of this study was to explore the similarities and differences in eye-movement patterns between pupils with high and low scores on PISA reading tasks. A sample of 92 students took part in the pre-test, which was based on PISA items, and administered to identify groups of students with high and low PISA reading scores. Based on student pre-test results, 20 students were selected for the main, eye-tracking test - 10 participants with low (average pre-test result M=431, SD=69) and 10 with high scores (M=695, SD=76). The eye-tracking test consisted of four different released PISA reading tasks, three of them continuous and one non-continuous. The continuous items were followed by one multiple-choice question each, at L1, L2 and L3 levels of difficulty. The non-continuous text was followed by three multiple-choice questions (also L1-L3). To explore continuous items, the following measures were explored: average fixation duration, median saccade amplitude, percentage of regressions and global reading speed (words per minute). Median saccade amplitudes were consistently shorter on all items in the low-scoring group. The non-continuous task was explored using several eye-tracking measures, but none were significant. However, the standard deviations of almost all variables were lower in the high-scoring group. Finally, qualitative analysis of heat maps showed that the high-scoring group was more efficient in searching for the relevant information in text, while the attention of the low-scoring students was more scattered around