Measuring digital literacy across ages and over time: Development and validation of a performance-based assessment

Abstract

The open access publication is available at https://doi.org/10.1007/s10639-025-13592-8Measuring digital literacy (DL) across ages and tracking its growth over time have remained challenging in the area of digital literacy assessment. The current analysis reports on the psychometric properties of a performance-based Digital Literacy Assessment (DLA) instrument grounded in the DigComp 2.1 framework. Utilising a longitudinal cohort study design, the DLA was administered to Hong Kong students across three age cohorts, from lower primary to upper secondary, over a two-year period. Data were collected in 2019, before the COVID-19 pandemic, and in 2021, during the pandemic. The analysis provides validity and reliability evidence for using the DLA in longitudinal studies to assess DL from late childhood through late adolescence. The results further suggest that students’ DL improved with grade level, with secondary students outperforming primary students but also displaying greater variability in scores. Over the two years, students generally demonstrated improvement in DL while inter-individual differences in DL growth rates widened. These findings indicate the widening of digital divides and highlight the need to investigate factors that contribute to diversity in DL development. In conclusion, our study provides evidence for the robustness of the DLA as an instrument to assess DL growth across ages and over time. Further, the DLA allowed us to uncover the substantial overlap in DL ability across different age groups and the widening second-level digital divide as children move into higher grades, and that the digital divide aggravated during the COVID-19 pandemic. Implications and challenges to the learning and assessment of DL are discussed