Dynamic computerized-adaptive assessment of reading processes: Incremental validity on text comprehension

This study explores the predictive and incremental validity of the Dynamic Computerized Adaptive Reading Processes Assessment Device (EDPL-BAI). This battery has 38 tasks organized in 6 modules to assess different processes related to reading comprehension (e.g. phoneme-grapheme association, executive functions, metacognitive processes). Method: A total of 1336 Chilean students from 3rd to 6th grade (45% females) from 12 public urban and rural schools participated in a longitudinal study consisting of three assessment cycles. On the first cycle, the students were assessed with a standardized reading comprehension test (CL-PT), the Raven Progressive Matrices Test (PMR), and the complete poll of items were calibrated in a non-dynamic version of EDPL-BAI. On the second cycle, seven months after, a sub sample of 1068 students were administered the adaptive and dynamic version of the EDPL-BAI (graduated prompts were displayed after wrong responses), while the remainder 268 students were assessed with the non-dynamic version of EDPL-BAI, and for all the students a pre and post-test (using selected items from EDPL-BAI), CL-PT and PMR were administered. Finally, at the third cycle, four month after, the non-dynamic version of EDPL-BAI and CL-PT were administered to the complete sample. Results and Conclusions: Multiple CFAs were conducted to determine the factorial structure of the EDPL-BAI, and a final model of 6 factors was retained. Predictive and incremental validity of the EDPL-BAI was explored with SEM analysis. The dynamic and non-dynamic versions of the EDPL-BAI were good predictors of reading comprehension. The dynamic version showed incremental validity over the non-dynamic version. Furthermore, graduated prompts predicted reading comprehension performanc

Integrating curriculum-based dynamic assessment in computerized adaptive testing: Development and predictive validity of the EDPL-BAI battery on reading competence

In recent decades, there have been significant changes in the conceptualization of reading as well as in the perception of how this activity should be assessed. Interest in the analysis of reading processes has led to the emergence of new explanatory models based primarily on the contributions of cognitive psychology. In parallel, there have been notable advances in measurement procedures, especially in models based on Item Response Theory (IRT), as well as in the capacity and performance of specific software programs that allow data to be managed and analyzed. These changes have contributed significantly to the rise of testing procedures such as computerized adaptive tests (CATs), whose fundamental characteristic is that the sequence of items presented in the tests is adapted to the level of competence that the subject manifests. Likewise, the incorporation of elements of dynamic assessment (DA) as the prompts are gradually offered allows for obtaining information about the type and degree of support required to optimize the subject’s performance. In this sense, the confluence of contributions from DA and CATs offers a new possibility for approaching the assessment of learning processes. In this article, we present a longitudinal research developed in two phases, through which a computerized dynamic adaptive assessment battery of reading processes (EDPL-BAI) was configured. The research frame involved 1,831 students (46% girls) from 13 public schools in three regions of Chile. The purpose of this study was to analyze the differential contribution on reading competence of dynamic scores obtained in a subsample composed of 324 (47% girls) students from third to sixth grade after the implementation of a set of adaptive dynamic tests of morpho-syntactic processes. The results achieved in the structural equation modeling indicate a good global fit. Individual relationships show a significant contribution of calibrated score that reflects estimated knowledge level on reading competence, as well as dynamic scores based on the assigned value of graduated prompts required by the students. These results showed significant predictive values on reading competence and incremental validity in relation to predictions made by static criterion tests

Desarrollos cuasi-profesionales: la empresa en el aula

La experiencia que se describe en este trabajo ha superado con éxito una meta ambiciosa: introducir en el aula universitaria, lo más fielmente posible, el proceso real de desarrollo de software. Este objetivo no es fácilmente abordable y hemos empleado cinco cursos en obtener un diseño satisfactorio. Este trabajo describe los tres puntos básicos en el desarrollo de la experiencia: la manera de introducir en el aula un problema real planteado por una organización real, la forma de organizar el trabajo mediante equipos estables de alumnos y las implicaciones en la evaluación, incluyendo la necesidad de desarrollar herramientas propias que ayuden en esta tarea. Resaltamos especialmente que este grado de acercamiento al mundo industrial nos ha reportado como beneficio, entre otros, un aumento de la motivación del alumno. Éste ve recompensado su esfuerzo no sólo mediante una calificación académica, sino que hemos introducido también un método que permite a los alumnos obtener un reconocimiento a su trabajo por parte del entorno profesional.Proyectos de innovación educativa PIE06-082 y PIE07-045 de la UMA, y CIDUA-109 de la Junta de Andalucía