Automatic coding of short text responses via clustering in educational assessment

Automatic coding of short text responses opens new doors in assessment. We implemented and integrated baseline methods of natural language processing and statistical modelling by means of software components that are available under open licenses. The accuracy of automatic text coding is demonstrated by using data collected in the Programme for International Student Assessment (PISA) 2012 in Germany. Free text responses of 10 items with Formula responses in total were analyzed. We further examined the effect of different methods, parameter values, and sample sizes on performance of the implemented system. The system reached fair to good up to excellent agreement with human codings Formula Especially items that are solved by naming specific semantic concepts appeared properly coded. The system performed equally well with Formula and somewhat poorer but still acceptable down to Formula Based on our findings, we discuss potential innovations for assessment that are enabled by automatic coding of short text responses. (DIPF/Orig.

A Theme-Rewriting Approach for Generating Algebra Word Problems

Texts present coherent stories that have a particular theme or overall setting, for example science fiction or western. In this paper, we present a text generation method called {\it rewriting} that edits existing human-authored narratives to change their theme without changing the underlying story. We apply the approach to math word problems, where it might help students stay more engaged by quickly transforming all of their homework assignments to the theme of their favorite movie without changing the math concepts that are being taught. Our rewriting method uses a two-stage decoding process, which proposes new words from the target theme and scores the resulting stories according to a number of factors defining aspects of syntactic, semantic, and thematic coherence. Experiments demonstrate that the final stories typically represent the new theme well while still testing the original math concepts, outperforming a number of baselines. We also release a new dataset of human-authored rewrites of math word problems in several themes.Comment: To appear EMNLP 201

Incidence of Executive Functions on Reading Comprehension Performance in Adolescents

Introducción.La comprensión lectora es una habilidad cognitiva compleja que se ha asocia-do a procesos ejecutivoscomola memoria de trabajo (MT) y la inhibición. Dado que el desa-rrollo de estas habilidades continúa hasta la adolescenciatardía, esta investigación propone estudiar el rol que sobre los distintos niveles de rendimiento en comprensión lectora ejercen ambos procesosen adolescentes de dos franjas etarias.Método.Se evaluó el desempeño en comprensión de textos, MT,inhibición,fluidez lectora y habilidades verbales en 104 adolescentes (12/13 años n= 53; 17/18 años n= 51).Resultados.Se encontró que los desempeños en comprensión lectora, MTe inhibición pre-sentan incrementossignificativos según la edad. A su vez, la correlación parcial mostró que lacomprensión lectora se asocia a: habilidades verbales, MTe inhibición.Por último, las prue-bas de diferencia de grupos señalan que los estudiantes con alto y bajo rendimiento en com-prensión difieren significativamente en habilidades verbales y funcionamiento ejecutivo en la franja 12/13 años. En cambio, sólo difieren significativamente en habilidades verbales en la franja 17/18 años.Conclusión.La comprensión lectora, la MT y la inhibición mejoran durante la adolescencia. Sin embargo, la relación entre estas habilidades difiere según el momento etario considerado.Introduction.Reading comprehension is a complex cognitive skill that has been associated with executive functions such as working memory (WM) and inhibition. Given that the development of these abilities continues through late adolescence, this study seeks to explore the role that both processes play with respect to varying levels of reading comprehension performance in adolescents in two age groups.Method.We evaluated performance in the comprehension of texts, WM, inhibition, reading fluency and verbal skills in 104 adolescents (12/13 years of agen= 53; 17/18 years of agen= 51).Results.We found that reading comprehension performance, WM and inhibition increased significantly with age. Further, a partial correlation showed that reading comprehension is associated with: verbal skills, WM and inhibition. Lastly, group difference testing indicates that students with high and low comprehension performance differ significantly in verbal skills and executive functioning in the 12/13 age group. In the 17/18 age group, however, they only differed significantly in terms of verbal skills.Conclusion.Reading comprehension, WM and inhibition improve during adolescence. However, the relationship among these abilities differs by age group.Fil: Demagistri, Maria Silvina. Universidad Nacional de Mar del Plata. Facultad de Psicología. Centro De Investigación en Procesos Básicos, Metodologías y Educación; ArgentinaFil: Richard's, Maria Marta. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentina. Universidad Nacional de Mar del Plata. Facultad de Psicología. Centro De Investigación en Procesos Básicos, Metodologías y Educación; ArgentinaFil: Canet Juric, Lorena. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentina. Universidad Nacional de Mar del Plata. Facultad de Psicología. Centro De Investigación en Procesos Básicos, Metodologías y Educación; Argentin