Skalierung und Linking

Breit, S., & Schreiner, C. (Hrsg.). (2016). Large-Scale Assessment mit R: Methodische Grundlagen der österreichischen Bildungsstandardüberprüfung. Wien: facultas, S. 185-22

Assuming measurement invariance of background indicators in international comparative educational achievement studies: a challenge for the interpretation of achievement differences

Abstract Background Large-scale cross-national studies designed to measure student achievement use different social, cultural, economic and other background variables to explain observed differences in that achievement. Prior to their inclusion into a prediction model, these variables are commonly scaled into latent background indices. To allow cross-national comparisons of the latent indices, measurement invariance is assumed. However, it is unclear whether the assumption of measurement invariance has some influence on the results of the prediction model, thus challenging the reliability and validity of cross-national comparisons of predicted results. Methods To establish the effect size attributed to different degrees of measurement invariance, we rescaled the ‘home resource for learning index’ (HRL) for the 37 countries ( $$n=166,709$$ n = 166 , 709 students) that participated in the IEA’s combined ‘Progress in International Reading Literacy Study’ (PIRLS) and ‘Trends in International Mathematics and Science Study’ (TIMSS) assessments of 2011. We used (a) two different measurement models [one-parameter model (1PL) and two-parameter model (2PL)] with (b) two different degrees of measurement invariance, resulting in four different models. We introduced the different HRL indices as predictors in a generalized linear mixed model (GLMM) with mathematics achievement as the dependent variable. We then compared three outcomes across countries and by scaling model: (1) the differing fit-values of the measurement models, (2) the estimated discrimination parameters, and (3) the estimated regression coefficients. Results The least restrictive measurement model fitted the data best, and the degree of assumed measurement invariance of the HRL indices influenced the random effects of the GLMM in all but one country. For one-third of the countries, the fixed effects of the GLMM also related to the degree of assumed measurement invariance. Conclusion The results support the use of country-specific measurement models for scaling the HRL index. In general, equating procedures could be used for cross-national comparisons of the latent indices when country-specific measurement models are fitted. Cross-national comparisons of the coefficients of the GLMM should take into account the applied measurement model for scaling the HRL indices. This process could be achieved by, for example, adjusting the standard errors of the coefficients

HypotHesis and tHeory article Matrices satisfying Regular Minimality

A matrix of discrimination measures (discrimination probabilities, numerical estimates of dissimilarity, etc.) satisfies Regular Minimality (RM) if every row and every column of the matrix contains a single minimal entry, and an entry minimal in its row is minimal in its column. We derive a formula for the proportion of RM-compliant matrices among all square matrices of a given size and with no tied entries. Under a certain "meta-probabilistic" model this proportion can be interpreted as the probability with which a randomly chosen matrix turns out to be RM-compliant

Matrices Satisfying Regular Minimality

A matrix of discrimination measures (discrimination probabilities, numerical estimates of dissimilarity, etc.) satisfies Regular Minimality (RM) if every row and every column of the matrix contains a single minimal entry, and an entry minimal in its row is minimal in its column. We derive a formula for the proportion of RM-compliant matrices among all square matrices of a given size and with no tied entries. Under a certain “meta-probabilistic” model this proportion can be interpreted as the probability with which a randomly chosen matrix turns out to be RM-compliant

What Text Features Make Reading Comprehension Difficult Across Elementary School? Investigating Difficulty and Changes in Difficulty

Beginning readers benefit in most situations from reading activities that are neither too difficult nor too easy. This study investigated which text features make reading comprehension difficult for third and fourth grade elementary school students. Specifically, 145 multiple-choice items from a reading comprehension test used in several cross-sectional studies (G3: N = 1387; G4: N = 868) and a longitudinal sub-study (N = 195) were analyzed using explanatory item response models to explain item difficulty and changes in item difficulty across grades. A multi-step feature selection procedure controlling for seven task features led to the selection of eight text features from a total of 268 linguistic text features examined. The results showed that lexical and syntactic features and text genre were the most relevant features and that the importance of specific text features changes from third to fourth grade. Expository text were more difficult on average than narrative texts. This difference was only partially explained by lexical and syntactic features in third grade, but almost completely in fourth grade. The results suggest that text features have a dynamic effect on reading comprehension difficulty throughout third to fourth grade; this is especially true of text genre. Our results can help to select more appropriate texts for elementary students and to improve our understanding of the complex interaction between reader, text and activity as it develops over time