Hierarchical Modeling with Large‐Scale Assessment Data: Influence of Intra-Class Correlation on Sampling Precision

Abstract

Most data collected in educational large scale assessments (LSA) is very well suited for multilevel modeling because sampled individuals are usually nested within clusters (e.g., students nested within schools). Hierarchical models allow for the effect of explanatory variables at different clustering levels; parameters can be determined as fixed or random effects depending on the research question. All model parameters are however estimated based on random samples and are therefore subject to sampling error. A Monte Carlo simulation was utilized to explore the connections between the sample sizes at different levels and intra-class correlation coefficients (ICCs) in settings that mimic scenarios typical for LSA. It was observed that varying levels of ICC influence the margins of sampling variance of the estimated model parameters in different amounts and even different directions. Assuming fixed sample sizes, the coefficients of variation (CVs) of the model parameters mean of random intercepts (γ00) and slope of random intercepts (γ01) increased with increasing ICC levels, as expected. However, the inverted relationship was observed for parameters U0 – variance of random intercepts, γ10 – mean of random slopes, and β1 – fixed slope: with increasing ICC, the CVs decreased. The findings can help us to determine sample sizes in LSA when particular hierarchical models are to be investigated.status: publishe