Utility of a goodness-of-fit index for the graded response model with small sample sizes : a Monte Carlo investigation.

Abstract

Item response theory (IRT) is expanding to diverse research settings, without accompanying access to easily implemented model fit methods. One simple model fit approach involves x2/df ratios. However, its utility is not known across several conditions salient to recent applied IRT research. A Monte Carlo simulation was implemented to investigate the effects of several factors (sample size, adjustment condition, type of misfit, and proportion of misfitting items) on x2/df ratios in the context of the Graded Response Model. Results suggested that: (a) adjusted x2/df ratios were appropriate for the largest sample size condition (N=10000), but were extremely inflated for small (N=400) and medium (N=1500) conditions; (b) x2/df ratios were differentially affected across sample sizes by type and amount of misfit; and (c) sensitivity of the x2/df\u3e 3 cut point for identifying misfit in single items was notably low across all study conditions. Implications, limitations, and future directions are discussed