New model-data fit indices for item response theory (IRT): an evaluation and application

Abstract

I reviewed the recently developed limited-information model fit statistics by Maydeu-Olivares and colleagues (e.g., Maydeu-Olivares & Joe, 2005; Maydeu-Olivares & Liu, 2012; Liu & Maydeu-Olivares, 2014) and conducted a simulation study to explore the properties of these new statistics under conditions often seen in practice. The results showed that the overall and piecewise fit statistics were to some extent sensitive to misfit caused by multidimensionality, although the limited-information fit statistics tended to flag more item pairs as misfit than the heuristic fit statistics. I also applied the fit statistics to three AP® exams, one personality inventory, and a rating scale used in organizational settings. Although a unidimensional IRT model was expected to fit the Physics B Exam better than the English Literature Exam, the average piecewise fit statistics showed no such difference. The fit statistics also suggested that a more advanced IRT model should be fitted to the self-rated personality inventory. Finally, the fit statistics seemed to be effective in detecting misfit caused by data skewness