Using data from over 14,000 student responses we create item response curves, fitted to the polytomous item response theory model for nominal responses, to evaluate the relative correctness of various incorrect responses to questions on the Force and Motion Conceptual Evaluation (FMCE). Based on this ranking of incorrect responses, we examine individual students\u27 pairs of responses to FMCE questions, using transition matrices and consistency plots, to show how student ideas develop over the span of an introductory mechanics course. Using data from two different schools (N ~= 200 each), we explore how these representations can show student learning even when individuals do not choose the correct answer. Comparing response pairs provides a rich picture of student learning that is unavailable in many traditional analyses
