Abstract: Guessing on closed-ended knowledge items is common. Under likely to hold assumptions, in presence of guessing, the most common estimator of learning, difference between pre- and post-process scores, is negatively biased. To account for guessing related error, we develop a latent class model of how people respond to knowledge questions and identify the model with the mild assumption that people do not lose knowledge over short periods of time. A Monte Carlo simulation over a broad range of informative processes and knowledge items shows that the simple difference score is negatively biased and the method we develop here, unbiased. To demonstrate its use, we apply our model to data from Deliberative Polls. We find that estimates of learning once adjusted for guessing are about 13\% higher. Adjusting for guessing also eliminates the gender gap in learning, and halves the pre-deliberation gender gap on political knowledge.
Replicating Guessing and Forgetting
Download replication materials. Open the readme file and follow the specific instructions for how to replicate the results