Approximations of Predictive Entropy Correlate with Reading Times

The lexical frequency of an upcoming word affects read-ing times even when the upcoming word is masked fromreaders (Angele et al., 2015). One explanation for thisobservation is that readers may slow down if there is highuncertainty about upcoming material. In line with thishypothesis, this study finds a positive correlation be-tween predictive entropy and self-paced reading times.This study also demonstrates that such predictive en-tropy can be effectively approximated by the surprisalof upcoming observations and that this future surprisalestimate is more predictive of reading times when thegrammar is more granular, which would be prohibitivelyexpensive for predictive entropy. These results suggestreaders engage in fine-grained predictive estimations ofcertainty about upcoming lexical and syntactic material,that such predictions influence reading times, and thatestimating that uncertainty can be done less expensivelyand more robustly with information-theoretic surprisal