Engbert, Ralf

Rothkegel, Lars O. M.

Schütt, Heiko H.

Trukenbrod, Hans A

Wichmann, Felix A.

English

Purdue E-Pubs

Predicting the fixation density over timeHeiko H. Schütt, Lars O. M. Rothkegel, Hans A. Trukenbrod,Ralf Engbert & Felix A. WichmannA BFigure 1: Overall saliency model performance onthe Corpus dataset. A simple weighted sum of aearly visual representation performs slightlybetter than classical saliency models, butDeepGaze II performs substantially better usinghigh-level information.A BFigure 2: Performance of the saliency models overtime. Empirical saliency and Central fixation biaswere estimated as kernel density estimates withleave one subject out crossvalidation based oneither fixations with the same number or allfixations except for the first.BA CFigure 3: Saliency model performance on theSearch dataset. The dot-dashed linerepresents the performance of the unadjustedDeepGaze II model prediction.When modelling human eye movements we usually separate bottom-up andtop-down effects, i.e. whether some effect is caused by the stimulus or by someinternal state of the observer like their tasks or intentions. We also separatethe orthogonal dimension whether the features used to guide eye movementsare low-level—like local contrast, luminance or orientation content—or high-level—like object locations, scene congruence or scene category. Furthermore,humans display systematic tendencies in their eye movements, preferring certainsaccade lengths and directions and a tendency to continue moving into the samedirection.It is unclear how these different factors interact to determine were we lookand most models only include a small selection of the mentioned influencefactors. To disentangle them, we analyse the dependencies between fixationsand the fixation densities over time. In our analysis we include how well fixationdensities are predicted by, first, low-level bottom-up saliency, including a saliencymodel based on our early spatial vision model (Schütt and Wichmann, 2017),and, second, a recent DNN-based saliency model including low- and high-levelbottom-up saliency (DeepGaze II, Kümmerer et al., 2016).To separate top-down effects, we use two datasets: One Corpus datasetin which 105 subjects looked at 90 images to memorize them and a searchdataset in which 10 subjects searched for 6 different targets with varying spatialfrequency and orientation content superimposed over 25 natural images 8 timeseach resulting in 480 searches per image.Based on the Corpus Dataset we separate the exploration into three phases:An onset response with the first saccade, an initial exploration lasting around10 fixations and a final equilibrium phase. First fixations are most predictablebut follow a different density than later ones. During the initial explorationfixations gradually become less predictable. Finally, the fixation density stopsbroadening and the equilibrium state is reached in which fixations are leastfocussed but still favour the same areas as during the exploration.The prediction quality of all saliency models follows the curve of predictableinformation. They predict fixations best at the beginning and gradually getworse. The simple saliency model based on our early spatial vision modelperforms as well as classical saliency models. However, DeepGaze II performssubstantially better by using high-level information throughout the whole trial.This advantage is present at the latest 200 ms after image onset; however,the predictive power of the early vision saliency model for the first fixation(s)is much better than for later fixations, and as a corollary the advantage ofDeepGaze II is relatively small for the first fixation(s).On the search dataset all saliency models perform badly after a small initialprediction success, even if the non-linearity and central fixation bias are newlyadjusted to the search data. Instead we observe that subjects adjust wherethey look and their eye movement dynamics to the target they search for.Specifically they make shorter saccades and exhibit shorter fixation times forhigher frequency targets than for lower frequency targets.Our observations confirm that bottom-up guidance of eye movements can beoverwritten almost entirely by task effects in static natural scenes. Nonethelessour data support some early bottom-up guidance, which includes high-levelfeatures already for the very first saccade.ReferencesKümmerer, M., Wallis, T. S. A., and Bethge, M. (2016). DeepGaze II: Readingfixations from deep features trained on object recognition. arXiv:1610.01563[cs, q-bio, stat].Schütt, H. H. and Wichmann, F. A. (2017). An image-computable psychophysi-cal spatial vision model. Journal of Vision, 17(12):12:1–35.1

Predicting the Fixation Density Over Time

https://docs.lib.purdue.edu/cgi/viewcontent.cgi?article=1128&amp;context=modvis

Predicting the Fixation Density Over Time

Abstract

Similar works

Full text

Available Versions

Purdue E-Pubs