Ntouskos, Valsamis

Pirri, Fiora

Potapova, Ekaterina

Vincze, Markus

Weiss, Astrid

Zillich, Michael

English

BieColl - Bielefeld eCollections

A Pilot Study on Eye-tracking in 3D SearchTasksEkaterina Potapova, Valsamis Ntouskos, Astrid Weiss, Michael Zillich, MarkusVincze and Fiora Pirri1 IntroductionEye-tracking is an important step in the evaluation of computational visual attentionmodels for comparison to human visual perception. There exist two types of visualattention: bottom-up and top-down attention. Bottom-up attention is driven by ob-ject properties, while top-down attention takes place when there is a specific searchtask at hand. There exist several datasets for bottom-up attention, less experimentswere made to collect eye-tracking for specific search tasks, such as driving a car byBorji et al. [1]. In this work we are interested in collecting statistically valuableeye-tracking data for visual search tasks that are typical in daily life and of greatinterest to researchers in robotics and HRI.The majority of existing datasets is concerned with the analysis of static imagesor video sequences. However, it has been shown that the inclusion of depth per-ception changes human attention behavior [2] and several attempts have been madeto collect eye-tracking data while free-viewing stereo image pairs [3], artificiallycreated 3D scenes [4], and even real 3D environments [7]. Therefore, we want tocreate a dataset in which people perform tasks in real 3D environments. Collectingsuch experimental 3D eye-tracking data will strongly benefit the community, andwill show the influence of depth and motion on human perception.Ekaterina Potapova, Astrid Weiss, Michael Zillich, Markus VinczeACIN, Vienna University of Technology, e-mail: {potapova,weiss,zillich,vincze}@acin.tuwien.ac.atValsamis Ntouskos, Fiora PirriALCOR, University of Rome, e-mail: {ntouskos,pirri}@dis.uniroma1.itThe research leading to these results has received funding from the Austrian Science Foundation(FWF) under grant agreements No. TRP 139-N23, InSitu and No. TRP 623-N23, V4HRC, and theEuropean Communitys Seventh Framework Programme FP7/2007-2013 under grant agreementNo. 600623, STRANDSDOI: 10.2390/biecoll-saga2013_1In Pfeiffer and Essig (2013): Proceedings of the First International Workshop on Solutions for Automatic Gaze-Data Analysis 2013 (SAGA 2013). DOI: 10.2390/biecoll-saga2013_0E. Potapova, V. Ntouskos, A. Weiss, M. Zillich, M. Vincze and F. PirriWe aim at producing a pilot dataset with 3D eye-tracking data from three par-ticipants to check if it is feasible to create a benchmark for saliency algorithms andcomputational attention models aimed at solving top-down search tasks.2 Task DescriptionIn the following we will present two exemplary tasks participants were asked toperform in our pilot study: a free-viewing and a counting task. We collected eye-tracking data in real cluttered environments using the Gaze Machine [5].2.1 Experimental SetupAs a setup for our experiments we chose a cluttered scene, as it offers a varietyof possibilities for bottom-up and top-down search tasks. The selected scene rep-resented a pile of toys as it can be found in every child’s room. Toys are stackedtogether and occluding each other. Therefore, 3D data is essential, because depthcan potentially help to separate items. Our experimental setup consisted of a set oftoys, randomly placed on the table. A white screen was placed in front of the partici-pant and lifted for a specific amount of time, ensuring a precisely defined duration ofviewing the scene. All questions to participants and their answers were documented.Additionally, experiments were video taped.2.2 Free-viewing TaskIn this task we studied bottom-up attention. In order to keep participants motivatedthe goal was to remember as many details as possible during a limited period of30 seconds. After the completion of the experiment participants were asked severalquestions, to check their performance, such as ’Where can this type of scene betypically observed?’, ’How many toys are in the scene?’, ’Could you please namesome toys you have seen?’. Points of regard were mapped onto image sequencesfor each participant separately to create preliminary fixation distance maps. We ex-pected participants to be attracted by regions being in contrast with the surrounding,similar to [6].A Pilot Study on Eye-tracking in 3D Search Tasks2.3 Counting TaskIn this task we studied top-down attention. We asked participants to search for cars inthe scene and count them loudly when they find one. Search patterns were recorded.The experiment was stopped when a participant reported that all cars found. Withsuch data it becomes possible to study human behaviour in terms of evaluation ofdirected top-down visual attention. We expected participants to sequentially searchthe scene from the most attractive region, i.e. saliency region, to the least attractive.3 ParticipantsAs a pilot study we invited three participants to check the feasibility of our method-ological approach. All three were male students at the University of Rome, with agefrom 27 to 29. All of them had normal or corrected to normal vision.4 Data ProcessingTo create a benchmark that can be used by other researchers, a 3D reconstructionmodel of the scene was created at first. Points of regard were localized and mappedonto this reconstruction. However, because of lighting conditions and image blurri-ness due to participants movements, it was not possible to localize points of regardon the unified model. Therefore, it was not possible to create hit maps in 3D. Theseissues will be addressed in the followed-up main study. Additionally, points of re-gard were mapped to one of two images in a stereo pair. Fixation distance maps werecreated for each frame for each participant. No averaged fixation distance maps werecreated at this points.5 Preliminary ResultsNo hit maps were created in the pilot study, therefore it is not possible to quan-titatively compare eye-tracking data with existing models of visual attention. Pre-liminary results show that attentional models designed to work on static images orvideos fail to predict fixations when they are applied in real world environments.The results of the free-viewing experiment were qualitatively compared to contrast-based attentional model described by Itti et al. [6]. Comparison showed that whilecontrast-based model predicts fixations on objects popping out from the scene basedon color, human fixations were mostly distributed in areas with multiple tiny objects.One interesting question here is, whether this behaviour was influenced by the giventask to motivate participants or by the nature of human attention. Potentially, it canE. Potapova, V. Ntouskos, A. Weiss, M. Zillich, M. Vincze and F. Pirrishow that humans are attracted by textured regions. This aspect should be investi-gated in more details.Fixation data from counting experiment showed that humans tend to return toprevious fixations. Currently existing models of human visual attention do not as-sume the possibility to go back and forth between already attended location. Poten-tially, this finding can indicate that humans tend to verify their visual experience byexamining same regions with interesting objects several times.In general, the pilot study greatly helped to adjust the setting for the upcomingstudies. Data, obtained from the pilot study can lead to designing an attentionalsystem comparable to human performance, when depth and orientation in space aretaken into account.6 Conclusion and Future WorkIn this paper we addressed the issue of creating a benchmark for visual attentionmodels based on eye-tracking data in realistic 3D environments. As an example asetup with cluttered toys was created. Participants performed two different tasks:free-viewing and counting. Pilot study showed that classical contrast-based modelsfail to predict human fixations in real 3D environments. Next steps will be to create aunified 3D reconstructed model of the scene and map fixation points onto this model.Our future work will also include the design of a general protocol to perform moreexperiments on visual search and algorithms to process eye-tracking data.References1. Borji A., Sihite D., Itti L. (2012). Probabilistic Learning of Task-Specific Visual Attention.In: Computer Vision and Pattern Recognition, 2012.2. Wexler M., Ouarti N. (2008). Depth Affects Where We Look. In: Current Biology 18, 2008.3. Lang C., Nguyen T., Katti H., Yadati K., Kankanhalli M., Yan S. (2012). Depth Matters:Influence of Depth Cues on Visual Saliency. In: European Conference on Computer Vision,2012.4. Vishwanath D., Kowler E. (2004). Saccadic localization in the presence of cues to three-dimensional shape. In: Journal of Vision 4, 2004.5. Pirri F., Pizzoli M., Rudi A. (2011). A general method for the point of regard estimation in3D space. In: Computer Vision and Pattern Recognition, 2011.6. Itti L., Koch C., Niebur E. (1998). A Model of Saliency-Based Visual Attention for RapidScene Analysis. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11),1998.7. Santner K., Fritz G., Paletta L., Mayer H. (2013). Visual Recovery of Saliency Maps fromHuman Attention in 3D Environments. In: IEEE International Conference on Robotics andAutomation, 2013.

A Pilot Study on Eye-tracking in 3D Search Tasks

https://biecoll.ub.uni-bielefeld.de/index.php/saga/article/download/543/638

A Pilot Study on Eye-tracking in 3D Search Tasks

Abstract

Similar works

Full text

Available Versions

BieColl - Bielefeld eCollections