Article thumbnail
Location of Repository

Eye Tracking: A Perceptual Interface for Content Based Image Retrieval

By Oyekoya Oyekoya


In this thesis visual search experiments are devised to explore the feasibility of an eye gaze driven search mechanism. The thesis first explores gaze behaviour on images possessing different levels of saliency. Eye behaviour was predominantly attracted by salient locations, but appears to also require frequent reference to non-salient background regions which indicated that information from scan paths might prove useful for image search. The thesis then specifically investigates the benefits of eye tracking as an image retrieval interface in terms of speed relative to selection by mouse, and in terms of the efficiency of eye tracking mechanisms in the task of retrieving target images. Results are analysed using ANOVA and significant findings are discussed. Results show that eye selection was faster than a computer mouse and experience gained during visual tasks carried out using a mouse would benefit users if they were subsequently transferred to an eye tracking system. Results on the image retrieval experiments show that users are able to navigate to a target image within a database confirming the feasibility of an eye gaze driven search mechanism. Additional histogram analysis of the fixations, saccades and pupil diameters in the human eye movement data revealed a new method of extracting intentions from gaze behaviour for image search, of which the user was not aware and promises even quicker search performances. The research has two implications for Content Based Image Retrieval: (i) improvements in query formulation for visual search and (ii) new methods for visual search using attentional weighting. Futhermore it was demonstrated that users are able to find target images at sufficient speeds indicating that pre-attentive activity is playing a role in visual search. A current review of eye tracking technology, current applications, visual perception research, and models of visual attention is discussed. A review of the potential of the technology for commercial exploitation is also presented

Publisher: UCL (University College London)
Year: 2007
OAI identifier:
Provided by: UCL Discovery

Suggested articles


  1. [104] Imperial College Innovations Ltd. Manipulation of Image Data.
  2. (2006). [19] Grecu Horia
  3. (2005). [43] Multimedia Understanding through Semantics, Computation and Learning, Network of Excellence.
  4. (1998). [47] Ohno Takehiko
  5. [60] Rensink RA (2001). Change Blindness: Implications for the nature of attention.
  6. [64] Scherffig Lasse (2005). It's in Your Eyes - Gaze Based Image Retrieval in Context. Edited by
  7. (2003). [78] Virpi Roto (2003), “Search on Mobile Phones,” Best Practices and Future Visions for Search User Interfaces: A Workshop,
  8. (2002). A Breadth-First Survey of Eye Tracking Applications,
  9. (2002). A cost effective solution for eye-gaze assistive technology,
  10. (1990). A gaze-responsive self-disclosing display.
  11. (1963). A method of measuring eye movement using a scleral search coil in a magnetic field,
  12. (2004). A New Enhancement to Histogram Based Approaches in Content Based Image Retrieval System,
  13. (1999). A three-level model of comparative visual search,
  14. (1998). A Tool for Creating Eye-aware Applications that Adapt to Changes
  15. (1978). Accurate three-dimensional eye tracker,
  16. (2002). Accurate visual memory for previously attended objects in natural scenes.
  17. (2005). Alcon Refractivehorizons, Inc Title: Hybrid Eye Tracking System and Associated Methods (WO 2005/063154) Publication Date: 14
  18. (2000). Algorithms for Defining Visual Regions of Interest: Comparison with Eye Fixations,
  19. (2003). An adaptive approach towards contentbased image retrieval,
  20. (2005). An Attention Based Similarity Measure Used to Identify Image Clusters,
  21. (2001). An estimator for visual attention through competitive novelty with application to image compression, Picture Coding Symposium,
  22. (1987). An evaluation of an Eye Tracker as a Device for Computer Input, Proceeding of SIGCHI+GI '87, Human Factors in Computing Systems. Also published as a special issue of the SIGCHI Bulletin.
  23. (2001). An evolutionary programming approach to the simulation of visual attention,
  24. (2006). An eye tracking interface for image search.
  25. (2006). An Eye Tracking Interface for Visual Exploration,
  26. (1998). Analysis and Models of Eye Movements in Comparative Visual Search, Göttingen: Cuvillier (Ph.D. thesis),
  27. (1999). Applicant: Dynamic Digital Depth Research Pty. Ltd. - 221 -Title: Eye Tracking Apparatus (WO 1999/027412) Publication Date: 03
  28. (2003). Applicant: Imperial College Innovations Ltd Title: Manipulation of Image Data (WO 2003/024319) Publication Date: 27
  29. (2001). Applicant: Memphis Eye and Cataract Associates Ambulatory Surgery Center Title: Eye Tracking and Positioning System for a Refractive Laser System
  30. (2001). Applicant: Memphis Eye and Cataract Associates Ambulatory Surgery Center Title: Method and System for Control of High Resolution High Speed Digital Micromirror Device for Laser Refractive Eye Surgery (WO 2001/085045) Publication Date: 15
  31. (2005). Applicant: New York University Title: Method and Apparatus for an Autostereoscopic Display having a Lenticular Lenslet Array (WO 2005/079376) Publication Date:
  32. (2001). Applicant: Nokia Corporation Title: Eye-Gaze Tracking (WO 2001/049167) Publication Date:
  33. (2000). Applicant: San Diego State University Foundation Title: Method and Apparatus for Eye Tracking (WO 2000/054654) Publication Date:
  34. (2005). Applicant: SR Labs Title: Method to improve the Data Entry and Management of Information related to Customers Relationship Management Systems (CRM) (WO 2005/124518) Publication Date:
  35. (1999). Applicant: University of Washington Title: Virtual Retinal Display with Eye Tracking (WO 1999/036826) Publication Date:
  36. (2003). Applicant: VISX Inc Title: Methods and Systems for Laser Calibration and Eye Tracker Camera Alignment (WO 2003/090867) Publication Date:
  37. (1999). Applied Science Laboratories Title: An Eye Tracker Using an Off-Axis, Ring Illumination Source (WO 1999/005988) Publication Date: 11
  38. (2000). Applying Eye Tracking As Alternative Approach for Activation of Controls and Functions in Aircraft,
  39. (1973). Attention and Effort, Prentice-Hall, Inc., Englewood Cliffs,
  40. (2004). Automatic foveation for video compression using a neurobiological model of visual attention.
  41. (1999). Bid Instruments Limited Title: Apparatus and Method for Visual Field Testing (WO 1999/022638) Publication Date: 14
  42. (2003). Blink Detection and Eye Tracking for Eye Localization,
  43. (2000). Button selection for general GUIs using eye and hand together,
  44. (2005). Combining Eye Movements and Collaborative Filtering for Proactive Information Retrieval,
  45. Communication by Gaze Interaction (COGAIN), IST-2003-511598: Deliverable 7.2. Available at [102] World Intellectual Property Organisation –
  46. (2001). Comparison of Cortical Activity and Oculomotor Performance in the Gap and Step Paradigms.
  47. (1997). Consumer Eye Movement Patterns on Yellow Pages Advertising.
  48. (2003). Contour detection based on nonclassical receptive field inhibition.
  49. (2003). Could I have the menu please? An eye tracking study of design conventions,
  50. (2005). Customvis Plc Title: Limbal-Based Eye Tracking (WO 2005/065527) Publication Date: 21
  51. difference is significant at the .05 level. a Adjustment for multiple comparisons: Least Significant Difference (equivalent to no adjustments). Multivariate Tests Value F Hypothesis df Error df Sig. Partial Eta Squared Pillai's trace
  52. Estimated Marginal Means Mode Estimates 95% Confidence Interval Measure Mode Mean Std. Error Lower Bound Upper Bound Eye
  53. (2005). Evaluating relevance feedback algorithms for searching on small displays.
  54. (2000). Evaluation of eye gaze interaction,
  55. (1995). Eye gaze control of multimedia systems,
  56. (1995). Eye movement tracking using compressed video images, Vision Sciences and Its Applications;
  57. (1967). Eye Movements and Vision,
  58. (1998). Eye movements in reading and information processing: 20 years of research.
  59. (2004). Eye Tracking as a New Interface for Image Retrieval,
  60. (2003). Eye Tracking methodology: Theory and Practice,
  61. (1998). Eye-movement contingent display changes are not compromised by flicker and phosphor persistence.
  62. (2001). Eye-R, a glasses-mounted eye motion detection interface, CHI '01 extended abstracts on Human factors in computer systems,
  63. EyeLink 1000 Tracking Modes Mode Sampling Rate Sample Access Delay* Filter (Off/Normal/High) Noise** (RMS) Filter(Off/Normal/High) Pupil-CR Pupil Only† 1000
  64. (1982). Eyes at the interface.
  65. (2003). Eyetools Title: Techniques for facilitating use of eye tracking data (WO 2003/050658) Publication Date:
  66. (2002). Fast hands-free writing by gaze direction,
  67. (1980). Feature Integration Theory of Attention.
  68. (1994). From blobs to boundary edges: evidence for time and spatial scale dependent scene recognition.
  69. (2002). Gaze-Contingent Multi-Modality Displays of Multi-Layered Geographical Maps,
  70. (1993). Guided Search 2.0: The Upgrade.
  71. (2000). Identifying fixations and saccades in eyetracking protocols,
  72. (2003). Infrared Eye: prototype 2,
  73. (2002). Koninklijke Philips Electronics N.V. Title: System for Automatically Adjusting a Lens Power through Gaze Tracking (WO 2002/054132) Publication Date: 11
  74. (1999). Leica Microsystems AG Title: Eye Tracking System (WO 1999/065381) Publication Date: 23
  75. (1998). Locating Salient Object Features, British Machine Vision Conference.
  76. (2005). Managing Multimedia Content: A Technology Roadmap,"
  77. Measure image Sum of Squares df
  78. Measure: MEASURE_1 Source image thresh random Type III Sum of Squares df
  79. Measure: MEASURE_1 Source Threshold Ranking Type III Sum of Squares df
  80. Measure: MEASURE_1 Transformed Variable: Average Source Type III Sum of Squares df
  81. (2001). Ming Title: A Hybrid Tracking System (WO 2001/074231) Publication Date:
  82. (2002). Modeling the role of salience in the allocation of overt visual attention.
  83. (2002). Naoki Mukawa and Atsushi
  84. (1999). Natural scene statistics at the centre of gaze.
  85. (2006). On Ranking The Effectiveness of Searches,
  86. (2002). Opthalmic Inventions, LLC Title: Topography-Guided Opthalmic Ablation and Eye-Tracking (WO 2002/056789) Publication Date: 25
  87. Partial Eta Squared image Sphericity Assumed
  88. (1958). Perception and Communication,
  89. (2006). Perceptual Image Retrieval Using Eye Movements,
  90. Product Comparison Chart Quick Glance 1 Quick Glance 2B Quick Glance 2S Quick Glance 2SH Portability Desktop Only Desktop/Laptop Desktop/Laptop Desktop/Laptop Connection to PC
  91. (2003). Qinetiq Limited Title: Eye Tracking Systems (WO 2003/017203) Publication Date:
  92. (2005). Queen's University at Kingston Title: Method and Apparatus for Calibration-Free Eye Tracking (WO 2005/046465) Publication Date: 26
  93. (1995). Query by image and video content: The QBIC system.
  94. (1996). Query Formulation as an Information Retrieval Problem.
  95. (2000). Refixation frequency and memory mechanisms in visual search Curr.Biol.
  96. (2005). Report on a market study and demographics of user population.
  97. (2000). Sarel, Oded Title: A System and Method for Automated Self Measurement of Alertness, Equilibrium and Coordination and for Verification of the Identity of the Person Performing Tasks
  98. (2000). Scientific Generics Limited Title: Eye Tracking System (WO 2000/026713) - 220 -Publication Date: 11
  99. (2004). Seeing Machines Pty Ltd Title: Eye Tracking System and Method (WO 2004/088348) Publication Date:
  100. (2002). Segmentation using Expectation-Maximisation and its Application to Querying.
  101. (2001). Selective Activation of Visual Cortex Neurons by Fixational Eye Movements: Implications for Neural Coding.
  102. (2002). Sensomotoric Instruments GMBH Title: Multidimensional Eye Tracking and Position Measurement System (WO 2002/064031) Publication Date: 22
  103. (1985). Shifts in selective visual attention: towards the underlying neural circuitry,
  104. (1976). Short-term conceptual memory for pictures.
  105. (2000). Similarity shape retrieval in MARS.
  106. (2003). Smart Eye AB Title: Method for Image Analysis (WO 2003/003910) Publication Date: 16
  107. SMI’s iView X The iViewX Hi-Speed 1250 eye tracker samples at up to 1250Hz.
  108. Source Measure image random Type III Sum of Squares df
  109. Source Measure Type III Sum of Squares df
  110. (2002). Speed browser controlled by eye movements,
  111. (2001). Swisscom Mobile AG Title: Method and System for Video Conferences (WO 2001/084838) Publication Date: 08
  112. Technical Details Hi-Speed (Remote)
  113. (2000). The Bayesian image retrieval system, PicHunter: theory, implementation, and Psychophysical experiments.
  114. The F tests the effect of Mode. This test is based on the linearly independent pairwise comparisons among the estimated marginal means. Ranking Estimates 95% Confidence Interval Measure Ranking Mean Std. Error Lower Bound Upper Bound
  115. (1999). The GAZE Groupware System: Mediating Joint Attention in Mutiparty Communication and Collaboration.
  116. (1967). The gaze selects informative details within pictures,
  117. (1995). Theo Engell-Nielsen
  118. (2001). Title: Display System with Eye Tracking (WO 2001/009685) Publication Date:
  119. (2002). Title: Dynamic Depth-of-Field Emulation based on Eye-Tracking (WO 2002/029718) Publication Date:
  120. (2003). Title: Eye Tracking using Edge of Corneal Flap (WO 2003/022173) Publication Date: 20
  121. (1999). Title: Fiber Optic Eye-Tracking System (WO 1999/035961) Publication Date: 22
  122. (1998). Title: Foveated Image Coding System and Method for Image Bandwidth Reduction (WO 1998/033315) Publication Date:
  123. (1998). Title: Method and Apparatus for Assessing Visual Field (WO 1998/040781) - 222 -Publication Date:
  124. (2003). Title: Method and Apparatus for Gaze Responsive Text Presentation (WO 2003/019341) Publication Date: 06
  125. (2002). Title: Method and Apparatus for Precision Laser Surgery (WO 2002/032353) Publication Date:
  126. (2000). Title: System and Method for Controlling Host System Interface with User Point-of-Interest Data (WO 2000/016185) Publication Date: 23
  127. (2005). Title: Three Dimensional Acquisition and Visualization System for Personal Electronic Devices (WO 2005/091650) Publication Date:
  128. (2001). Title: Two Camera Off-Axis Eye Tracker (WO 2001/024688) Publication Date:
  129. (1997). To see or not to see: the need for attention to perceive changes in scenes.
  130. (2005). Tobii Technology AB Title: Arrangement, Method and Computer Program for Controlling a Computer Apparatus based on Eye-Tracking
  131. (2001). Using Human Observers' Eye Movements in Automatic Image Classifiers,
  132. (1998). Visual attention mediated by biased competition in extrastriate visual cortex.
  133. (2005). Visual correlates of fixation selection: effects of scale and time,
  134. (1998). Visual Querying by Colour Perceptive Regions.
  135. (2006). Visual saliency and semantic incongruency influence eye movements when inspecting pictures,
  136. (2001). Visual scene memory and the guidance of saccadic eye movements. Vision Res.
  137. (1992). Visual Search, Eye Movements and Display Units, Human Factors Report.
  138. (2005). Visual-Based Image Retrieval (VBIR) - A New Approach for Natural and Intuitive Image Retrieval.
  139. (1999). Visx Incorporated Title: Eye Tracking Device for Laser Eye Surgery using Corneal Margin Detection (WO 1999/018868) Publication Date:
  140. (2003). Xing Xie, Wei-Ying Ma, Hong-Jiang Zhang, He-Qin Zhou

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.