Untersuchung von Relevanzeigenschaften in einem kontrollierten Eytracking-Experiment

In diesem Artikel wird ein Eyetracking-Experiment beschrieben, bei dem untersucht wurde, wann und auf Basis welcher Informationen Relevanzentscheidungen bei der themenbezogenen Dokumentenbewertung fallen und welche Faktoren auf die Relevanzentscheidung einwirken. Nach einer kurzen Einführung werden einschlägige Studien aufgeführt, in denen Blickverfolgung (Eye tracking) als Untersuchungsmethode für Interaktionsverhalten mit Ergebnislisten (information seeking behaviour) verwendet wurde. Nutzerverhalten wird hierbei vor allem durch unterschiedliche Aufgaben-Typen, durch unterschiedlich dargestellte Informationen und durch den Rang eines Ergebnisses auf der Trefferliste beeinflusst. Durch Eyetracking-Untersuchungen lassen sich Nutzer außerdem in verschiedene Klassen von Bewertungs- und Lesetypen einordnen. Diese Informationen können als implizites Feedback genutzt werden, um so die Suche zu personalisieren und um die Relevanz von Suchergebnissen ohne aktives Zutun des Users zu erhöhen. In einem explorativen Eyetracking-Experiment mit zwölf Studenten der Hochschule Darmstadt werden anhand der Länge der Gesamtbewertung, Anzahl der Fixationen, Anzahl der besuchten Metadatenelemente und Länge des Scanpfades zwei typische Bewertungstypen identifiziert. Das Metadatenfeld Abstract wird im Experiment zuverlässig als wichtigste Dokumenteigenschaft für die Zuteilung von Relevanz ermittelt. (Autorenreferat)The article describes an eyetracking experiment which examines relevance judgements within the context of subject-related document assessments. We analyze in the study on what information the judgements of our test persons are based on and which document specific properties influence the relevance decisions. In the state of the art the authors present relevant studies that use eyetracking methodology as a research method to investigate information seeking behaviour models. The three factors that particularly influence user behaviour are: different task types, search results presentation, and document ranking. Furthermore, the results of these eyetracking studies help us to classify users into typical evaluation and reading types. This information can then be used as implicit feedback to personalize the search. Relevance of search results could thus be improved without any further involvement by the users. In an exploratory eyetracking experiment with twelve students from the University of Applied Sciences in Darmstadt, we were able to identify two typical evaluation types, based on total length of the evaluation, number of fixations, number of visited metadata elements and length of the scan path. This experiment shows that the metadata field abstract is clearly the most important document property to assign topical relevance to scientific articles. (author's abstract

A Qualitative Look at Eye-tracking for Implicit Relevance Feedback

Abstract. Our goal in this study was to explore the potentials of extracting features from eye-tracking data that have the potential to improve performance in implicit relevance feedback. We view this type of data as an example of the searcher ’ immediate context and as containing useful clues of the indications of the interaction between the searcher and the IR system. In particular, we explored if we could qualitatively identify features have potential to improve performance in implicit relevance feedback, and how such features correlate with document elements assessed as relevant or non-relevant. The results point to so-called thorough reading as one of the most promising features for identifying relevant information as input for implicit relevance feedback – in particular when it is related to the total time the searcher has looked an element

Exploring the applicability of implicit relevance measures in varying reading speed for adaptive I.R. systems

This thesis goes further in the study of implicit indicators used to infer interest in documents for information retrieval tasks. We study the behavior of two different categories of implicit indicators: fixation-derived features (number of fixations, average time of fixations, regression ratio, length of forward saccades), and physiology (pupil dilation, electrodermal activity). Based on the limited number of participants at our disposal we study how these measures react when addressing documents at three different reading rates. Most of the fixation-derived features are reported to differ significantly when reading at different speeds. Furthermore, the ability of pupil size and electrodermal activity to indicate perceived relevance is found intrinsically dependent on speed of reading. That is, when users read at comfortable reading speed, these measures are found to be able to correctly discriminate relevance judgments, but fail when increasing the addressed speed of reading. Therefore, the outcomes of this thesis strongly suggest to take into account reading speed when designing highly adaptive information retrieval systems

Exploring the applicability of implicit relevance measures in varying reading speed for adaptive I.R. systems

Projecte realitzat en el marc d’un programa de mobilitat amb la University of Helsinki. Faculty of Science. Department of Computer ScienceThis thesis goes further in the study of implicit indicators used to infer interest in documents for information retrieval tasks. We study the behavior of two different categories of implicit indicators: fixation-derived features and physiology (pupil size, electrodermal activity). Based on the limited number of participants at our disposal we study how these measures react when addressing documents at three different reading rates. Most of the fixation-derived features are reported to differ significantly when reading at different speeds. Furthermore, the ability of pupil size and electrodermal activity to indicate perceived relevance is found intrinsically dependent on speed of reading. That is, when users read at comfortable reading speed, these measures are found to be able to correctly discriminate relevance judgments, but fail when increasing the addressed speed of reading. Therefore, the outcomes of this thesis strongly suggest to take into account reading speed when designing highly adaptive information retrieval systems

Interactive video retrieval using implicit user feedback.

PhDIn the recent years, the rapid development of digital technologies and the low cost of recording media have led to a great increase in the availability of multimedia content worldwide. This availability places the demand for the development of advanced search engines. Traditionally, manual annotation of video was one of the usual practices to support retrieval. However, the vast amounts of multimedia content make such practices very expensive in terms of human effort. At the same time, the availability of low cost wearable sensors delivers a plethora of user-machine interaction data. Therefore, there is an important challenge of exploiting implicit user feedback (such as user navigation patterns and eye movements) during interactive multimedia retrieval sessions with a view to improving video search engines. In this thesis, we focus on automatically annotating video content by exploiting aggregated implicit feedback of past users expressed as click-through data and gaze movements. Towards this goal, we have conducted interactive video retrieval experiments, in order to collect click-through and eye movement data in not strictly controlled environments. First, we generate semantic relations between the multimedia items by proposing a graph representation of aggregated past interaction data and exploit them to generate recommendations, as well as to improve content-based search. Then, we investigate the role of user gaze movements in interactive video retrieval and propose a methodology for inferring user interest by employing support vector machines and gaze movement-based features. Finally, we propose an automatic video annotation framework, which combines query clustering into topics by constructing gaze movement-driven random forests and temporally enhanced dominant sets, as well as video shot classification for predicting the relevance of viewed items with respect to a topic. The results show that exploiting heterogeneous implicit feedback from past users is of added value for future users of interactive video retrieval systems