Dual Eye-Tracking Methods for the Study of Remote Collaborative Problem Solving

Applied eye-tracking has been extensively used for the study of psychological processes. More recently, some researchers have used this technique to study the interaction between people by tracking and analyzing eye-movements of two persons synchronously. However, this is generally accomplished by observing people in simple controlled settings. In this thesis, we use a similar methodology, dual eye-tracking, to study people in more natural, semantically richer, tasks with the aim of identifying dual eye-movement patterns which reflect collaborative processes. Eye-tracking, and more globally eye-movements analyses, is a complex domain involving several methodological issues, which have not yet been satisfactorily solved. This work is also an attempt to offer solutions to several of these issues. The first part of this thesis is dedicated to the improvement of the methodology of eye-tracking data analysis. We present several developments pertaining to the general methodology of eye-tracking. More specifically, we identify problems and offer solutions to the following aspects: fixation identification, systematic position errors correction and hit detection. We also tackle more specific questions concerning the method dual eye-tracking. We present issues that arise in dual eye-tracking data collection and analysis and propose some solutions. On the technical side, we deal with the question of the synchronous recording of two streams of eye-movements and on the analytical side, we extend gaze cross-recurrence, a measure of eye-movements coupling, to complex realistic collaborative tasks. The second part is devoted to experimental studies of collaboration through the use of dual eye-tracking methods. We first present four exploratory studies which allowed us to set up the stage by identifying interesting phenomena and experimental difficulties. The main results of these experiments revolve around the relationship between gaze and speech. In these respects, we extended some results found in the literature to more natural settings and we developed a computational model to make actual predictions about dialogue and visual references. Finally, the main study of this thesis is about computer program understanding. This study is composed of two experiments: a solo programming experiment, from which we identified gaze patterns of a single programmer and a pair-programming task in which we explored how gaze patterns during program comprehension are affected by collaboration

This is it ! : Indicating and looking in collaborative work at distance

Little is known of the interplay between deixis and eye movements in remote collaboration. This paper presents quantitative results from an experiment where participant pairs had to collaborate at a distance using chat tools that differed in the way messages could be enriched with spatial information from the map in the shared workspace. We studied how the availability of what we defined as an Explicit Referencing mechanism (ER) affected the coordination of the eye movements of the participants. The manipulation of the availability of ER did not produce any significant difference on the gaze coupling. However, we found a primary relation between the pairs recurrence of eye movements and their task performance. Implications for design are discussed

Unravelling cross-recurrence: coupling across timescales

We present an extension of cross-recurrence analysis for dual gaze analysis which is suited for complex situations where for instance the objects of interest are not all visible at all times or when stimulus exploration is not homogeneous. The typicical situation is a visual stimulus that is scrolled or that is explored sequentially. We use a recurrence simulation to illustrate how to measure the actual coupling between behavior streams without biases introduced by the complexity of the situation. Our method takes into account underlying random baselines to compute an unbiased version of the coupling

Understanding Collaborative Program Comprehension: Interlacing Gaze and Dialogues

We study the interaction of the participants in a pair program comprehension task across different time scales in a dual eye-tracking setup. We identify four layers of interaction episodes at different time scales. Each layer spans across the whole interaction. The present study concerns the relationship between different layers at different time scales. The first and third layers are based on the utterances of the participants while the second and fourth layers are based on participants' gaze

Collaboration and abstract representations: towards predictive models based on raw speech and eye-tracking data

This study aims to explore the possibility of using machine learning techniques to build predictive models of performance in collaborative induction tasks. More specifically, we explored how signal-level data, like eye-gaze data and raw speech may be used to build such models. The results show that such low level features have effectively some potential to predict performance in such tasks. Implications for future applications design are shortly discussed

Gaze Evidence for Different Activities in Program Understanding

We present an empirical study that illustrates the potential of dual eye-tracking to detect successful understanding and social processes during pair programming. The gaze of forty pairs of programmers was recorded during a program understanding task. An analysis of the gaze transitions between structural elements of the code, declarations of identifiers and expressions shows that pairs with better understanding do less systematic execution of the code and more “tracing” of the data flow by alternating between identifiers and expressions. Interaction consists of moments where partners’ attention converges on the same same part of the code and moments where it diverges. Moments of convergence are accompanied by more systematic execution of the code and less transitions among identifiers and expressions

Effects of sharing text selections on gaze recurrence and interaction quality in a pair programming task

We present experimental findings from a dual eye-tracking study that illustrate the effect of different ways of sharing se- lection in remote pair-programming scenario. Forty pairs of engineering students had to complete several program under- standing tasks. The cross-recurrence of gaze, i.e. how much programmers look at the same spot within a short time span, varies with speech and selection. Gaze recurrence is higher during spoken episodes compared to silent episodes. Gaze recurrence is highest during spoken selections, when the se- lection is broadcast to both collaborators either as a dual se- lection (the programmers can each select their own portion of text) or a shared selection (programmers share a single selection). Broadcast selections serve as indexing sites for the selector as they attract non-selector’s gaze shortly af- ter they become visible. Spoken selections are more precise than silent selections which suggest their deictic function

Facilitating peer knowledge modeling: Effects of a knowledge awareness tool on collaborative learning outcomes and processes

We report an empirical study where we investigated the effects, on the collaborative outcomes and processes, of a cognition-related awareness tool providing learners with cues about their peer’s level of prior knowledge. Sixty-four university students participated in a remote computer-mediated dyadic learning scenario. Co-learners were provided (or not) with a visual representation of their peer’s level of prior knowledge through what we refer to as a knowledge awareness tool (KAT). The results show that, providing co-learners with objective cues about the level of their peer’s prior knowledge positively impacts learning outcomes. In addition, this effect seems to be mediated by the fact that co-learners provided with these objective cues become more accurate in estimating their partner’s knowledge – accuracy that predicts higher outcomes. Analyses on the process level of the verbal interactions indicate that the KAT seems to sensitize co-learners to the fragile nature of their partner’s as well as their own prior knowledge. The beneficial effect of the KAT seems to mainly rely on this induction of epistemic uncertainty that implicitly triggers compensation socio-cognitive strategies; strategies that appear to be beneficial to the learning process

Using dual eye-tracking to unveil coordination and expertise in collaborative Tetris

The use of dual eye-tracking is investigated in a collabora- tive game setting. The automatic collection of information about partner’s gaze will eventually serve to build adaptive interfaces. Following this agenda, and in order to identify stable gaze patterns, we investigate the impact of social and task related context upon individual gaze and action during a collaborative Tetris game. Results show that experts as well as novices adapt their playing style when interacting in mixed ability pairs. We also present machine learning results about the prediction of player’s social context