Scan path visualization and comparison using visual aggregation techniques

We demonstrate the use of different visual aggregation techniques to obtain non-cluttered visual representations of scanpaths. First, fixation points are clustered using the mean-shift algorithm. Second, saccades are aggregated using the Attribute-Driven Edge Bundling (ADEB) algorithm that handles a saccades direction, onset timestamp, magnitude or their combination for the edge compatibility criterion. Flow direction maps, computed during bundling, can be visualized separately (vertical or horizontal components) or as a single image using the Oriented Line Integral Convolution (OLIC) algorithm. Furthermore, cosine similarity between two flow direction maps provides a similarity map to compare two scanpaths. Last, we provide examples of basic patterns, visual search task, and art perception. Used together, these techniques provide valuable insights about scanpath exploration and informative illustrations of the eye movement data

Study of pupil diameter and eye movements to enhance flight safety. Etude de diamètre pupillaire et de mouvements oculaires pour la sécurité aérienne

L'analyse d'événements aériens révèle que la plupart des accidents aéronautiques ont pour origine une surveillance inadaptée de paramètres de vol induite par une vigilance réduite, le stress ou une charge de travail importante. Une solution prometteuse pour améliorer la sécurité aérienne est d'étudier le regard des pilotes. La pupille est un bon indicateur de l'état attentionnel/cognitif tandis que les mouvements oculaires révèlent des stratégies de prises d'information. La question posée dans ce manuscrit est d'évaluer l'apport de l'oculométrie pour la sécurité aérienne par les contributions suivantes : 1-2) Les deux premières études de ce doctorat ont démontré que les effets d'interaction entre la luminance et la charge cognitive sur la réaction pupillaire. La composante pupillaire impactée dépend de la nature de la charge - soutenue ou transitoire. 3) Un cadre mathématique développé fournit un moyen d'illustration de schémas visuels pour l'analyse qualitative. Ce cadre ouvre également la voie à de nouvelles méthodes pour comparer quantitativement ces schémas visuels. 4) Une technique originale d'analyse de fixations et de construction d'un ratio "exploration-exploitation" est proposée et est appliquée dans deux cas d'études en simulateur de vol. 5) Enfin, on propose un cadre théorique d'intégration de l'oculométrie dans les cockpits. Ce cadre comporte quatre étapes présentées dans, à la fois, l'ordre chronologique de l'intégration et la complexité technique de réalisation.Most aviation accidents include failures in monitoring or decision-making which are hampered by arousal, stress or high workload. One promising avenue to further enhance the flight safety is looking into the pilots' eyes. The pupil is a good indicator of cognitive/attentional states while eye movements reveal monitoring strategies. This thesis reflected upon the application of eye tracking in aviation with following contributions: 1-2) The two pupil experiments revealed that the luminance impacts the cognitive pupil reaction. Depending on the nature of the cognitive load - sustained or transient - the corresponding pupillary component would be impacted. The same amount of cognitive load under dimmer luminance condition would elicit larger tonic pupil diameter in a sustained load paradigm and larger phasic pupil response in a transient load paradigm. 3) We designed a novel mathematical framework and method that provide comprehensive illustrations of scanpaths for qualitative analysis. This framework also makes a lane for new methods of scanpaths comparison. 4) The developed technique of analysis of fixations and construction of "explore-exploit" ratio is presented and verifed on the data from two experiments in flight simulators. 5) Eventually, we proposed a framework of eye tracking integration into the cockpits. It contains four stages presented in both chronological order of its integration and technical complexity

Uncertainty visualization of gaze estimation to support operator-controlled calibration

      In this paper, we investigate how visualization assets can support the qualitative evaluation of gaze estimation uncertainty. Although eye tracking data are commonly available, little has been done to visually investigate the uncertainty of recorded gaze information. This paper tries to fill this gap by using innovative uncertainty computation and visualization. Given a gaze processing pipeline, we estimate the location of this gaze position in the world camera. To do so we developed our own gaze data processing which give us access to every stage of the data transformation and thus the uncertainty computation. To validate our gaze estimation pipeline, we designed an experiment with 12 participants and showed that the correction methods we proposed reduced the Mean Angular Error by about 1.32 cm, aggregating all 12 participants’ results. The Mean Angular Error is 0.25° (SD=0.15°) after correction of the estimated gaze. Next, to support the qualitative assessment of this data, we provide a map which codes the actual uncertainty in the user point of view.

Intelligent cockpit: eye tracking integration to enhance the pilot-aircraft interaction

In this research, we use eye tracking to monitor the attentional behavior of pilots in the cockpit. We built a cockpit monitoring database that serves as a reference for real-time assessment of the pilot’s monitoring strategies, based on numerous flight simulator sessions with eye-tracking recordings. Eye tracking may also be employed as a passive input for assistive system, future studies will also explore the possibility to adapt the notifications’ modality using gaze

The impact of luminance on tonic and phasic pupillary responses to sustained cognitive load

Pupillary reactions independent of light conditions have been linked to cognition for a long time. However, the light conditions can impact the cognitive pupillary reaction. Previous studies underlined the impact of luminance on pupillary reaction, but it is still unclear how luminance modulates the sustained and transient components of pupillary reaction – tonic pupil diameter and phasic pupil response. In the present study, we investigated the impact of the luminance on these two components under sustained cognitive load. Fourteen participants performed a novel working memory task combining mathematical computations with a classic n-back task. We studied both tonic pupil diameter and phasic pupil response under low (1-back) and high (2-back) working memory load and two luminance levels (gray and white). We found that the impact of working memory load on the tonic pupil diameter was modulated by the level of luminance, the increase in tonic pupil diameter with the load being larger under lower luminance. In contrast, the smaller phasic pupil response found under high load remained unaffected by luminance. These results showed that luminance impacts the cognitive pupillary reaction – tonic pupil diameter (phasic pupil response) being modulated under sustained (respectively, transient) cognitive load. These findings also support the relationship between the locus-coeruleus system, presumably functioning in two firing modes – tonic and phasic – and the pupil diameter. We suggest that the tonic pupil diameter tracks the tonic activity of the locus-coeruleus while phasic pupil response reflects its phasic activity. Besides, the designed novel cognitive paradigm allows the simultaneous manipulation of sustained and transient components of the cognitive load and is useful for dissociating the effects on the tonic pupil diameter and phasic pupil response

Integrating electrodes to headsets for human-system interaction and psycho-physiological monitoring

Research in human monitoring led to development of powerful tools for users-systems communication via recording electrophysiological data and sending them to the computer system. Eye and gaze tracking are standing as important methods for a broad range of human monitoring applications such as in neuroscience, psychology, industrial engineering, aeronautics, military, and medical expertise. Nowadays, the most used eye-tracking technique is video-based tracking based on infrared illumination. However, the tools using this technique present a certain number of disadvantages. Notably, for the head-mounted tools, such systems obstruct the visual field and therefore are not suitable for integration in real operational environments. An alternative technique is the electro-oculography (EOG) which consists in measuring the standing potential between the front and back of the eyes. This potential increases when the cornea approaches an electrode and it decreases when the cornea moves in the opposite direction. EOG represents one of the easiest methods to estimate eye movements by using low-cost low-energy consumption devices without obstructing the visual field nor handling infrared light. This technique is particularly convenient for the head-mounted peripherals such as audio or virtual reality headsets. Typical features of signal processing, such as the mean of fixation duration, saccadic rates, and blinks are explored in various environments including air traffic control, pilots and co-pilots of civil and military aircraft. Interestingly, the EOG, as well as electroencephalography (EEG) features were correlated to drivers’ and pilots’ workload, drowsiness and fatigue

Improving eye-tracking calibration accuracy using symbolic regression

Eye tracking systems have recently experienced a diversity of novel calibration procedures, including smooth pursuit and vestibulo-ocular reflex based calibrations. These approaches allowed collecting more data compared to the standard 9-point calibration. However, the computation of the mapping function which provides planar gaze positions from pupil features given as input is mostly based on polynomial regressions, and little work has investigated alternative approaches. This paper fills this gap by providing a new calibration computation method based on symbolic regression. Instead of making prior assumptions on the polynomial transfer function between input and output records, symbolic regression seeks an optimal model among different types of functions and their combinations. This approach offers an interesting perspective in terms of flexibility and accuracy. Therefore, we designed two experiments in which we collected ground truth data to compare vestibulo-ocular and smooth pursuit calibrations based on symbolic regression, both using a marker or a finger as a target, resulting in four different calibrations. As a result, we improved calibration accuracy by more than 30%, with reasonable extra computation time