A USER’S COGNITIVE WORKLOAD PERSPECTIVE IN NEGOTIATION SUPPORT SYSTEMS: AN EYE-TRACKING EXPERIMENT

Replying to several research calls, I report promising results from an initial experiment which com-pares different negotiation support system approaches concerning their potential to reduce a user’s cognitive workload. Using a novel laboratory-based non-intrusive objective measurement technique which derives the user’s cognitive workload from pupillary responses and eye-movements, I experi-mentally evaluated a standard, a chat-based, and an argumentation-based negotiation support system and found that a higher assistance level of negotiation support systems actually leads to a lower user’s cognitive workload. In more detail, I found that an argumentation-based system which fully automates the generation of the user’s arguments significantly decreases the user’s cognitive workload compared to a standard system. In addition I found that a negotiation support system implementing an additional chat function significantly causes higher cognitive workload for users compared to a standard system

Mobile pupillometry in manual assembly : a pilot study exploring the wearability and external validity of a renowned mental workload lab measure

Human operators in the upcoming Industry 4.0 workplace will face accelerating job demands such as elevated cognitive complexity. Unobtrusive objective measures of mental workload (MWL) are therefore in high demand as indicated by both theory and practice. This pilot study explored the wearability and external validity of pupillometry, a MWL measure robustly validated in laboratory settings and now deployable in work settings demanding operator mobility. In an ecologically valid work environment, 21 participants performed two manual assemblies - one of low and one of high complexity - while wearing eye-tracking glasses for pupil size measurement. Results revealed that the device was perceived as fairly wearable in terms of physical and mental comfort. In terms of validity, no significant differences in mean pupil size were found between the assemblies even though subjective mental workload differed significantly. Exploratory analyses on the pupil size when attending to the assembly instructions only, were inconclusive. The present work suggests that current lab-based procedures might not be adequate yet for in-the-field mobile pupillometry. From a broader perspective, these findings also invite a more nuanced view on the current validity of lab-validated physiological MWL-measures when applied in real-life settings. We therefore conclude with some key insights for future development of mobile pupillometry

Arousal‐based pupil modulation is dictated by luminance

Pupillometry has become a standard measure for assessing arousal state. However, environmental factors such as luminance, a primary dictator of pupillary responses, often vary across studies. To what degree does luminance interact with arousal-driven pupillary changes? Here, we parametrically assessed luminance-driven pupillary responses across a wide-range of luminances, while concurrently manipulating cognitive arousal using auditory math problems of varying difficulty. At the group-level, our results revealed that the modulatory effect of cognitive arousal on pupil size interacts multiplicatively with luminance, with the largest effects occurring at low and mid-luminances. However, at the level of individuals, there were qualitatively distinct individual differences in the modulatory effect of cognitive arousal on luminance-driven pupillary responses. Our findings suggest that pupillometry as a measure for assessing arousal requires more careful consideration: there are ranges of luminance levels that are more ideal in observing pupillary differences between arousal conditions than others.https://www.nature.com/articles/s41598-022-05280-1.pdfPublished versio

Psychophysiological indices of recognition memory

It has recently been found that during recognition memory tests participants’ pupils dilate more when they view old items compared to novel items. This thesis sought to replicate this novel ‘‘Pupil Old/New Effect’’ (PONE) and to determine its relationship to implicit and explicit mnemonic processes, the veracity of participants’ responses, and the analogous Event-Related Potential (ERP) old/new effect. Across 9 experiments, pupil-size was measured with a video-based eye-tracker during a variety of recognition tasks, and, in the case of Experiment 8, with concurrent Electroencephalography (EEG). The main findings of this thesis are that: - the PONE occurs in a standard explicit test of recognition memory but not in “implicit” tests of either perceptual fluency or artificial grammar learning; - the PONE is present even when participants are asked to give false behavioural answers in a malingering task, or are asked not to respond at all; - the PONE is present when attention is divided both at learning and during recognition; - the PONE is accompanied by a posterior ERP old/new effect; - the PONE does not occur when participants are asked to read previously encountered words without making a recognition decision; - the PONE does not occur if participants preload an “old/new” response; - the PONE is not enhanced by repetition during learning. These findings are discussed in the context of current models of recognition memory and other psychophysiological indices of mnemonic processes. It is argued that together these findings suggest that the increase in pupil-size which occurs when participants encounter previously studied items is not under conscious control and may reflect primarily recollective processes associated with recognition memory

A Review of Psychophysiological Measures to Assess Cognitive States in Real-World Driving

As driving functions become increasingly automated, motorists run the risk of becoming cognitively removed from the driving process. Psychophysiological measures may provide added value not captured through behavioral or self-report measures alone. This paper provides a selective review of the psychophysiological measures that can be utilized to assess cognitive states in real-world driving environments. First, the importance of psychophysiological measures within the context of traffic safety is discussed. Next, the most commonly used physiology-based indices of cognitive states are considered as potential candidates relevant for driving research. These include: electroencephalography and event-related potentials, optical imaging, heart rate and heart rate variability, blood pressure, skin conductance, electromyography, thermal imaging, and pupillometry. For each of these measures, an overview is provided, followed by a discussion of the methods for measuring it in a driving context. Drawing from recent empirical driving and psychophysiology research, the relative strengths and limitations of each measure are discussed to highlight each measures' unique value. Challenges and recommendations for valid and reliable quantification from lab to (less predictable) real-world driving settings are considered. Finally, we discuss measures that may be better candidates for a near real-time assessment of motorists' cognitive states that can be utilized in applied settings outside the lab. This review synthesizes the literature on in-vehicle psychophysiological measures to advance the development of effective human-machine driving interfaces and driver support systems

Cognitive theories and forensic applications: the pupillary correlates of familiar and unfamiliar face processing

This thesis used pupillometry to investigate whether pupils respond differently to faces that differ in familiarity. We aimed to see whether pupillometry measures cognitive processes involved in face processing, and whether it could be applied forensically. We started by evaluating three explanations for pupillary changes that occur when processing faces. The first was cognitive load (mental effort), because faces that have only been seen briefly are more difficult to recognise than well-known faces. The second was cognitive engagement (interest), because faces contain socially-important information. The third was memory strength (forensically applicable), as eyewitnesses have to recall a perpetrator’s face in an attempt to identify them if they appear in a lineup. While pupillary responses reflected cognitive engagement to some extent, cognitive load best accounted for decreasing pupil sizes when learning new faces, and memory strength explained the pupillary changes seen in lineups. The theories all had some influence on pupil sizes, but their influence varied according to context, saliency, and the task at hand. Then we investigated whether pupillometry measured implicit recognition of a perpetrator in a lineup, and found that it did. Pupil sizes reflected memory strength in participants who believed their memory to be strong: there were differences in pupil sizes (between looking at the perpetrator and the distractors) in participants who identified him, but not in those who did not. The pupillary responses of participants who ‘guessed’ indicated that they were indeed guessing. There were no pupillary changes when the perpetrator was not in the lineup, even when participants misidentified a distractor. We concluded that pupillary responses are independent of explicit identification responses, and could be used forensically to support traditional measures of eyewitness identification and credibility

Les yeux ont des oreilles : examen de la pupillométrie en tant qu'indice psychophysiologique de la capture attentionnelle auditive

Tableau d'honneur de la Faculté des études supérieures et postdoctorales, 2018-2019.La présence de sons non pertinents est reconnue pour affecter le fonctionnement cognitif. Plus précisément, la présentation d’un son qui dévie du contexte auditif a le potentiel d’affecter la performance à une tâche réalisée simultanément. Plusieurs études s’intéressant à l’impact délétère des sons déviants sur la performance à une tâche montrent qu’il découle de la réorientation de l’attention déclenchée lorsqu’un son déviant est détecté par l’organisme. Cette capture attentionnelle (ou réponse d’orientation) est également reconnue pour engendrer plusieurs réponses physiologiques associées à l’état d’alerte. Ces réponses physiologiques, sous certaines conditions, sont considérées comme des indices psychophysiologiques de la capture attentionnelle auditive permettant de démontrer l’occurrence d’une réorientation de l’attention de la tâche en cours vers le son déviant. De récentes études suggèrent qu’il existe un lien entre ces indices et la réponse pupillaire, soit l’augmentation rapide du diamètre pupillaire. Quelques études ont tenté d’évaluer si cette réponse pouvait remplir les critères d’un indice valide de la réponse d’orientation; cependant, les résultats de ces études sont parfois contradictoires ou incomplets. La présente thèse propose donc une évaluation systématique de l’utilisation de la réponse pupillaire à titre d’indice psychophysiologique de la capture attentionnelle auditive. Les résultats de la première étude montrent que la réponse pupillaire respecte les critères d’un index valide de la réponse d’orientation, supportant ainsi la validité de cet index. L’Étude 2 montre que cet indice peut être utilisé dans des contextes dans lesquels les sujets effectuent simultanément à l’écoute des sons une tâche visuelle provoquant des changements systématiques de luminosité ou des mouvements oculaires qui affectent la taille de la pupille. Ces expériences appuient ainsi l’utilisabilité de cet index. Enfin, la dernière étude permet d’établir l’utilité de ce proxy puisque ce dernier permet de distinguer l’origine de deux phénomènes de distraction auditive différents. Dans l’ensemble, les résultats de la thèse révèlent que la réponse pupillaire représente un indice psychophysiologique adéquat qui pourrait être intégré aux études sur la distraction auditive ou dans des contextes plus appliqués où les mesures de la capture attentionnelle auditive peuvent s’avérer pertinentes.The presence of task-irrelevant sound is known to impede cognitive functioning. More precisely, presenting a sound that deviates from the auditory background has been shown to disrupt performance on an ongoing task. Several studies regarding performance disruption by deviant sounds showed that this effect originates from the reorientation of attention triggered when the organism has detected the deviant sound. Such attentional capture (or orientation response) also elicits many physiological responses related to one’s state of alertness. These physiological responses, under certain conditions, are considered as psychophysiological indices of auditory attentional capture, permitting to show that attention has been reoriented from an ongoing task toward the deviant auditory stimulus. Recent work suggested a relationship between these indices and the pupillary dilation response, that is a rapid increase in the pupil size. A few studies have attempted to assess whether the pupillary dilation response could respect the criteria for an index to be considered as a proxy for attentional capture; yet, results of these studies either lack consistency or are incomplete. Hence, the current thesis proposes a systematic assessment of using the pupillary dilation response as a psychophysiological proxy for the auditory attentional capture. Results of the first study suggest that the pupillary dilation response respects criteria of a valid auditory attentional capture index, thus supporting the validity of this index. Study 2 shows that this index can be used in contexts in which participants are exposed to irrelevant deviant sound while concurrently performing a task inducing luminance changes or ocular movements, both influencing the pupil size. These experiments hence supported the usability of this index. The last study allows establishing the utility of this proxy since it can be used to dissociate the origin of two different auditory distraction phenomena. Overall, results of the current thesis reveal that the pupillary dilation response is a psychophysiological index that could be added to several auditory distraction paradigms or in applied contexts in which the occurrence of attention reorientation toward a sound may be relevant to assess

High Working Memory Load Impairs Language Processing during a Simulated Piloting Task: An ERP and Pupillometry Study

Given the important amount of visual and auditory linguistic information that pilots have to process, operating an aircraft generates a high working-memory load (WML). In this context, the ability to focus attention on relevant information and to remain responsive to concurrent stimuli might be altered. Consequently, understanding the effects of WML on the processing of both linguistic targets and distractors is of particular interest in the study of pilot performance. In the present work, participants performed a simplified piloting task in which they had to follow one of three colored aircraft, according to specific written instructions (i.e., the written word for the color corresponding to the color of one of the aircraft) and to ignore either congruent or incongruent concurrent auditory distractors (i.e., a spoken name of color). The WML was manipulated with an n-back sub-task. Participants were instructed to apply the current written instruction in the low WML condition, and the 2-back written instruction in the high WML condition. Results revealed a major effect of WML at behavioral (i.e., decline of piloting performance), electrophysiological, and autonomic levels (i.e., greater pupil diameter). Increased WML consumed resources that could not be allocated to the processing of the linguistic stimuli, as indexed by lower P300/P600 amplitudes. Also, significantly, lower P600 responses were measured in incongruent vs. congruent trials in the low WML condition, showing a higher difficulty reorienting attention toward the written instruction, but this effect was canceled in the high WML condition. This suppression of interference in the high load condition is in line with the engagement/distraction trade-off model. We propose that P300/P600 components could be reliable indicators of WML and that they allow an estimation of its impact on the processing of linguistic stimuli