Examination of Driver Visual and Cognitive Responses to Billboard Elicited Passive Distraction Using Eye-Fixation Related Potential

Distractions external to a vehicle contribute to visual attention diversion that may cause traffic accidents. As a low-cost and efficient advertising solution, billboards are widely installed on side of the road, especially the motorway. However, the effect of billboards on driver distraction, eye gaze, and cognition has not been fully investigated. This study utilises a customised driving simulator and synchronised electroencephalography (EEG) and eye tracking system to investigate the cognitive processes relating to the processing of driver visual information. A distinction is made between eye gaze fixations relating to stimuli that assist driving and others that may be a source of distraction. The study compares the driver’s cognitive responses to fixations on billboards with fixations on the vehicle dashboard. The measured eye-fixation related potential (EFRP) shows that the P1 components are similar; however, the subsequent N1 and P2 components differ. In addition, an EEG motor response is observed when the driver makes an adjustment of driving speed when prompted by speed limit signs. The experimental results demonstrate that the proposed measurement system is a valid tool in assessing driver cognition and suggests the cognitive level of engagement to the billboard is likely to be a precursor to driver distraction. The experimental results are compared with the human information processing model found in the literature

Decision-making in information seeking on texts: an eye-fixation-related potentials investigation

Reading on a web page is known to be not linear and people need to make fast decisions about whether they have to stop or not reading. In such context, reading, and decision-making processes are intertwined and this experiment attempts to separate them through electrophysiological patterns provided by the Eye-Fixation-Related Potentials technique (EFRPs). We conducted an experiment in which EFRPs were recorded while participants read blocks of text that were semantically highly related, moderately related, and unrelated to a given goal. Participants had to decide as fast as possible whether the text was related or not to the semantic goal given at a prior stage. Decision making (stopping information search) may occur when the paragraph is highly related to the goal (positive decision) or when it is unrelated to the goal (negative decision). EFRPs were analyzed on and around typical eye fixations: either on words belonging to the goal (target), subjected to a high rate of positive decisions, or on low frequency unrelated words (incongruent), subjected to a high rate of negative decisions. In both cases, we found EFRPs specific patterns (amplitude peaking between 51 to 120 ms after fixation onset) spreading out on the next words following the goal word and the second fixation after an incongruent word, in parietal and occipital areas. We interpreted these results as delayed late components (P3b and N400), reflecting the decision to stop information searching. Indeed, we show a clear spill-over effect showing that the effect on word N spread out on word N + 1 and N + 2

EEG & Eye Tracking user experiments for spatial memory task on maps

The aim of this research is to evaluate the use of ET and EEG for studying the cognitive processes of expert and novice map users and to explore these processes by comparing two types of spatial memory experiments through cognitive load measurements. The first experiment consisted of single trials and participants were instructed to study a map stimulus without any time constraints in order to draw a sketch map afterwards. According to the ET metrics (i.e., average fixation duration and the number of fixations per second), no statistically significant differences emerged between experts and novices. A similar result was also obtained with EEG Frontal Alpha Asymmetry calculations. On the contrary, in terms of alpha power across all electrodes, novices exhibited significantly lower alpha power, indicating a higher cognitive load. In the second experiment, a larger number of stimuli were used to study the effect of task difficulty. The same ET metrics used in the first experiment indicated that the difference between these user groups was not statistically significant. The cognitive load was also extracted using EEG event-related spectral power changes at alpha and theta frequency bands. Preliminary data exploration mostly suggested an increase in theta power and a decrease in alpha power

Active visual search in non-stationary scenes: coping with temporal variability and uncertainty

Objective. State-of-the-art experiments for studying neural processes underlying visual cognition often constrain sensory inputs (e.g., static images) and our behavior (e.g., fixed eye-gaze, long eye fixations), isolating or simplifying the interaction of neural processes. Motivated by the non-stationarity of our natural visual environment, we investigated the electroencephalography (EEG) correlates of visual recognition while participants overtly performed visual search in non-stationary scenes. We hypothesized that visual effects (such as those typically used in human–computer interfaces) may increase temporal uncertainty (with reference to fixation onset) of cognition-related EEG activity in an active search task and therefore require novel techniques for single-trial detection. Approach. We addressed fixation-related EEG activity in an active search task with respect to stimulus-appearance styles and dynamics. Alongside popping-up stimuli, our experimental study embraces two composite appearance styles based on fading-in, enlarging, and motion effects. Additionally, we explored whether the knowledge obtained in the pop-up experimental setting can be exploited to boost the EEG-based intention-decoding performance when facing transitional changes of visual content. Main results. The results confirmed our initial hypothesis that the dynamic of visual content can increase temporal uncertainty of the cognition-related EEG activity in active search with respect to fixation onset. This temporal uncertainty challenges the pivotal aim to keep the decoding performance constant irrespective of visual effects. Importantly, the proposed approach for EEG decoding based on knowledge transfer between the different experimental settings gave a promising performance. Significance. Our study demonstrates that the non-stationarity of visual scenes is an important factor in the evolution of cognitive processes, as well as in the dynamic of ocular behavior (i.e., dwell time and fixation duration) in an active search task. In addition, our method to improve single-trial detection performance in this adverse scenario is an important step in making brain–computer interfacing technology available for human–computer interaction applications.EC/FP7/611570/EU/Symbiotic Mind Computer Interaction for Information Seeking/MindSeeBMBF, 01GQ0850, Bernstein Fokus Neurotechnologie - Nichtinvasive Neurotechnologie für Mensch-Maschine Interaktio

Eye Movements and Fixation-Related Potentials in Reading: A Review

The present review is addressed to researchers in the field of reading and psycholinguistics who are both familiar with and new to co-registration research of eye movements (EMs) and fixation related-potentials (FRPs) in reading. At the outset, we consider a conundrum relating to timing discrepancies between EM and event related potential (ERP) effects. We then consider the extent to which the co-registration approach might allow us to overcome this and thereby discriminate between formal theoretical and computational accounts of reading. We then describe three phases of co-registration research before evaluating the existing body of such research in reading. The current, ongoing phase of co-registration research is presented in comprehensive tables which provide a detailed summary of the existing findings. The thorough appraisal of the published studies allows us to engage with issues such as the reliability of FRP components as correlates of cognitive processing in reading and the advantages of analysing both data streams (i.e., EMs and FRPs) simultaneously relative to each alone, as well as the current, and limited, understanding of the relationship between EM and FRP measures. Finally, we consider future directions and in particular the potential of analytical methods involving deconvolution and the potential of measurement of brain oscillatory activity

Mobile Brain and Body Imaging during Walking Motor Tasks

Mobile brain and body imaging (MoBI) presents new and promising methods for moving traditional research studies out of a controlled laboratory and into the real world. Most current neuroimaging techniques require subjects to be stationary in laboratory settings because of both hardware and software limitations. Recent developments in mobile brain imaging have utilized Electroencephalography (EEG) in conjunction with advanced signal processing techniques such as Independent Component Analysis (ICA) to overcome these obstacles and study humans doing complex tasks in non-traditional environments. In my first study, I used high density EEG to examine the cortical dynamics of subjects walking on a split-belt treadmill with legs moving independently of each other at different speeds to investigate how humans adapt to novel perturbations. I found significantly increased low and high frequency spectral power across all sensorimotor and parietal neural sources during split-belt adaptation compared to normal walking, which provides insight into the brain areas and patterns used to accommodate locomotor adaptation. In my second study I combined multi-modal sensing and biometric devices including EEG, eye tracking, heart rate, accelerometers, and salivary cortisol into a portable setup that subjects wore indoors on a treadmill using virtual reality as well as outdoors in a public arboretum. Subjects walked for 1 hour each indoors and outdoors while completing a free viewing visual search oddball task in virtual reality and in real life. I reported on the methods for how to set this experiment up, synchronize all data, and standardize the data in order to make it usable as an open access dataset that has been made available to the public online. My third study used this data set to examine the P300 event-related potential response during both indoors in virtual reality and outdoors in the arboretum. I found a significantly increased amplitude response between 250 to 400 ms across the centro-parietal electrodes that distinguished target flags from distractor flags during visual search for both indoor and outdoor environments. And finally, for my fourth study I used the same data set to look at the behavioral and neural correlates associated with gait dynamics when subjects walked indoors on a treadmill vs outdoors in variable terrain while also doing the visual search task. I found significant EEG power differences across multiple neural sources that showed increased spectral fluctuations throughout the gait cycle when subjects walked outdoors compared to indoors on a treadmill. The collective studies in this dissertation present new ways of using mobile brain and body imaging devices to expand our knowledge of the neural dynamics involved in humans moving in complex ways and in variable environments outside of traditional laboratories.PHDBiomedical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/147691/1/ghanada_1.pd

Effects of cognitive tasks on car drivers’ behaviors and physiological responses

The effects of drivers’ engagement in cognitive tasks (i.e., non-visual, cognitively loading activities unrelated to the task of driving) are debated and unclear. Numerous experiments show impaired driver behaviors, yet naturalistic studies typically do not find an increased crash risk. In the future, autonomous driving (AD) is expected to improve traffic safety while allowing safe engagement in cognitive (and other) tasks. Having the opportunity to perform non-driving related tasks while traveling may then motivate drivers to use AD, provided they can actually engage in the tasks. Unfortunately, research on drivers’ engagement in cognitive tasks suffers severe methodological limitations since reliable and unintrusive measures of cognitive load are lacking.The aim of this thesis is therefore to advance the understanding of task-induced cognitive load in the context of traffic safety. This aim is split into two objectives: A) to better understand how drivers’ involvement in cognitive tasks can affect safety-relevant driver behaviors and decisions and B) to provide methodological guidance about assessing cognitive load in drivers using physiological measures.To accomplish Objective A, effects of cognitive tasks on driver behaviors were studied during routine driving and in a safety-critical event in a driving simulator. Also, drivers’ ability to engage in a non-driving related task while using AD in real traffic was explored. In line with the cognitive control hypothesis (Engstr\uf6m et al., 2017), it was found that cognitive tasks negatively affected driver behaviors in situations where cognitive control was needed, for example in intersections—but not in a lead vehicle braking scenario where responses were triggered automatically by visual looming. It was also found that although the number of off-path glances decreased during cognitive load, the timing of the remaining glances was unaffected. Clearly, cognitive load has different effects on different mechanisms. When using AD, drivers were indeed capable of engaging in a non-driving related task—suggesting that AD will be able to fulfill drivers’ desire to perform such tasks while traveling, which may motivate AD usage and thus improve traffic safety (given that AD is truly safer than manual driving). Finally, a simulator study addressing Objective B showed that the measurability of cognitive load was greatly improved by recognizing that multiple coexisting mental responses give rise to different physiological responses. This approach can provide less context-dependent measurements and allows for a better, more detailed understanding of the effects of cognitive tasks.These findings can help improve traffic safety—both by being used in system development, and as part of the systems themselves

Proceedings of the 3rd International Mobile Brain/Body Imaging Conference : Berlin, July 12th to July 14th 2018

The 3rd International Mobile Brain/Body Imaging (MoBI) conference in Berlin 2018 brought together researchers from various disciplines interested in understanding the human brain in its natural environment and during active behavior. MoBI is a new imaging modality, employing mobile brain imaging methods like the electroencephalogram (EEG) or near infrared spectroscopy (NIRS) synchronized to motion capture and other data streams to investigate brain activity while participants actively move in and interact with their environment. Mobile Brain / Body Imaging allows to investigate brain dynamics accompanying more natural cognitive and affective processes as it allows the human to interact with the environment without restriction regarding physical movement. Overcoming the movement restrictions of established imaging modalities like functional magnetic resonance tomography (MRI), MoBI can provide new insights into the human brain function in mobile participants. This imaging approach will lead to new insights into the brain functions underlying active behavior and the impact of behavior on brain dynamics and vice versa, it can be used for the development of more robust human-machine interfaces as well as state assessment in mobile humans.DFG, GR2627/10-1, 3rd International MoBI Conference 201