Physiological Measurements for Real-time Fatigue Monitoring in Train Drivers: Review of the State of the Art and Reframing the Problem

The impact of fatigue on train drivers is one of the most important safety-critical issues in rail. It affects drivers’ performance, significantly contributing to railway incidents and accidents. To address the issue of real-time fatigue detection in drivers, most reliable and applicable psychophysiological indicators of fatigue need to be identified. Hence, this paper aims to examine and present the current state of the art in physiological measures for real-time fatigue monitoring that could be applied in the train driving context. Three groups of such measures are identified: EEG, eye-tracking and heart-rate measures. This is the first paper to provide the analysis and review of these measures together on a granular level, focusing on specific variables. Their potential application to monitoring train driver fatigue is discussed in respective sections. A summary of all variables, key findings and issues across these measures is provided. An alternative reconceptualization of the problem is proposed, shifting the focus from the concept of fatigue to that of attention. Several arguments are put forward in support of attention as a better-defined construct, more predictive of performance decrements than fatigue, with serious ramifications on human safety. Proposed reframing of the problem coupled with the detailed presentation of findings for specific relevant variables can serve as a guideline for future empirical research, which is needed in this field

Exploring cognition in visual search and vigilance tasks with eye tracking and pupillometry

Recent findings in experimental psychology suggest that pupillometry, the measurement of pupil size, can provide insight into cognitive processes associated with effort and target detection in visual search tasks and monitoring performance in vigilance tasks. With the increasing availability, affordability and flexibility of video-based eye tracking hardware, these experimental findings point to lucrative practical applications such as real-time biobehavioural monitoring systems to assist with socially important tasks in operational settings. The aim of the current thesis was to explore this potential with further experimental work paying close attention to methodological issues which complicate cognitive interpretations of pupillary responses, such as physical stimulus confounds and eye movement-related measurement error in video-based systems. Six original experiments were designed to specifically explore the relationship between pupil size, cognition and behavioural performance in classic visual search and vigilance paradigms. Experiments 1-2 examined the pupillometric effects of effort and target detection in visual search with briefly presented stimuli. Pupil responses showed small variability with respect to manipulations of set size and target presence but were influenced substantially by the requirement for a motor response. Experiments 3-4 explored the cognitive pupil dynamics of free-viewing visual search with data-driven correction for eye movement artefacts. Group-level averages revealed small transient pupil dilations following fixations on targets but not distractors, an effect which was not contingent on a motor response or correction for gaze position artefacts. Experiments 5-6 looked at the relationship between pupil size and detection performance measures in two types of vigilance task. Changes in baseline and stimulus-evoked pupil responses loosely mirrored changes in performance, but the relationships were neither linear nor consistent. Overall, the thesis affirms the practical potential for using cognitive pupillometry in research and applied settings, but emphasises the constraints arising from methodological and theoretical limitations

Effects of Signal Probability on Multitasking-Based Distraction in Driving, Cyberattack & Battlefield Simulation

Multitasking-based failures of perception and action are the focus of much research in driving, where they are attributed to distraction. Similar failures occur in contexts where the construct of distraction is little used. Such narrow application was attributed to methodology which cannot precisely account for experimental variables in time and space, limiting distraction\u27s conceptual portability to other contexts. An approach based upon vigilance methodology was forwarded as a solution, and highlighted a fundamental human performance question: Would increasing the signal probability (SP) of a secondary task increase associated performance, as is seen in the prevalence effect associated with vigilance tasks? Would it reduce associated performance, as is seen in driving distraction tasks? A series of experiments weighed these competing assumptions. In the first, a psychophysical task, analysis of accuracy and response data revealed an interaction between the number of concurrent tasks and SP of presented targets. The question was further tested in the applied contexts of driving, cyberattack and battlefield target decision-making. In line with previous prevalence effect inquiry, presentation of stimuli at higher SP led to higher accuracy. In line with existing distraction work, performance of higher numbers of concurrent tasks tended to elicit slower response times. In all experiments raising either number of concurrent tasks or SP of targets resulted in greater subjective workload, as measured by the NASA TLX, even when accompanied by improved accuracy. It would seem that distraction in previous experiments has been an aggregate effect including both delayed response time and prevalence-based accuracy effects. These findings support the view that superior experimental control of SP reveals nomothetic patterns of performance that allow better understanding and wider application of the distraction construct both within and in diverse contexts beyond driving

Visual Attention-Related Processing: Perspectives from Ageing, Cognitive Decline and Dementia

Visual attention is essential for environmental interactions, but our ability to respond to stimuli gradually declines across the lifespan, and such deficits are even more pronounced in various states of cognitive impairment. Examining the integrity of related components, from elements of attention capture to executive control, will improve our understanding of related declines by helping to explain behavioural and neural effects, which will ultimately contribute towards our knowledge of the extent of dysfunctional attention processes and their impact upon everyday life. Accordingly, this Special Issue represents a body of literature that fundamentally advances insights into visual attention processing, featuring studies spanning healthy ageing, mild cognitive impairment, and dementi

Applied and laboratory-based autonomic and neurophysiological monitoring during sustained attention tasks

Fluctuations during sustained attention can cause momentary lapses in performance which can have a significant impact on safety and wellbeing. However, it is less clear how unrelated tasks impact current task processes, and whether potential disturbances can be detected by autonomic and central nervous system measures in naturalistic settings. In a series of five experiments, I sought to investigate how prior attentional load impacts semi-naturalistic tasks of sustained attention, and whether neurophysiological and psychophysiological monitoring of continuous task processes and performance could capture attentional lapses. The first experiment explored various non-invasive electrophysiological and subjective methods during multitasking. The second experiment employed a manipulation of multitasking, task switching, to attempt to unravel the negative lasting impacts of multitasking on neural oscillatory activity, while the third experiment employed a similar paradigm in a semi-naturalistic environment of simulated driving. The fourth experiment explored the feasibility of measuring changes in autonomic processing during a naturalistic sustained monitoring task, autonomous driving, while the fifth experiment investigated the visual demands and acceptability of a biological based monitoring system. The results revealed several findings. While the first experiment demonstrated that only self-report ratings were able to successfully disentangle attentional load during multitasking; the second and third experiment revealed deficits in parieto-occipital alpha activity and continuous performance depending on the attentional load of a previous unrelated task. The fourth experiment demonstrated increased sympathetic activity and a smaller distribution of fixations during an unexpected event in autonomous driving, while the fifth experiment revealed the acceptability of a biological based monitoring system although further research is needed to unpick the effects on attention. Overall, the results of this thesis help to provide insight into how autonomic and central processes manifest during semi-naturalistic sustained attention tasks. It also provides support for a neuro- or biofeedback system to improve safety and wellbeing

A psychophysiological insight into driver state during highly automated driving

The aim of this research was to investigate and validate the usage of physiological measures as an objective indicator of driver state in dynamic driving environments, and understand if such a methodology can be used to measure driver discomfort, and high workload. The work addressed questions relating to: (i) detecting and removing motion artefacts from electrodermal activity (EDA) signals in dynamic driving environments; (ii) primary factors contributing to driver discomfort during automation, measured in terms of their physiological state; (iii) understanding changes in drivers’ workload levels at different stages of automation, as indicated by electrocardiogram (ECG) and EDA-based measures and; (iv) how drivers’ attentional demands and workload levels are affected at different stages of automation, measured using eye tracking-based metrics. A series of experiments were developed to manipulate drivers’ discomfort and workload levels. The analysis around driver discomfort focused on automated driving, whereas drivers’ workload levels were investigated during automation, and during resumption of control from automation, in a series of car-following scenarios. Our results indicated that phasic EDA was able to pick up discomfort experienced by the driver during automation, and correlated to drivers’ subjective ratings of discomfort. Narrower roads, higher resultant acceleration forces and how the automated vehicle negotiated different road geometries all influenced driver discomfort. We observed that drivers’ workload levels were captured by ECG and EDA-based signals, with phasic component of EDA signal being more sensitive to short term variations in driver workload. Similar results were observed in drivers’ pupil diameter values, as well as subjective ratings of workload. Factors such as engagement in a non-driving related task (NDRT), presence of a lead vehicle while maintaining a short time headway, and takeovers, all seemed to increase drivers’ workload levels. Future work can build on this research by incorporating sensor fusion of ECG and EDA-based data, along with eye tracking, to help improve the accuracy and capabilities of future driver state monitoring systems

Aerospace Medicine and Biology: A continuing bibliography with indexes (supplement 314)

This bibliography lists 139 reports, articles, and other documents introduced into the NASA scientific and technical information system in August, 1988

Regression Based Continuous Driving Fatigue Estimation: Towards Practical Implementation

Mental fatigue in drivers is one of the leading causes that give rise to traffic accidents. Electroencephalography (EEG) based driving fatigue studies showed promising performance in fatigue monitoring. However, complex methodologies are not suitable for practical implementation. In our simulation based setup that retained the constraints of real driving, we took a step closer to fatigue estimation in a practical scenario. We adopted a pre-processing pipeline with low computational complexity, which can be easily and practically implemented in real-time. Moreover, regression-based continuous fatigue estimation was achieved using power spectral features in conjunction with time as the fatigue label. We sought to compare three regression models and three time windows to demonstrate their effects on the performance of fatigue estimation. Dynamic time warping was proposed as a new measure for evaluating the performance of fatigue estimation. The results derived from the validation of the proposed framework on 19 subjects showed that our proposed framework was promising towards practical implementation. Fatigue estimation by the support vector regression with radial basis function kernel and 5-second window length achieved the best performance. We also provided a comprehensive analysis on the spatial distribution of channels and frequency bands mostly contributing to fatigue estimation, which can inform the feature and channel reduction for real-time fatigue monitoring in practical driving. After reducing the number of electrodes by 75%, the proposed framework retained comparable performance in fatigue estimation. This study demonstrates the feasibility and adaptability of our proposed framework in practical implementation of mental fatigue estimation

Analyzing Action Game Players\u27 Performance During Distracted Driving

Driving is a complex task that is highly reliant on attention. Research states that distractions cause performance errors thus it is important to find ways to reduce driver distraction or assist drivers with ways to improve their cognitive resources if distraction is unavoidable. Moreover, research indicates that action video game players outperform non-players on labbased tests of visual and cognitive abilities. However, research also exists that is contrary to these findings. Some researchers suggest that methodological deficiencies could be the cause of the significant findings in the literature. With such fervor of debate on the subject, the question remains of whether players acquire skills through playing action video games and if so can these games be used as research or training tools to enhance performance on realistic tasks. To answer this question, 45 male participants were tested using psychometric measures of spatial ability (Spatial orientation and visualization) and failures of attention (Cognitive Failures Questionnaire), and then all participants drove four 10-minute drives in a driving simulator. The first drive was a practice, followed by a control drive. Participants were then distracted using a hands free phone conversation. Following that, participants completed a final control drive. Both overall video game experience and action video game experience was positively related to higher spatial ability scores. Additionally, participants with higher action game experience exhibited fewer lane deviations during driving overall, but not during the distraction condition. On the other hand, participants with higher spatial ability scores exhibited fewer lane deviations during the distraction condition, but not during the control drives. Furthermore, action video game experience was not significant on the Cognitive Failures Questionnaire. Therefore, it was concluded that individuals who have higher action game experience do not show improvements on any iv abilities of attention tested in this study. However, higher experience action video game players may perform better in simulated environments than those with less experience

Towards multimodal driver state monitoring systems for highly automated driving

Real-time monitoring of drivers’ functional states will soon become a required safety feature for commercially available vehicles with automated driving capability. Automated driving technology aims to mitigate human error from road transport with the progressive automatisation of specific driving tasks. However, while control of the driving task remains shared between humans and automated systems, the inclusion of this new technology is not exempt from other human factors-related challenges. Drivers’ functional states are essentially a combination of psychological, emotional, and cognitive states, and they generate a constant activity footprint available for measurement through neural and peripheral physiology, among other measures. These factors can determine drivers’ functional states and, thus, drivers’ availability to safely perform control transitions between human and vehicle. This doctoral project aims at investigating the potential of electrocardiogram (ECG), electrodermal activity (EDA) and functional near-infrared spectroscopy (fNIRS) as measures for a multimodal driver state monitoring (DSM) system for highly automated driving (i.e., SAE levels 3 and 4). While current DSM systems relying on gaze behaviour measures have proven valid and effective, several limitations and challenges could only be overcome using eye-tracking in tandem with physiological parameters. This thesis investigates whether ECG, EDA and fNIRS would be good candidates for such a purpose. Two driving simulator studies were performed to measure mental workload, trust in automation, stress and perceived risk, all identified as modulators of drivers’ functional states and that could eventually determine drivers’ availability to take-over manual control. The main findings demonstrate that DSM systems should adopt multiple physiological measures to capture changes in functional states relevant for driver readiness. Future DSM systems will benefit from the knowledge generated by this research by applying machine learning methods to these measures for determining drivers’ availability for optimal take-over performance