Employing consumer electronic devices in physiological and emotional evaluation of common driving activities

It is important to equip future vehicles with an on-board system capable of tracking and analysing driver state in real-time in order to mitigate the risk of human error occurrence in manual or semi-autonomous driving. This study aims to provide some supporting evidence for adoption of consumer grade electronic devices in driver state monitoring. The study adopted repeated measure design and was performed in high- fidelity driving simulator. Total of 39 participants of mixed age and gender have taken part in the user trials. The mobile application was developed to demonstrate how a mobile device can act as a host for a driver state monitoring system, support connectivity, synchronisation, and storage of driver state related measures from multiple devices. The results of this study showed that multiple physiological measures, sourced from consumer grade electronic devices, can be used to successfully distinguish task complexities across common driving activities. For instance, galvanic skin response and some heart rate derivatives were found to be correlated to overall subjective workload ratings. Furthermore, emotions were captured and showed to be affected by extreme driving situations

Exploring the utility of EDA and skin temperature as individual physiological correlates of motion sickness

Motion sickness (MS) is known to be a potentially limiting factor for future self-driving vehicles – specifically in regards to occupant comfort and well-being. With this as a consideration comes the desire to accurately measure, track and even predict MS state in real-time. Previous research has considered physiological measurements to measure MS state, although, this is mainly measured after an MS exposure and not throughout exposure(s) to a MS task. A unique contribution of this paper is in the real-time tracking of subjective MS alongside real-time physiological measurements of Electrodermal Activity (EDA) and skin temperature. Data was collected in both simulator-based (controlled) and on-road (naturalistic) studies. 40 participants provided at total of 61 data sets, providing 1,603 minutes of motion sickness data for analysis. This study is in agreement that these measures are related to MS but evidenced a total lack of reliability for these measures at an individual level for both simulator and on-road experimentation. It is likely that other factors, such as environment and emotional state are more impactful on these physiological measures than MS itself. At a cohort level, the applicability of physiological measures is not considered useful for measuring MS accurately or reliably in real-time. Recommendations for further research include a mixed-measures approach to capture other data types (such as subject activity) and to remove contamination of physiological measures from environmental changes

Effect of cognitive load on drivers’ State and task performance during automated driving: Introducing a novel method for determining stabilisation time following take-over of control

This research paper explores the impact of cognitive load on drivers’ physiological state and driving performance during an automated driving to manual control transition scenario, using a driving simulator. Whilst driving in the automated mode, cognitive load was manipulated using the “N-Back” task, which participants engaged with via a visual display. Results suggest that non-optimal levels of workload during the automated driving conditions impair driving performance, especially lateral control of the vehicle, and the magnitude of this impairment varied with increasing cognitive load. In addition to these findings, the present paper introduces a novel method for determining stabilisation times of both driver state and driving performance indicators following a transition of vehicle control. Using this method we demonstrate that mean and standard deviation of lane position impairments were found to take longer to stabilise following transition to manual driving following a higher level of cognitive load during the automated driving period, taking up to 22 s for driving performance to normalise after take-over. In addition, heart rate parameters take between 20 and 30 s to stabilise following a planned take-over request. Finally, this paper demonstrates how the magnitude of cognitive load can be estimated in context of automated driving using physiological measures, captured by consumer electronic devices. We discuss the impact our findings have on the design of SAE Level 3 systems. Relevant suggestions are provided to the research community and automakers working on future implementation of vehicles capable of conditional automation

Towards multimodal driver state monitoring systems for highly automated driving

Real-time monitoring of drivers’ functional states will soon become a required safety feature for commercially available vehicles with automated driving capability. Automated driving technology aims to mitigate human error from road transport with the progressive automatisation of specific driving tasks. However, while control of the driving task remains shared between humans and automated systems, the inclusion of this new technology is not exempt from other human factors-related challenges. Drivers’ functional states are essentially a combination of psychological, emotional, and cognitive states, and they generate a constant activity footprint available for measurement through neural and peripheral physiology, among other measures. These factors can determine drivers’ functional states and, thus, drivers’ availability to safely perform control transitions between human and vehicle. This doctoral project aims at investigating the potential of electrocardiogram (ECG), electrodermal activity (EDA) and functional near-infrared spectroscopy (fNIRS) as measures for a multimodal driver state monitoring (DSM) system for highly automated driving (i.e., SAE levels 3 and 4). While current DSM systems relying on gaze behaviour measures have proven valid and effective, several limitations and challenges could only be overcome using eye-tracking in tandem with physiological parameters. This thesis investigates whether ECG, EDA and fNIRS would be good candidates for such a purpose. Two driving simulator studies were performed to measure mental workload, trust in automation, stress and perceived risk, all identified as modulators of drivers’ functional states and that could eventually determine drivers’ availability to take-over manual control. The main findings demonstrate that DSM systems should adopt multiple physiological measures to capture changes in functional states relevant for driver readiness. Future DSM systems will benefit from the knowledge generated by this research by applying machine learning methods to these measures for determining drivers’ availability for optimal take-over performance

Understanding and managing motion sickness in future vehicles : innovation report

Almost everyone is susceptible to motion sickness, and around one in three people are known to be highly susceptible. It has been argued that the use of automated vehicles will increase motion sickness severity and onset frequency for those who already regularly suffer from it, as well as for those who are susceptible, but don’t regularly get motion sick in traditional vehicles. This is primarily due to the engagement with non-driving activities which cause sensory conflict, the relinquishing of control which prevents apprehension of current and upcoming motion, and the limited ability to self-mitigate due to potential vehicle designs and the inability to take control of the dynamic driving task in a fully automated vehicle. This research first contextualised the relationship between motion sickness and future automotive technologies – covering both research focused driving simulators as well as ‘real-world’ use cases for on-road partially to fully automated vehicles. A framework for future research was developed and three core projects were established, positioned to cover the breadth of the field. Following this framework, the first project explored the impact of motion sickness on human performance, this was followed by the development of a method of reducing susceptibility to motion sickness and finally, objective measurements of motion sickness were explored. Motion sickness is a consideration for not only the day-to-day utility of future automated vehicles, but also within the development and simulator-based testing of such technology. Despite the myriad benefits of driving simulators for developing future technology, one significant side effect is simulator-induced motion sickness or ‘simulation sickness’. The first project, using both simulator-based and real-world experimentation, explored the effect of motion sickness on human performance – informing our understanding about transferability of simulator data to ‘real-world’ as well as providing insights into the relationship between motion sickness and productivity for future vehicles. The second research project proposes, develops, tests and validates a novel method of reducing motion sickness susceptibility by way of specific visual-cognitive training activities. Experimentation began using a high fidelity driving simulator where it was first shown how it is possible to increase visuospatial skills through a novel assimilation and application of a pen-and-paper training pack. Subsequently, this increased visuospatial skill reduced both subjective simulator sickness by 58%, and dropouts due to severe motion sickness by 60%. This simulator-based study was followed up with an on-road study where the visuospatial training pack was further validated for ‘real-world’ utility and was shown to be responsible for a reduction in motion sickness by 52% across the experimental group. Further to the core findings presented, an industry-focused workshop identified ways in which this new knowledge can be exploited for consumer-focused utility. This research also contributes to the fundamental understanding of the relationship between visuospatial ability and motion sickness susceptibility. Through extensive simulator-based and on-road motion sickness experimentation, the third research project pulls together physiological and subjective motion sickness data to explore concepts for objectively measuring and detecting motion sickness in real-time. Building upon literature from both motion sickness and machine learning fields, a wide range of data types, from demographics, to vehicle conditions, to occupant activity and route design are highlighted to be potentially useful in future objective motion sickness studies. Based on these sources of data, and many more, a new model is proposed through which motion sickness related data can be collected to aid in the objective measurement of motion sickness. The research conducted here provides a novel contribution in understanding motion sickness related human performance degradation and provides an interesting discussion about the impact this may have for both simulator trials, and automated vehicle utility. Through the design and validation of a novel training tool for reducing motion sickness susceptibility (in simulators and ‘real-world’) this research adds to the knowledge about our fundamental understanding of motion sickness and provides an innovative solution to address the issue of motion sickness. Further contributions are found within the research looking at objective measurements of motion sickness and among other various design recommendations