Psychophysiological responses to takeover requests in conditionally automated driving

In SAE Level 3 automated driving, taking over control from automation raises significant safety concerns because drivers out of the vehicle control loop have difficulty negotiating takeover transitions. Existing studies on takeover transitions have focused on drivers' behavioral responses to takeover requests (TORs). As a complement, this exploratory study aimed to examine drivers' psychophysiological responses to TORs as a result of varying non-driving-related tasks (NDRTs), traffic density and TOR lead time. A total number of 102 drivers were recruited and each of them experienced 8 takeover events in a high fidelity fixed-base driving simulator. Drivers' gaze behaviors, heart rate (HR) activities, galvanic skin responses (GSRs), and facial expressions were recorded and analyzed during two stages. First, during the automated driving stage, we found that drivers had lower heart rate variability, narrower horizontal gaze dispersion, and shorter eyes-on-road time when they had a high level of cognitive load relative to a low level of cognitive load. Second, during the takeover transition stage, 4s lead time led to inhibited blink numbers and larger maximum and mean GSR phasic activation compared to 7s lead time, whilst heavy traffic density resulted in increased HR acceleration patterns than light traffic density. Our results showed that psychophysiological measures can indicate specific internal states of drivers, including their workload, emotions, attention, and situation awareness in a continuous, non-invasive and real-time manner. The findings provide additional support for the value of using psychophysiological measures in automated driving and for future applications in driver monitoring systems and adaptive alert systems.University of Michigan McityPeer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/162593/1/AAP_physiological_responses_HF_template.pdfSEL

Employing consumer electronic devices in physiological and emotional evaluation of common driving activities

It is important to equip future vehicles with an on-board system capable of tracking and analysing driver state in real-time in order to mitigate the risk of human error occurrence in manual or semi-autonomous driving. This study aims to provide some supporting evidence for adoption of consumer grade electronic devices in driver state monitoring. The study adopted repeated measure design and was performed in high- fidelity driving simulator. Total of 39 participants of mixed age and gender have taken part in the user trials. The mobile application was developed to demonstrate how a mobile device can act as a host for a driver state monitoring system, support connectivity, synchronisation, and storage of driver state related measures from multiple devices. The results of this study showed that multiple physiological measures, sourced from consumer grade electronic devices, can be used to successfully distinguish task complexities across common driving activities. For instance, galvanic skin response and some heart rate derivatives were found to be correlated to overall subjective workload ratings. Furthermore, emotions were captured and showed to be affected by extreme driving situations

Preliminary study for the measurement of Biosignals in Driving Simulators

openThis preliminary study focuses on the goal of developing and testing a setup and method for non-invasive monitoring of individuals using biosensors in a professional driving simulator (VI-grade Compact Simulator). This involves the synchronization and integration of hardware and software components. To detect the emotional and cognitive state of the driver, it is crucial to identify which signals provide reliable information about their condition. The objective of this study is to observe individuals in a controlled and repeatable environment designed to stimulate cognitive workload. This was achieved using a multimodal assessment method (iMotions), which includes eye tracking, galvanic skin response (GSR), electromyography (EMG), and respiration measurements, all conducted during two distinct controlled driving simulation scenarios. Four healthy subjects (average age = 24, standard deviation = ±2) were monitored during the first scenario, a highway with repeated emergency maneuvers (slalom through cones and double lane change), and the second, five laps of the Paul Ricard circuit. All of this for a total duration of approximately 20 minutes. The participants were not aware that the scenarios were designed to provoke different reactions. This experimental thesis aims to be the continuation and evolution of a testing phase previously conducted during an internship at iMotions, a company that develops multimodal streaming software and distributes commercial hardware. The hardware was supplied to the NAVLAB at the University of Padua, where the simulator is located. The results obtained, at first analysis, appear to be consistent with the literature, suggesting that a multimodal approach to physiological signals may characterize emotional and cognitive states in driving scenarios.This preliminary study focuses on the goal of developing and testing a setup and method for non-invasive monitoring of individuals using biosensors in a professional driving simulator (VI-grade Compact Simulator). This involves the synchronization and integration of hardware and software components. To detect the emotional and cognitive state of the driver, it is crucial to identify which signals provide reliable information about their condition. The objective of this study is to observe individuals in a controlled and repeatable environment designed to stimulate cognitive workload. This was achieved using a multimodal assessment method (iMotions), which includes eye tracking, galvanic skin response (GSR), electromyography (EMG), and respiration measurements, all conducted during two distinct controlled driving simulation scenarios. Four healthy subjects (average age = 24, standard deviation = ±2) were monitored during the first scenario, a highway with repeated emergency maneuvers (slalom through cones and double lane change), and the second, five laps of the Paul Ricard circuit. All of this for a total duration of approximately 20 minutes. The participants were not aware that the scenarios were designed to provoke different reactions. This experimental thesis aims to be the continuation and evolution of a testing phase previously conducted during an internship at iMotions, a company that develops multimodal streaming software and distributes commercial hardware. The hardware was supplied to the NAVLAB at the University of Padua, where the simulator is located. The results obtained, at first analysis, appear to be consistent with the literature, suggesting that a multimodal approach to physiological signals may characterize emotional and cognitive states in driving scenarios

Exploring the utility of EDA and skin temperature as individual physiological correlates of motion sickness

Motion sickness (MS) is known to be a potentially limiting factor for future self-driving vehicles – specifically in regards to occupant comfort and well-being. With this as a consideration comes the desire to accurately measure, track and even predict MS state in real-time. Previous research has considered physiological measurements to measure MS state, although, this is mainly measured after an MS exposure and not throughout exposure(s) to a MS task. A unique contribution of this paper is in the real-time tracking of subjective MS alongside real-time physiological measurements of Electrodermal Activity (EDA) and skin temperature. Data was collected in both simulator-based (controlled) and on-road (naturalistic) studies. 40 participants provided at total of 61 data sets, providing 1,603 minutes of motion sickness data for analysis. This study is in agreement that these measures are related to MS but evidenced a total lack of reliability for these measures at an individual level for both simulator and on-road experimentation. It is likely that other factors, such as environment and emotional state are more impactful on these physiological measures than MS itself. At a cohort level, the applicability of physiological measures is not considered useful for measuring MS accurately or reliably in real-time. Recommendations for further research include a mixed-measures approach to capture other data types (such as subject activity) and to remove contamination of physiological measures from environmental changes

The Application of Physiological Metrics in Validating User Experience Evaluation on Automotive Human Machine Interface Systems

Automotive in-vehicle information systems have seen an era of continuous development within the industry and are recognised as a key differentiator for prospective customers. This presents a significant challenge for designers and engineers in producing effective next generation systems which are helpful, novel, exciting, safe and easy to use. The usability of any new human machine interface (HMI) has an implicit cost in terms of the perceived aesthetic perception and associated user experience. Achieving the next engaging automotive interface, not only has to address the user requirements but also has to incorporate established safety standards whilst considering new interaction technologies. An automotive (HMI) evaluation may combine a triad of physiological, subjective and performance-based measurements which are employed to provide relevant and valuable data for product evaluation. However, there is also a growing interest and appreciation that determining real-time quantitative metrics to drivers’ affective responses provide valuable user affective feedback. The aim of this research was to explore to what extent physiological metrics such as heart rate variability could be used to quantify or validate subjective testing of automotive HMIs. This research employed both objective and subjective metrics to assess user engagement during interactions with an automotive infotainment system. The mapping of both physiological and self-report scales was examined over a series of studies in order to provide a greater understanding of users’ responses. By analysing the data collected it may provide guidance within the early stages of in-vehicle design evaluation in terms of usability and user satisfaction. This research explored these metrics as an objective, quantitative, diagnostic measure of affective response, in the assessment of HMIs. Development of a robust methodology was constructed for the application and understanding of these metrics. Findings from the three studies point towards the value of using a combination of methods when examining user interaction with an in-car HMI. For the next generation of interface systems, physiological measures, such as heart rate variability may offer an additional dimension of validity when examining the complexities of the driving task that drivers perform every day. There appears to be no boundaries on technology advancements and with this, comes extra pressure for car manufacturers to produce similar interactive and connective devices to those that are already in use in homes. A successful in-car HMI system will be intuitive to use, aesthetically pleasing and possess an element of pleasure however, the design components that are needed for a highly usable HMI have to be considered within the context of the constraints of the manufacturing process and the risks associated with interacting with an in-car HMI whilst driving. The findings from the studies conducted in this research are discussed in relation to the usability and benefits of incorporating physiological measures that can assist in our understanding of driver interaction with different automotive HMIs

Psychological and psychophysiological effects of music intensity and lyrics on simulated urban driving

© 2021 The Authors. The main aim of this study was to investigate the effect of musical characteristics (i.e., presence of lyrics and loudness) in the context of simulated urban driving. Previous work has seldom isolated musical characteristics and examined these both singularly and interactively. We investigated the potentially distracting effects of processing lyrics through exposing young drivers to the same piece of music with/without lyrics and at different sound intensities (60 dBA [soft] and 75 dBA [loud]) using a counterbalanced, within-subjects design (N = 34; Mage = 22.2 years, SD = 2.0 years). Six simulator conditions were included that comprised low-intensity music with/without lyrics, high-intensity music with/without lyrics, plus two controls – ambient in-car noise and spoken lyrics. Between-subjects variables of driving style (defensive vs. assertive) and sex (women vs. men) were explored. A key finding was that the no lyrics/soft condition yielded lower affective arousal scores when compared to the other music conditions. There was no main effect of condition for HRV data (SDNN and RMSSD). Exploratory analyses showed that, for assertive drivers, NASA-TLX Performance scores were lower in the no lyrics/soft condition compared to the lyrics/loud condition. Moreover, women exhibited higher mean heart rate than men in the presence of lyrics. Although some differences emerged in subjective outcomes, these were not replicated in HRV, which was used as an objective index of emotionality. Drivers should consider the use of soft, non-lyrical music to optimise their affective state during urban driving.UKRI Economic and Social Research Council grant (ES/R005559/ 1); Direct Line Group (UK)

A Mobile Lifelogging Platform to Measure Anxiety and Anger During Real-Life Driving

The experience of negative emotions in everyday life, such as anger and anxiety, can have adverse effects on long-term cardiovascular health. However, objective measurements provided by mobile technology can promote insight into this psychobiological process and promote self-awareness and adaptive coping. It is postulated that the creation of a mobile lifelogging platform can support this approach by continuously recording personal data via mobile/wearable devices and processing this information to measure physiological correlates of negative emotions. This paper describes the development of a mobile lifelogging system that measures anxiety and anger during real-life driving. A number of data streams have been incorporated in the platform, including cardiovascular data, speed of the vehicle and first-person photographs of the environment. In addition, thirteen participants completed five days of data collection during daily commuter journeys to test the system. The design of the system hardware and associated data streams are described in the current paper, along with the results of preliminary data analysis

Do differences in personality traits affect how drivers experience music at different intensities?

Various researchers have investigated contributing factors towards the number of acute traffic incidences in and around Southern Africa. Some of these contributing factors include: the skills component of the driver predominately attributed to driving experience as well as the behavioural component influenced by the driver’s natural predisposition, individual differences and personality traits. In order to manage these factors drivers have developed varying coping mechanisms. One of these coping mechanisms is listening to music while driving, which is readily available in most cars and extensively used predominately during long duration driving. Listening to music neither increases one’s driving duration (as opposed to taking several breaks), nor does it interfere with the physical movements of driving (in the manner that eating and drinking may), but it might impact the concentration and attention of some drivers. This is based on the notion that music is assumed to impact arousal and cognitive ability. While there are several studies on the effect of music on driving performance and personality traits very few studies have looked at whether music has a positive or negative effect on driving performance based on differences in personality traits; and whether the extent of this effect might differ for different intensities of music? Consequently, this study aims to understand and determine the extent to which different personality traits predict the effect that listening to different music intensities has on driving performance. The impact of differing music conditions on the different personality traits used a repeated measures design and a between group design with respect to the personality traits with a sample size of (n=25)-16 females and 9 males-and their ages ranged between 19-35 years of age. The average age and standard deviation for this sample size was 22 years±2. A low-fidelity driving simulator task was utilised in order to provide a controllable, repeatable and a safe environment as compared to a real road situation. Personality was assessed using an online Big-Five Inventory scale (extraversion, agreeableness, conscientiousness, neuroticism, openness). All the different personality groups completed three conditions (45 minutes each) in a randomised order (without music, moderately loud music and loud music). Psychophysiological parameters i.e. heart rate frequency (HRF), heart rate variability (HRV) and eye movements (pupil diameter, eye speeds, fixation duration, blink frequency and blink duration) and driving performance were measured continuously. Subjective performance Multidimensional Driving Style Inventory was measured once-off prior to completion of the testing sessions, whilst the NASA-Task Load Index scale and Perceived control of participants were assessed after each condition. The expected outcomes revealed that music had an effect on objective driving performance (tracking deviation and reaction time) and psychophysiological measures only for participants of certain personality types while other personality types were unaffected by music. The subjective performance measures did not follow the same trend as objective performance measures. The conditions did not reveal an effect on driving performance, for most of the psychophysiological parameters and subjective measures. There was mainly a significant time on task effect and interactional effects on the psychophysiological measures (physiological and oculomotor) parameters at (p<0.05), but not on the subjective measures as anticipated. The study illustrated that the there are differences between personality traits. There was difficulty in the interpretation of the results based on the complexity of the findings for which each hypothesis was partially accepted. The research may establish practical implications for traffic safety campaigns in South Africa, as well as influence driving education for citizens. Assessing the personality trait would help to form an understanding as to which of the personality traits might be affected negatively from listening to music while driving and those that might benefit. Moreover, this study may assist motorists in understanding the implications of listening to music while driving as this may sometimes elicit risky driving behaviour and possibly cause an accident that may result in death