Driver State Monitoring

Vedieť šoférovať auto je v dnešnom svete považované za dôležité, či dokonca nutné. Avšak, s pribúdajúcou premávkou je nevyhnutné, že sa stanú nehody, ktoré môžu mať aj fatálne následky.  Množstvo informácií, ktoré musí vodič vedieť spracovať sa neustále zvyšuje a s tým aj stresové situácie, počas ktorých vodič nemusí vedieť racionálne uvažovať.  Táto práca sa zameriava na nájdenie techník a postupov ako implementovať monitor, ktorý by predikoval level stresu vodiča počas jazdy, aby sa predišlo takýmto nehodám.V práci boli implementované dva typy modelov - osobný a všeobecný. Oba využívali fyziologické dáta spolu s kinematickými, ktoré boli nahrané počas jazdy. Oba modely boli schopé predpovedať úroveň stresu s vysokou pravdepodobnosťou.Driving is considered an important, or even necessary skill in today's world. However, with so much traffic, it is inevitable that accidents occur that might have fatal consequences. The amount of information the driver is suppose to process could lead to stress situations, during which he or she is unable to take rational decisions. This thesis aims to find the most suitable technologies and methodologies to implement a driver's state monitor, that could predict driver stress level to mitigate the possibility of such accidents. The thesis proposes personal and average monitors to assess driver's stress levels by the means of physiological and kinematics data collected during a drive. Both monitors proved to have high predicting power.

Towards hybrid driver state monitoring : review, future perspectives and the role of consumer electronics

The purpose of this paper is to bring together multiple literature sources which present innovative methodologies for the assessment of driver state, driving context and performance by means of technology within a vehicle and consumer electronic devices. It also provides an overview of ongoing research and trends in the area of driver state monitoring. As part of this review a model of a hybrid driver state monitoring system is proposed. The model incorporates technology within a vehicle and multiple broughtin devices for enhanced validity and reliability of recorded data. Additionally, the model draws upon requirement of data fusion in order to generate unified driver state indicator(-s) that could be used to modify in-vehicle information and safety systems hence, make them driver state adaptable. Such modification could help to reach optimal driving performance in a particular driving situation. To conclude, we discuss the advantages of integrating hybrid driver state monitoring system into a vehicle and suggest future areas of research

Employing consumer electronic devices in physiological and emotional evaluation of common driving activities

It is important to equip future vehicles with an on-board system capable of tracking and analysing driver state in real-time in order to mitigate the risk of human error occurrence in manual or semi-autonomous driving. This study aims to provide some supporting evidence for adoption of consumer grade electronic devices in driver state monitoring. The study adopted repeated measure design and was performed in high- fidelity driving simulator. Total of 39 participants of mixed age and gender have taken part in the user trials. The mobile application was developed to demonstrate how a mobile device can act as a host for a driver state monitoring system, support connectivity, synchronisation, and storage of driver state related measures from multiple devices. The results of this study showed that multiple physiological measures, sourced from consumer grade electronic devices, can be used to successfully distinguish task complexities across common driving activities. For instance, galvanic skin response and some heart rate derivatives were found to be correlated to overall subjective workload ratings. Furthermore, emotions were captured and showed to be affected by extreme driving situations

Vision-based Driver State Monitoring Using Deep Learning

Road accidents cause thousands of injuries and losses of lives every year, ranking among the top lifetime odds of death causes. More than 90% of the traffic accidents are caused by human errors [1], including sight obstruction, failure to spot danger through inattention, speeding, expectation errors, and other reasons. In recent years, driver monitoring systems (DMS) have been rapidly studied and developed to be used in commercial vehicles to prevent human error-caused car crashes. A DMS is a vehicle safety system that monitors driver’s attention and warns if necessary. Such a system may contain multiple modules that detect the most accident-related human factors, such as drowsiness and distractions. Typical DMS approaches seek driver distraction cues either from vehicle acceleration and steering (vehicle-based approach), driver physiological signals (physiological approach), or driver behaviours (behavioural-based approach). Behavioural-based driver state monitoring has numerous advantages over vehicle-based and physiological-based counterparts, including fast responsiveness and non-intrusiveness. In addition, the recent breakthrough in deep learning enables high-level action and face recognition, expanding driver monitoring coverage and improving model performance. This thesis presents CareDMS, a behavioural approach-based driver monitoring system using deep learning methods. CareDMS consists of driver anomaly detection and classification, gaze estimation, and emotion recognition. Each approach is developed with state-of-the-art deep learning solutions to address the shortcomings of the current DMS functionalities. Combined with a classic drowsiness detection method, CareDMS thoroughly covers three major types of distractions: physical (hands-off-steering wheel), visual (eyes-off-road ahead), and cognitive (minds-off-driving). There are numerous challenges in behavioural-based driver state monitoring. Current driver distraction detection methods either lack detailed distraction classification or unknown driver anomalies generalization. This thesis introduces a novel two-phase proposal and classification network architecture. It can suspect all forms of distracted driving and recognize driver actions simultaneously, which provide downstream DMS important information for warning level customization. Next, gaze estimation for driver monitoring is difficult as drivers tend to have severe head movements while driving. This thesis proposes a video-based neural network that jointly learns head pose and gaze dynamics together. The design significantly reduces per-head-pose gaze estimation performance variance compared to benchmarks. Furthermore, emotional driving such as road rage and sadness could seriously impact driving performance. However, individuals have various emotional expressions, which makes vision-based emotion recognition a challenging task. This work proposes an efficient and versatile multimodal fusion module that effectively fuses facial expression and human voice for emotion recognition. Visible advantages are demonstrated compared to using a single modality. Finally, a driver state monitoring system, CareDMS, is presented to convert the output of each functionality into a specific driver’s status measurement and integrates various measurements into the driver’s level of alertness

JLR heart : employing wearable technology in non-intrusive driver state monitoring. Preliminary study

This paper presents the results from a preliminary study where a wearable consumer electronic device was used to assess driver’s state by capturing human physiological response in non-intrusive manner. Majority of state of the art studies have employed medical equipment drivers’ state evaluation. Despite the potential gain in road safety this method of measuring physiology is unlikely to be accepted by private vehicle consumers due to its invasiveness, complexity, and high cost. This study was aiming to investigate possibility of employing a consumer grade wearable device to measure physiological parameters related to cognitive workload in realtime while driving i.e., drivers’ heart rate. Furthermore, validity of captured heart activity metrics was analyzed to determine if wearable devices could be embedded into driving at its current technological state. The driving context was reproduced in desktop driving simulator, with 14 participants agreeing to take part in the study (µ = 28, σ = 8.5 years). Drivers were exposed to various road types, including pure Motorway, Rural, and Urban scenario modes. An accident was simulated in order to generate sudden cognitive arousal and capture participants’ physiological response to the generated distress. It was found that a smartwatch is capable of reliable heart activity tracking in driving context. The results, supporting the relationship between cognitive workload level, generated by various complexity driving tasks, and Heart Rate Variability, were also presented

Integrating trust in automation into driver state monitoring systems

Inappropriate trust in highly automated vehicles (HAVs) has been identified as one of the causes in several accidents [1–3]. These accidents have evidenced the need to include a Driver State Monitoring System (DSMS) [4] into those HAVs which may require occasional manual driving. DSMS make use of several psychophysiological sensors to monitor the drivers’ state, and have already been included in current production vehicles to detect drowsiness, fatigue and distractions [5]. However, DSMS have never been used to monitor Trust in Automation (TiA) states within HAVs yet. Based on recent findings, this paper proposes a new methodology to integrate TiA state-classification into DSMSs for future vehicles

Effects of an unexpected and expected event on older adults’ autonomic arousal and eye fixations during autonomous driving

© Copyright © 2020 Stephenson, Eimontaite, Caleb-Solly, Morgan, Khatun, Davis and Alford. Driving cessation for some older adults can exacerbate physical, cognitive, and mental health challenges due to loss of independence and social isolation. Fully autonomous vehicles may offer an alternative transport solution, increasing social contact and encouraging independence. However, there are gaps in understanding the impact of older adults’ passive role on safe human–vehicle interaction, and on their well-being. 37 older adults (mean age ± SD = 68.35 ± 8.49 years) participated in an experiment where they experienced fully autonomous journeys consisting of a distinct stop (an unexpected event versus an expected event). The autonomous behavior of the vehicle was achieved using the Wizard of Oz approach. Subjective ratings of trust and reliability, and driver state monitoring including visual attention strategies (fixation duration and count) and physiological arousal (skin conductance and heart rate), were captured during the journeys. Results revealed that subjective trust and reliability ratings were high after journeys for both types of events. During an unexpected stop, overt visual attention was allocated toward the event, whereas during an expected stop, visual attention was directed toward the human–machine interface (HMI) and distributed across the central and peripheral driving environment. Elevated skin conductance level reflecting increased arousal persisted only after the unexpected event. These results suggest that safety-critical events occurring during passive fully automated driving may narrow visual attention and elevate arousal mechanisms. To improve in-vehicle user experience for older adults, a driver state monitoring system could examine such psychophysiological indices to evaluate functional state and well-being. This information could then be used to make informed decisions on vehicle behavior and offer reassurance during elevated arousal during unexpected events

A methodology to explore the road safety impact of fitness to drive solutions for commercial drivers: The PANACEA project

In Europe, one in four road deaths occurred in an accident involving a goods vehicle in 2018 (ETSC 2020). Commercial drivers are at higher risk for suffering from physiological, psychological and prescribed medication and illicit drug use, including alcohol misuse. Fitness to drive or driver state monitoring systems integrate technologies able to detect altered driver states and provide them feedback. They constitute an emerging phenomenon, and their effects on changing people's behaviour to drive more safely, and in general, their impact on road safety should be better investigated. The scope of this paper is to present a methodology able to simulate different scenarios to understand how a driver state monitoring system can support improving road safety in the European Union. A conceptual framework is presented to support the definition of the impact assessment methodology and is applied to the PANACEA European research project. The project develops an integrated solution for driving ability assessment of commercial drivers, paired with a countermeasure and coaching solution. The PANACEA system uses algorithms and technologies for detecting, monitoring and assessing alcohol consumption, licit (barbituric) and illicit (methadone substitute) drugs, fatigue and cognitive load (Commercial Health Toolkits (CHTs)). It also provides strategic, tactical and operational countermeasures that will be tested and evaluated to assess their effectiveness and acceptance by the system's users. The methodology presented is able to assess both a single and multiple countermeasures among those developed within the project. Different scenarios have been considered by modifying the variables according to the screening prevalence, solution acceptance level, driving context and time. The methodology uses the results from the project pilot studies in terms of accuracy, sensitivity, and specificity of CHTs and countermeasures results in combination with evidence from the existing literature

Towards multimodal driver state monitoring systems for highly automated driving

Real-time monitoring of drivers’ functional states will soon become a required safety feature for commercially available vehicles with automated driving capability. Automated driving technology aims to mitigate human error from road transport with the progressive automatisation of specific driving tasks. However, while control of the driving task remains shared between humans and automated systems, the inclusion of this new technology is not exempt from other human factors-related challenges. Drivers’ functional states are essentially a combination of psychological, emotional, and cognitive states, and they generate a constant activity footprint available for measurement through neural and peripheral physiology, among other measures. These factors can determine drivers’ functional states and, thus, drivers’ availability to safely perform control transitions between human and vehicle. This doctoral project aims at investigating the potential of electrocardiogram (ECG), electrodermal activity (EDA) and functional near-infrared spectroscopy (fNIRS) as measures for a multimodal driver state monitoring (DSM) system for highly automated driving (i.e., SAE levels 3 and 4). While current DSM systems relying on gaze behaviour measures have proven valid and effective, several limitations and challenges could only be overcome using eye-tracking in tandem with physiological parameters. This thesis investigates whether ECG, EDA and fNIRS would be good candidates for such a purpose. Two driving simulator studies were performed to measure mental workload, trust in automation, stress and perceived risk, all identified as modulators of drivers’ functional states and that could eventually determine drivers’ availability to take-over manual control. The main findings demonstrate that DSM systems should adopt multiple physiological measures to capture changes in functional states relevant for driver readiness. Future DSM systems will benefit from the knowledge generated by this research by applying machine learning methods to these measures for determining drivers’ availability for optimal take-over performance