Car that Knows Before You Do: Anticipating Maneuvers via Learning Temporal Driving Models

Advanced Driver Assistance Systems (ADAS) have made driving safer over the last decade. They prepare vehicles for unsafe road conditions and alert drivers if they perform a dangerous maneuver. However, many accidents are unavoidable because by the time drivers are alerted, it is already too late. Anticipating maneuvers beforehand can alert drivers before they perform the maneuver and also give ADAS more time to avoid or prepare for the danger. In this work we anticipate driving maneuvers a few seconds before they occur. For this purpose we equip a car with cameras and a computing device to capture the driving context from both inside and outside of the car. We propose an Autoregressive Input-Output HMM to model the contextual information alongwith the maneuvers. We evaluate our approach on a diverse data set with 1180 miles of natural freeway and city driving and show that we can anticipate maneuvers 3.5 seconds before they occur with over 80\% F1-score in real-time.Comment: ICCV 2015, http://brain4cars.co

Owl and Lizard: Patterns of Head Pose and Eye Pose in Driver Gaze Classification

Accurate, robust, inexpensive gaze tracking in the car can help keep a driver safe by facilitating the more effective study of how to improve (1) vehicle interfaces and (2) the design of future Advanced Driver Assistance Systems. In this paper, we estimate head pose and eye pose from monocular video using methods developed extensively in prior work and ask two new interesting questions. First, how much better can we classify driver gaze using head and eye pose versus just using head pose? Second, are there individual-specific gaze strategies that strongly correlate with how much gaze classification improves with the addition of eye pose information? We answer these questions by evaluating data drawn from an on-road study of 40 drivers. The main insight of the paper is conveyed through the analogy of an "owl" and "lizard" which describes the degree to which the eyes and the head move when shifting gaze. When the head moves a lot ("owl"), not much classification improvement is attained by estimating eye pose on top of head pose. On the other hand, when the head stays still and only the eyes move ("lizard"), classification accuracy increases significantly from adding in eye pose. We characterize how that accuracy varies between people, gaze strategies, and gaze regions.Comment: Accepted for Publication in IET Computer Vision. arXiv admin note: text overlap with arXiv:1507.0476

DETECTION OF DRIVER’S VISUAL DISTRACTION USING DUAL CAMERAS

Most serious accidents are caused by the driver’s visual distraction. Therefore, early detection of a driver’s visual distraction is very important. The detection system mostly used is the dashboard camera because it is cheap and convenient. However, some studies have focused on various methods using additional equipment such as vehicle-mounted devices, wearable devices, and specific cameras that are common. However, these proposals are expensive. Therefore, the main goal of our research is to create a low-cost, non-intrusive, and lightweight driver’s visual distraction detection (DVDD) system using only a simple dual dashboard camera. Currently, most research has focused only on tracking and estimating the driver’s gaze. In our study, additionally, we also aim to monitor the road environment and then evaluate the driver’s visual distraction detection based on the two pieces of information. The proposed system has two main modules: 1) gaze mapping and 2) moving object detection. The gaze mapping module receives video captured through a camera placed in front of the driver, and then predicts a driver’s gaze direction to one of predefined 16 gaze regions. Concurrently, the moving object detection module identifies the moving objects from the front view and determines in which part of the predefined 16 gaze regions it appears. By combining and evaluating the two modules, the state of the distraction of the driver can be estimated. If the two module outputs are different gaze regions or non-neighbor gaze regions, the system considers that the driver is visually distracted and issues a warning. We conducted experiments based on our self-built real-driving DriverGazeMapping dataset. In the gaze mapping module, we compared the two methods MobileNet and OpenFace with the SVM classifier. The two methods outperformed the baseline gaze mapping module. Moreover, in the OpenFace with SVM classifier method, we investigated which features extracted by OpenFace affected the performance of the gaze mapping module. Of these, the most effective feature was the combination of a gaze angle and head position_R features. The OpenFace with SVM method using gaze angle and head position_R features achieved a 6.25% higher accuracy than the method using MobileNet. Besides, the moving object detection module using the Lukas-Kanade dense method was faster and more reliable than in the previous study in our experiments

Evaluation and Validation of Distraction Detection Algorithms on Multiple Data Sources

This study aims to evaluate algorithms designed to detect distracted driving. This includes the comparison of how efficiently they detect the state of distraction and likelihood of a crash. Four algorithms that utilize measures of cumulative glance, past glance behavior, and glance eccentricity were used to understand the distracted state of the driver and were validated on two separate data sources (i.e., simulator and naturalistic data). Additionally, an independent method for distraction detection was designed using data mining methods. This approach utilized measures like steering degree, lane offset, lateral and longitudinal velocity, and acceleration. The results showed a higher likelihood of distracted events when cumulative glances were considered. However, the state of distraction was observed to be higher when glance eccentricity was added. Additionally, it was observed that glance behavior using the four legacy algorithms were better detectors of the state of distraction as compared to the data mining method that used vehicular measures. This research has implications in understanding the state of distraction, predicting the power of different methods, and comparing approaches in different contexts (naturalistic vs simulator). These findings provide the fundamental building blocks towards designing advanced mitigation systems that give drivers feedback in instances of high crash likelihood

Physiological-based Driver Monitoring Systems: A Scoping Review

A physiological-based driver monitoring system (DMS) has attracted research interest and has great potential for providing more accurate and reliable monitoring of the driver’s state during a driving experience. Many driving monitoring systems are driver behavior-based or vehicle-based. When these non-physiological based DMS are coupled with physiological-based data analysis from electroencephalography (EEG), electrooculography (EOG), electrocardiography (ECG), and electromyography (EMG), the physical and emotional state of the driver may also be assessed. Drivers’ wellness can also be monitored, and hence, traffic collisions can be avoided. This paper highlights work that has been published in the past five years related to physiological-based DMS. Specifically, we focused on the physiological indicators applied in DMS design and development. Work utilizing key physiological indicators related to driver identification, driver alertness, driver drowsiness, driver fatigue, and drunk driver is identified and described based on the PRISMA Extension for Scoping Reviews (PRISMA-Sc) Framework. The relationship between selected papers is visualized using keyword co-occurrence. Findings were presented using a narrative review approach based on classifications of DMS. Finally, the challenges of physiological-based DMS are highlighted in the conclusion. Doi: 10.28991/CEJ-2022-08-12-020 Full Text: PD

Multi-sensor movement analysis for transport safety and health applications

Recent increases in the use of and applications for wearable technology has opened up many new avenues of research. In this paper, we consider the use of lifelogging and GPS data to extend fine-grained movement analysis for improving applications in health and safety. We first design a framework to solve the problem of indoor and outdoor movement detection from sensor readings associated with images captured by a lifelogging wearable device. Second we propose a set of measures related with hazard on the road network derived from the combination of GPS movement data, road network data and the sensor readings from a wearable device. Third, we identify the relationship between different socio-demographic groups and the patterns of indoor physical activity and sedentary behaviour routines as well as disturbance levels on different road settings

Prediction of drivers’ performance in highly automated vehicles

Purpose: The aim of this research was to assess the predictability of driver’s response to critical hazards during the transition from automated to manual driving in highly automated vehicles using their physiological data.Method: A driving simulator experiment was conducted to collect drivers’ physiological data before, during and after the transition from automated to manual driving. A total of 33 participants between 20 and 30 years old were recruited. Participants went through a driving scenario under the influence of different non-driving related tasks. The repeated measures approach was used to assess the effect of repeatability on the driver’s physiological data. Statistical and machine learning methods were used to assess the predictability of drivers’ response quality based on their physiological data collected before responding to a critical hazard. Findings: - The results showed that the observed physiological data that was gathered before the transition formed strong indicators of the drivers’ ability to respond successfully to a potential hazard after the transition. In addition, physiological behaviour was influenced by driver’s secondary tasks engagement and correlated with the driver’s subjective measures to the difficulty of the task. The study proposes new quality measures to assess the driver’s response to critical hazards in highly automated driving. Machine learning results showed that response time is predictable using regression methods. In addition, the classification methods were able to classify drivers into low, medium and high-risk groups based on their quality measures values. Research Implications: Proposed models help increase the safety of automated driving systems by providing insights into the drivers’ ability to respond to future critical hazards. More research is required to find the influence of age, drivers’ experience of the automated vehicles and traffic density on the stability of the proposed models. Originality: The main contribution to knowledge of this study is the feasibility of predicting drivers’ ability to respond to critical hazards using the physiological behavioural data collected before the transition from automated to manual driving. With the findings, automation systems could change the transition time based on the driver’s physiological state to allow for the safest transition possible. In addition, it provides an insight into driver’s readiness and therefore, allows the automated system to adopt the correct driving strategy and plan to enhance drivers experience and make the transition phase safer for everyone.</div