Driver Distraction Identification with an Ensemble of Convolutional Neural Networks

The World Health Organization (WHO) reported 1.25 million deaths yearly due to road traffic accidents worldwide and the number has been continuously increasing over the last few years. Nearly fifth of these accidents are caused by distracted drivers. Existing work of distracted driver detection is concerned with a small set of distractions (mostly, cell phone usage). Unreliable ad-hoc methods are often used.In this paper, we present the first publicly available dataset for driver distraction identification with more distraction postures than existing alternatives. In addition, we propose a reliable deep learning-based solution that achieves a 90% accuracy. The system consists of a genetically-weighted ensemble of convolutional neural networks, we show that a weighted ensemble of classifiers using a genetic algorithm yields in a better classification confidence. We also study the effect of different visual elements in distraction detection by means of face and hand localizations, and skin segmentation. Finally, we present a thinned version of our ensemble that could achieve 84.64% classification accuracy and operate in a real-time environment.Comment: arXiv admin note: substantial text overlap with arXiv:1706.0949

Sensing for HOV/HOT Lanes Enforcement

The use and creation of combined high-occupancy vehicle/high-occupancy toll (HOV/HOT Lanes) have become more common in urban areas since all types of road users can take advantage of the lane either as a high- occupancy vehicle or opting in to pay a congestion adjusted free. However, to maintain working integrity of the lanes for all users, stepped enforcement to discourage cheating has been needed as more lanes are added. This study evaluated the capability of a novel image sensor device to automate detection of in-vehicle occupants to flag law enforcement of HOV/HOT lane violators. The sensor device synchronously captures three co-registered images, one in the visible spectrum and two others in the infrared bands. The key idea is that the infrared bands can enhance correct occupancy detection through known phenomenological spectral properties of objects and humans residing inside the vehicle. Several experiments were conducted to determine this capability across varied conditions and scenarios to assess detection segmentation algorithms of vehicle passengers and drivers. Although occupancy detection through vehicle glass could be achieved in many cases, improvements must be made to such a detection system to increase robustness and reliability as a law enforcement tool. These improvements were guided by the experimental results, as well as suggested methods for deployment if this or similar technologies were to be deployed in the future

Sensing for HOV/HOT Lanes Enforcement

The use and creation of combined high-occupancy vehicle/high-occupancy toll (HOV/HOT Lanes) have become more common in urban areas since all types of road users can take advantage of the lane either as a high- occupancy vehicle or opting in to pay a congestion adjusted free. However, to maintain working integrity of the lanes for all users, stepped enforcement to discourage cheating has been needed as more lanes are added. This study evaluated the capability of a novel image sensor device to automate detection of in-vehicle occupants to flag law enforcement of HOV/HOT lane violators. The sensor device synchronously captures three co-registered images, one in the visible spectrum and two others in the infrared bands. The key idea is that the infrared bands can enhance correct occupancy detection through known phenomenological spectral properties of objects and humans residing inside the vehicle. Several experiments were conducted to determine this capability across varied conditions and scenarios to assess detection segmentation algorithms of vehicle passengers and drivers. Although occupancy detection through vehicle glass could be achieved in many cases, improvements must be made to such a detection system to increase robustness and reliability as a law enforcement tool. These improvements were guided by the experimental results, as well as suggested methods for deployment if this or similar technologies were to be deployed in the future

Deep Learning-based Driver Behavior Modeling and Analysis

Driving safety continues receiving widespread attention from car designers, safety regulators, and automotive research community as driving accidents due to driver distraction or fatigue have increased drastically over the years. In the past decades, there has been a remarkable push towards designing and developing new driver assistance systems with much better recognition and prediction capabilities. Equipped with various sensory systems, these Advanced Driver Assistance Systems (ADAS) are able to accurately perceive information on road conditions, predict traffic situations, estimate driving risks, and provide drivers with imminent warnings and visual assistance. In this thesis, we focus on two main aspects of driver behavior modeling in the design of new generation of ADAS. We first aim at improving the generalization ability of driver distraction recognition systems to diverse driving scenarios using the latest tools of machine learning and connectionist modeling, namely deep learning. To this end, we collect a large dataset of images on various driving situations of drivers from the Internet. Then we introduce Generative Adversarial Networks (GANs) as a data augmentation tool to enhance detection accuracy. A novel driver monitoring system is also introduced. This monitoring system combines multi-information resources, including a driver distraction recognition system, to assess the danger levels of driving situations. Moreover, this thesis proposes a multi-modal system for distraction recognition under various lighting conditions and presents a new Convolutional Neural Network (CNN) architecture, which can operate real-time on a resources-limited computational platform. The new CNN is built upon a novel network bottleneck of Depthwise Separable Convolution layers. The second part of this thesis focuses on driver maneuver prediction, which infers the direction a driver will turn to before a green traffic light is on and predicts accurately whether or not he/she will change the current driving lane. Here, a new method to label driving maneuver records is proposed, by which driving feature sequences for the training of prediction systems are more closely related to their labels. To this end, a new prediction system, which is based on Quasi-Recurrent Neural Networks, is introduced. In addition, and as an application of maneuver prediction, a novel driving proficiency assessment method is proposed. This method exploits the generalization abilities of different maneuver prediction systems to estimate drivers' driving abilities, and it demonstrates several advantages against existing assessment methods. In conjunction with the theoretical contribution, a series of comprehensive experiments are conducted, and the proposed methods are assessed against state-of-the-art works. The analysis of experimental results shows the improvement of results as compared with existing techniques

Why so serious? – Comparing two traffic conflict techniques for assessing encounters in shared space

In Germany, approximately 2.7 million crashes occurred in 2019. Especially vulnerable road users (VRU) have a high risk of being seriously injured or killed in traffic. Within the safe system approach, changes to the traffic infrastructure have been implemented to increase VRU safety. The creation of so-called shared spaces, in which all road users are encouraged to negotiate priority, is part of these efforts. Even though the concept has been known and applied for more than 40 years, comparatively little is known about interactions between different road users and methods to quantify interactions in shared spaces. The aim of this study is to investigate similarities and differences in quantifying the level of severity of encounters between pedestrians and motorised vehicles applying the Swedish traffic conflicts technique (STCT) and the pedestrian-vehicle conflicts analysis (PVCA). The STCT integrates the factors conflicting speed (CS) and time-to-accident (TA) to arrive at a severity level. In contrast, with four factors, the PVCA integrates more elements: time-to-collision (TTC, corresponding to TA), severity of evasive action, complexity of evasive action, and distance-to-collision (DTC). Trajectory and video data of a shared space were recorded using the Application Platform for Intelligent Mobile Units (AIM) in Ulm, Germany. 1364 interactions were randomly selected. Due to different exclusion criteria, such as interaction partners not being a car or pedestrian, missing values, and detection errors, 69 encounters were available for analyses. Using the PVCA, nine encounters were classified as critical and 60 as non-critical interactions. In contrast, computing the values based on the STCT, only three of the 69 encounters were categorised as critical. The results of a Spearman rank correlation did not show a significant correlation between the severity categories of the PVCA and severity levels of the STCT (r = 0.03, p = 0.78). An additional analysis of the encounters ranked as critical by the PVCA but as non-critical by the STCT showed that all six encounters had a large temporal distance (> 2 s) combined with very small spatial distance (< 5 m for vehicles and < 2.5 m for pedestrians). While the PVCA and STCT yielded similar results in most encounters, this could not be confirmed for all. Results indicate that spatial distance may contribute to the severity of encounters between pedestrians and vehicles in a shared space

Context-aware intelligent decisions: online assessment of heavy goods vehicle driving risk

There is a growing interest in assessing the impact of drivers' actions and behaviours on road safety due to the numerous road fatalities and costs attributed to them. For Heavy Goods Vehicle (HGV) drivers, assessing the road safety risks of their behaviours is a subject of interest for researchers, governments and transport companies, as nations rely on HGVs for the delivery of goods and services. However, HGV driving is a complex, dynamic, uncertain and multifaceted task, mostly influenced by individual traits and external contextual factors. Advanced computational and artificial intelligence (AI) methods have provided promising solutions to automatically characterise the manner by which drivers operate vehicle controls and assess their impact on road safety. However, several challenges and limitations are faced by the current intelligence-supported driving risk assessment approaches proposed by researchers, such as: (1) the lack of comprehensive driving risk datasets; (2) information about the impact of inevitable contextual factors on HGV drivers' responses is not considered, such as drivers' physical and mental states, weather conditions, traffic conditions, road geometry, road types, and work schedules; (3) ambiguity in the definition of driving behaviours is not considered; and (4) imprecision of AI models, and variability in experts' subjective views are not considered. To overcome the aforementioned challenges and limitations, this multidisciplinary research aims at exploring multiple sources of data including information about the impact of contextual factors captured from crucial stakeholders in the HGV sector to develop a reliable context-aware driving risk assessment framework. To achieve this aim, AI methods are explored to accurately detect drivers' driving styles, affective states and driving postures using telematics data, facial images, and driver posture images respectively. Subsequently, due to the lack of comprehensive driving risk datasets, fuzzy expert systems (FESs) are explored to fuse detected driving behaviours and perceived external factors using knowledge from domain experts. The key findings of this research are: (1) recurrent neural networks are effective in capturing the temporal dynamics and differences between the different types of driver distraction postures and affective states; (2) there is a trade-off between efficiency and privacy in processing facial images using AI approaches; (3) the fusion of driver behaviours and external factors using FESs produces realistic, reliable and fair driving risk assessments; and (4) a hierarchical representation of a decision-making process simplifies reasoning compared to flat representations

Context-aware intelligent decisions: online assessment of heavy goods vehicle driving risk

There is a growing interest in assessing the impact of drivers' actions and behaviours on road safety due to the numerous road fatalities and costs attributed to them. For Heavy Goods Vehicle (HGV) drivers, assessing the road safety risks of their behaviours is a subject of interest for researchers, governments and transport companies, as nations rely on HGVs for the delivery of goods and services. However, HGV driving is a complex, dynamic, uncertain and multifaceted task, mostly influenced by individual traits and external contextual factors. Advanced computational and artificial intelligence (AI) methods have provided promising solutions to automatically characterise the manner by which drivers operate vehicle controls and assess their impact on road safety. However, several challenges and limitations are faced by the current intelligence-supported driving risk assessment approaches proposed by researchers, such as: (1) the lack of comprehensive driving risk datasets; (2) information about the impact of inevitable contextual factors on HGV drivers' responses is not considered, such as drivers' physical and mental states, weather conditions, traffic conditions, road geometry, road types, and work schedules; (3) ambiguity in the definition of driving behaviours is not considered; and (4) imprecision of AI models, and variability in experts' subjective views are not considered. To overcome the aforementioned challenges and limitations, this multidisciplinary research aims at exploring multiple sources of data including information about the impact of contextual factors captured from crucial stakeholders in the HGV sector to develop a reliable context-aware driving risk assessment framework. To achieve this aim, AI methods are explored to accurately detect drivers' driving styles, affective states and driving postures using telematics data, facial images, and driver posture images respectively. Subsequently, due to the lack of comprehensive driving risk datasets, fuzzy expert systems (FESs) are explored to fuse detected driving behaviours and perceived external factors using knowledge from domain experts. The key findings of this research are: (1) recurrent neural networks are effective in capturing the temporal dynamics and differences between the different types of driver distraction postures and affective states; (2) there is a trade-off between efficiency and privacy in processing facial images using AI approaches; (3) the fusion of driver behaviours and external factors using FESs produces realistic, reliable and fair driving risk assessments; and (4) a hierarchical representation of a decision-making process simplifies reasoning compared to flat representations

Towards the “Perfect” Weather Warning

This book is about making weather warnings more effective in saving lives, property, infrastructure and livelihoods, but the underlying theme of the book is partnership. The book represents the warning process as a pathway linking observations to weather forecasts to hazard forecasts to socio-economic impact forecasts to warning messages to the protective decision, via a set of five bridges that cross the divides between the relevant organisations and areas of expertise. Each bridge represents the communication, translation and interpretation of information as it passes from one area of expertise to another and ultimately to the decision maker, who may be a professional or a member of the public. The authors explore the partnerships upon which each bridge is built, assess the expertise and skills that each partner brings and the challenges of communication between them, and discuss the structures and methods of working that build effective partnerships. The book is ordered according to the “first mile” paradigm in which the decision maker comes first, and then the production chain through the warning and forecast to the observations is considered second. This approach emphasizes the importance of co-design and co-production throughout the warning process. The book is targeted at professionals and trainee professionals with a role in the warning chain, i.e. in weather services, emergency management agencies, disaster risk reduction agencies, risk management sections of infrastructure agencies. This is an open access book