Driver Distraction Identification with an Ensemble of Convolutional Neural Networks

The World Health Organization (WHO) reported 1.25 million deaths yearly due to road traffic accidents worldwide and the number has been continuously increasing over the last few years. Nearly fifth of these accidents are caused by distracted drivers. Existing work of distracted driver detection is concerned with a small set of distractions (mostly, cell phone usage). Unreliable ad-hoc methods are often used.In this paper, we present the first publicly available dataset for driver distraction identification with more distraction postures than existing alternatives. In addition, we propose a reliable deep learning-based solution that achieves a 90% accuracy. The system consists of a genetically-weighted ensemble of convolutional neural networks, we show that a weighted ensemble of classifiers using a genetic algorithm yields in a better classification confidence. We also study the effect of different visual elements in distraction detection by means of face and hand localizations, and skin segmentation. Finally, we present a thinned version of our ensemble that could achieve 84.64% classification accuracy and operate in a real-time environment.Comment: arXiv admin note: substantial text overlap with arXiv:1706.0949

A Novel Driver Distraction Behavior Detection Based on Self-Supervised Learning Framework with Masked Image Modeling

Driver distraction causes a significant number of traffic accidents every year, resulting in economic losses and casualties. Currently, the level of automation in commercial vehicles is far from completely unmanned, and drivers still play an important role in operating and controlling the vehicle. Therefore, driver distraction behavior detection is crucial for road safety. At present, driver distraction detection primarily relies on traditional Convolutional Neural Networks (CNN) and supervised learning methods. However, there are still challenges such as the high cost of labeled datasets, limited ability to capture high-level semantic information, and weak generalization performance. In order to solve these problems, this paper proposes a new self-supervised learning method based on masked image modeling for driver distraction behavior detection. Firstly, a self-supervised learning framework for masked image modeling (MIM) is introduced to solve the serious human and material consumption issues caused by dataset labeling. Secondly, the Swin Transformer is employed as an encoder. Performance is enhanced by reconfiguring the Swin Transformer block and adjusting the distribution of the number of window multi-head self-attention (W-MSA) and shifted window multi-head self-attention (SW-MSA) detection heads across all stages, which leads to model more lightening. Finally, various data augmentation strategies are used along with the best random masking strategy to strengthen the model's recognition and generalization ability. Test results on a large-scale driver distraction behavior dataset show that the self-supervised learning method proposed in this paper achieves an accuracy of 99.60%, approximating the excellent performance of advanced supervised learning methods

Detection of Driver Drowsiness and Distraction Using Computer Vision and Machine Learning Approaches

Drowsiness and distracted driving are leading factor in most car crashes and near-crashes. This research study explores and investigates the applications of both conventional computer vision and deep learning approaches for the detection of drowsiness and distraction in drivers. In the first part of this MPhil research study conventional computer vision approaches was studied to develop a robust drowsiness and distraction system based on yawning detection, head pose detection and eye blinking detection. These algorithms were implemented by using existing human crafted features. Experiments were performed for the detection and classification with small image datasets to evaluate and measure the performance of system. It was observed that the use of human crafted features together with a robust classifier such as SVM gives better performance in comparison to previous approaches. Though, the results were satisfactorily, there are many drawbacks and challenges associated with conventional computer vision approaches, such as definition and extraction of human crafted features, thus making these conventional algorithms to be subjective in nature and less adaptive in practice. In contrast, deep learning approaches automates the feature selection process and can be trained to learn the most discriminative features without any input from human. In the second half of this research study, the use of deep learning approaches for the detection of distracted driving was investigated. It was observed that one of the advantages of the applied methodology and technique for distraction detection includes and illustrates the contribution of CNN enhancement to a better pattern recognition accuracy and its ability to learn features from various regions of a human body simultaneously. The comparison of the performance of four convolutional deep net architectures (AlexNet, ResNet, MobileNet and NASNet) was carried out, investigated triplet training and explored the impact of combining a support vector classifier (SVC) with a trained deep net. The images used in our experiments with the deep nets are from the State Farm Distracted Driver Detection dataset hosted on Kaggle, each of which captures the entire body of a driver. The best results were obtained with the NASNet trained using triplet loss and combined with an SVC. It was observed that one of the advantages of deep learning approaches are their ability to learn discriminative features from various regions of a human body simultaneously. The ability has enabled deep learning approaches to reach accuracy at human level.

Driver behavior classification and lateral control for automobile safety systems

Advanced driver assistance systems (ADAS) have been developed to help drivers maintain stability, improve road safety, and avoid potential collision. The data acquisition equipment that can be used to measure the state and parameter information of the vehicle may not be available for a standard passenger car due to economical and technical limitations. This work focuses on developing three technologies (longitudinal tire force estimation, driver behavior classification and lateral control) using low-cost sensors that can be utilized in ADAS. For the longitudinal tire force estimation, a low cost 1Hz positioning global system (GPS) and a steering angle sensor are used as the vehicle data acquisition equipment. A nonlinear extended two-wheel vehicle dynamic model is employed. The sideslip angle and the yaw rate are estimated by discrete Kalman Filter. A time independent piecewise optimization scheme is proposed to provide time-continuous estimates of longitude tire force, which can be transferred to the throttle/brake pedal position. The proposed method can be validated by the estimation results. Driver behavior classification systems can detect unsafe driver behavior and avoid potentially dangerous situations. To realize this strategy, a machine learning classification method, Gaussian Mixture model (GMM), is applied to classify driver behavior. In this application, a low cost 1Hz GPS receiver is considered as the vehicle data acquisition equipment instead of other more costly sensors (such as steering angle sensor, throttle/brake position sensor, and etc.). Since the driving information is limited, the nonlinear extended two-wheel vehicle dynamic model is adopted to reconstruct the driver behavior. Firstly, the sideslip angle and the yaw rate are calculated since they are not available from the GPS measurements. Secondly, a piecewise optimization scheme is proposed to reproduce the steering angle and the longitudinal force. Finally, a GMM classifier is trained to identify abnormal driver behavior. The simulation results demonstrated that the proposed scenario can detect the unsafe driver behavior effectively. The lateral control system developed in this study is a look-down reference system which uses a magnetic sensor at the front bumper to measure the front lateral displacement and a GPS to measure the vehicle\u27s heading orientation. Firstly, the steering angles can be estimated by using the data provided by the front magnetic sensor and GPS. The estimation algorithm is an observer for a new extended single-track model, in which the steering angle and its derivative are viewed as two state variables. Secondly, the road curvature is determined based on the linear relationship with respect to the steering angle. Thirdly, an accurate and real-time estimation of the vehicle\u27s lateral displacements can be accomplished according to a state observer. Finally, the closed loop controller is used as a compensator for automated steering. The proposed estimation and control algorithms are validated by simulation results. The results showed that this lateral steering control system achieved a good and robust performance for vehicles following or tracking a reference path

A systematic literature review on the relationship between autonomous vehicle technology and traffic-related mortality.

학위논문(석사) -- 서울대학교대학원 : 행정대학원 글로벌행정전공, 2023. 2. 최태현.The society is anticipated to gain a lot from Autonomous Vehicles (AV), such as improved traffic flow and a decrease in accidents. They heavily rely on improvements in various Artificial Intelligence (AI) processes and strategies. Though some researchers in this field believe AV is the key to enhancing safety, others believe AV creates new challenges when it comes to ensuring the security of these new technology/systems and applications. The article conducts a systematic literature review on the relationship between autonomous vehicle technology and traffic-related mortality. According to inclusion and exclusion criteria, articles from EBSCO, ProQuest, IEEE Explorer, Web of Science were chosen, and they were then sorted. The findings reveal that the most of these publications have been published in advanced transport-related journals. Future improvements in the automobile industry and the development of intelligent transportation systems could help reduce the number of fatal traffic accidents. Technologies for autonomous cars provide effective ways to enhance the driving experience and reduce the number of traffic accidents. A multitude of driving-related problems, such as crashes, traffic, energy usage, and environmental pollution, will be helped by autonomous driving technology. More research is needed for the significant majority of the studies that were assessed. They need to be expanded so that they can be tested in real-world or computer-simulated scenarios, in better and more realistic scenarios, with better and more data, and in experimental designs where the results of the proposed strategy are compared to those of industry standards and competing strategies. Therefore, additional study with improved methods is needed. Another major area that requires additional research is the moral and ethical choices made by AVs. Government, policy makers, manufacturers, and designers all need to do many actions in order to deploy autonomous vehicles on the road effectively. The government should develop laws, rules, and an action plan in particular. It is important to create more effective programs that might encourage the adoption of emerging technology in transportation systems, such as driverless vehicles. In this regard, user perception becomes essential since it may inform designers about current issues and observations made by people. The perceptions of autonomous car users in developing countries like Azerbaijan haven't been thoroughly studied up to this point. The manufacturer has to fix the system flaw and needs a good data set for efficient operation. In the not-too-distant future, the widespread use of highly automated vehicles (AVs) may open up intriguing new possibilities for resolving persistent issues in current safety-related research. Further research is required to better understand and quantify the significant policy implications of Avs, taking into consideration factors like penetration rate, public adoption, technological advancements, traffic patterns, and business models. It only needs to take into account peer-reviewed, full-text journal papers for the investigation, but it's clear that a larger database and more documents would provide more results and a more thorough analysis.자율주행차(AV)를 통해 교통 흐름이 개선되고 사고가 줄어드는 등 사회가 얻는 것이 많을 것으로 예상된다. 그들은 다양한 인공지능(AI) 프로세스와 전략의 개선에 크게 의존한다. 이 분야의 일부 연구자들은 AV가 안전성을 향상시키는 열쇠라고 믿지만, 다른 연구자들은 AV가 이러한 새로운 기술/시스템 및 애플리케이션의 보안을 보장하는 것과 관련하여 새로운 문제를 야기한다고 믿는다. 이 논문은 자율주행차 기술과 교통 관련 사망률 사이의 관계에 대한 체계적인 문헌 검토를 수행한다. 포함 및 제외 기준에 따라 EBSCO, ProQuest, IEEE Explorer 및 Web of Science의 기사를 선택하고 분류했다.연구 결과는 이러한 출판물의 대부분이 고급 운송 관련 저널에 게재되었음을 보여준다. 미래의 자동차 산업의 개선과 지능형 교통 시스템의 개발은 치명적인 교통 사고의 수를 줄이는 데 도움이 될 수 있다. 자율주행 자동차 기술은 운전 경험을 향상시키고 교통 사고의 수를 줄일 수 있는 효과적인 방법을 제공한다. 충돌, 교통, 에너지 사용, 환경 오염과 같은 수많은 운전 관련 문제들은 자율 주행 기술에 의해 도움을 받을 것이다. 평가된 대부분의 연구에 대해 더 많은 연구가 필요하다. 실제 또는 컴퓨터 시뮬레이션 시나리오, 더 좋고 현실적인 시나리오, 더 좋고 더 많은 데이터, 그리고 제안된 전략 결과가 산업 표준 및 경쟁 전략의 결과와 비교되는 실험 설계에서 테스트될 수 있도록 확장되어야 한다. 따라서 개선된 방법에 대한 추가 연구가 필요하다. 추가 연구가 필요한 또 다른 주요 분야는 AV의 도덕적, 윤리적 선택이다. 정부, 정책 입안자, 제조업체 및 설계자는 모두 자율 주행 차량을 효과적으로 도로에 배치하기 위해 많은 조치를 취해야 한다. 정부는 특히 법, 규칙, 실행 계획을 개발해야 한다. 운전자 없는 차량과 같은 운송 시스템에서 새로운 기술의 채택을 장려할 수 있는 보다 효과적인 프로그램을 만드는 것이 중요하다. 이와 관련하여, 설계자에게 현재 이슈와 사람에 의한 관찰을 알려줄 수 있기 때문에 사용자 인식이 필수적이 된다.제조업체는 시스템 결함을 수정해야 하며 효율적인 작동을 위해 좋은 데이터 세트가 필요하다. 멀지 않은 미래에, 고도로 자동화된 차량(AV)의 광범위한 사용은 현재의 안전 관련 연구에서 지속적인 문제를 해결하기 위한 흥미로운 새로운 가능성을 열어줄 수 있다. 보급률, 공공 채택, 기술 발전, 교통 패턴 및 비즈니스 모델과 같은 요소를 고려하여 Avs의 중요한 정책 영향을 더 잘 이해하고 정량화하기 위한 추가 연구가 필요하다. 조사를 위해 동료 검토를 거친 전문 저널 논문만 고려하면 되지만, 데이터베이스가 커지고 문서가 많아지면 더 많은 결과와 더 철저한 분석이 제공될 것이 분명하다.Abstract 3 Table of Contents 6 List of Tables 7 List of Figures 7 List of Appendix 7 CHAPTER 1: INTRODUCTION 8 1.1. Background 8 1.2. Purpose of Research 13 CHAPTER 2: AUTONOMOUS VEHICLES 21 2.1. Intelligent Traffic Systems 21 2.2. System Architecture for Autonomous Vehicles 22 2.3. Key components in AV classification 27 CHAPTER 3: METHODOLOGY AND DATA COLLECTION PROCEDURE 35 CHAPTER 4: FINDINGS AND DISCUSSION 39 4.1. RQ1: Do autonomous vehicles reduce traffic-related deaths 40 4.2. RQ2: Are there any challenges to using autonomous vehicles 63 4.3. RQ3: As a developing country, how effective is the use of autonomous vehicles for reducing traffic mortality 72 CHAPTER 5: CONCLUSION 76 5.1. Summary 76 5.2. Implications and Recommendations 80 5.3. Limitation of the study 91 Bibliography 93 List of Tables Table 1: The 6 Levels of Autonomous Vehicles Table 2: Search strings Table 3: Inclusion and exclusion criteria List of Figures Figure 1: Traffic Death Comparison with Europe Figure 2: Research strategy and study selection process List of Appendix Appendix 1: List of selected articles석