A Fuzzy-Logic Approach to Dynamic Bayesian Severity Level Classification of Driver Distraction Using Image Recognition

open access articleDetecting and classifying driver distractions is crucial in the prevention of road accidents. These distractions impact both driver behavior and vehicle dynamics. Knowing the degree of driver distraction can aid in accident prevention techniques, including transitioning of control to a level 4 semi- autonomous vehicle, when a high distraction severity level is reached. Thus, enhancement of Advanced Driving Assistance Systems (ADAS) is a critical component in the safety of vehicle drivers and other road users. In this paper, a new methodology is introduced, using an expert knowledge rule system to predict the severity of distraction in a contiguous set of video frames using the Naturalistic Driving American University of Cairo (AUC) Distraction Dataset. A multi-class distraction system comprises the face orientation, drivers’ activities, hands and previous driver distraction, a severity classification model is developed as a discrete dynamic Bayesian (DDB). Furthermore, a Mamdani-based fuzzy system was implemented to detect multi- class of distractions into a severity level of safe, careless or dangerous driving. Thus, if a high level of severity is reached the semi-autonomous vehicle will take control. The result further shows that some instances of driver’s distraction may quickly transition from a careless to dangerous driving in a multi-class distraction context

Context-Aware Driver Distraction Severity Classification using LSTM Network

Advanced Driving Assistance Systems (ADAS) has been a critical component in vehicles and vital to the safety of vehicle drivers and public road transportation systems. In this paper, we present a deep learning technique that classifies drivers’ distraction behaviour using three contextual awareness parameters: speed, manoeuver and event type. Using a video coding taxonomy, we study drivers’ distractions based on events information from Regions of Interest (RoI) such as hand gestures, facial orientation and eye gaze estimation. Furthermore, a novel probabilistic (Bayesian) model based on the Long shortterm memory (LSTM) network is developed for classifying driver’s distraction severity. This paper also proposes the use of frame-based contextual data from the multi-view TeleFOT naturalistic driving study (NDS) data monitoring to classify the severity of driver distractions. Our proposed methodology entails recurrent deep neural network layers trained to predict driver distraction severity from time series data

Monitoring fatigue and drowsiness in motor vehicle occupants using electrocardiogram and heart rate - A systematic review

Introdução: A fadiga é um estado complexo que pode resultar em diminuição da vigilância, frequentemente acompanhada de sonolência. A fadiga durante a condução contribui significativamente para acidentes de trânsito em todo o mundo, destacando-se a necessidade de técnicas de monitorização eficazes. Existem várias tecnologias para aumentar a segurança do condutor e reduzir os riscos de acidentes, como sistemas de deteção de fadiga que podem alertar os condutores à medida que a sonolência se instala. Em particular, a análise dos padrões de frequência cardíaca pode oferecer informações valiosas sobre a condição fisiológica e o nível de vigilância do condutor, permitindo-lhe compreender os seus níveis de fadiga. Esta revisão tem como objetivo estabelecer o estado atual das estratégias de monitorização para ocupantes de veículos, com foco específico na avaliação da fadiga pela frequência cardíaca e variabilidade da frequência cardíaca. Métodos: Realizamos uma pesquisa sistemática da literatura nas bases de dados Web of Science, SCOPUS e Pubmed, utilizando os termos veículo, condutor, monitoração fisiológica, fadiga, sono, eletrocardiograma, frequência cardíaca e variabilidade da frequência cardíaca. Examinamos artigos publicados entre 1 de janeiro de 2018 e 31 de janeiro de 2023. Resultados: Um total de 371 artigos foram identificados, dos quais 71 foram incluídos neste estudo. Entre os artigos incluídos, 57 utilizam o eletrocardiograma (ECG) como sinal adquirido para medir a frequência cardíaca, sendo que a maioria das leituras de ECG foi obtida através de sensores de contacto (n=41), seguidos por sensores vestíveis não invasivos (n=11). Relativamente à validação, 23 artigos não mencionam qualquer tipo de validação, enquanto a maioria se baseia em avaliações subjetivas de fadiga relatadas pelos próprios participantes (n=27) e avaliações feitas por observadores com base em vídeos (n=11). Dos artigos incluídos, apenas 14 englobam um sistema de estimativa de fadiga e sonolência. Alguns relatam um desempenho satisfatórios, no entanto, o tamanho reduzido da amostra limita a abrangência de quaisquer conclusões. Conclusão: Esta revisão destaca o potencial da análise da frequência cardíaca e da instrumentação não invasiva para a monitorização contínua do estado do condutor e deteção de sonolência. Uma das principais questões é a falta de métodos suficientes de validação e estimativa para a fadiga, o que contribui para a insuficiência dos métodos na criação de sistemas de alarme proativos. Esta área apresenta grandes perspetivas, mas ainda está longe de ser implementada de forma fiável.Background: Fatigue is a complex state that can result in decreased alertness, often accompanied by drowsiness. Driving fatigue has become a significant contributor to traffic accidents globally, highlighting the need for effective monitoring techniques. Various technologies exist to enhance driver safety and minimize accident risks, such as fatigue detection systems that can alert drivers as drowsiness sets in. In particular, measuring heart rate patterns may offer valuable insights into the occupant's physiological condition and level of alertness, and may allow them to understand their fatigue levels. This review aims to establish the current state of the art of monitoring strategies for vehicle occupants, specifically focusing on fatigue assessed by heart rate and heart rate variability. Methods: We performed a systematic literature search in the databases of Web Of Science, SCOPUS and Pubmed, using the terms vehicle, driver, physiologic monitoring, fatigue, sleep, electrocardiogram, heart rate and heart rate variability. We examine articles published between 1st of january 2018 and 31st of January 2023. Results: A total of 371 papers were identified from which 71 articles were included in this study. Among the included papers, 57 utilized electrocardiogram (ECG) as the acquired signal for heart rate (HR) measures, with most ECG readings obtained through contact sensors (n=41), followed by non-intrusive wearable sensors (n=11). Regarding validation, 23 papers do not report validation, while the majority rely on subjective self-reported fatigue ratings (n=27) and video-based observer ratings(n=11). From the included papers, only 14 comprise a fatigue and drowsiness estimation system. Some report acceptable performances, but reduced sample size limits the reach of any conclusions. Conclusions: This review highlights the potential of HR analysis and non-intrusive instrumentation for continuous monitoring of driver's status and detecting sleepiness. One major issue is the lack of sufficient validation and estimation methods for fatigue, contributing to the insufficiency of methods in providing proactive alarm systems. This area shows great promise but is still far from being reliably implemented

A review of technologies for heart attack monitoring systems

Every year, approximately 1.35 million people die in car accidents. One of the causes of traffic accidents is a heart attack while driving. Common heart attack warning signs are pain or discomfort in the chest or one or both arms or shoulders, light-headedness, faintness, cold sweat, and shortness of breath. When having a heart attack, a car driver has strong pain in the centre or left side of the chest. Current technology for heart attack detection is based on sensory signal properties such as the electrocardiogram (ECG), heart rate and oxygen saturation (SpO2). This paper is intended to give the readers an overview of technologies for heart attack monitoring system that has been used at the hospital, at the home and in the vehicle. The result shows that ECG, heart rate and SpO2 properties are mostly used by numerous researchers for heart attack monitoring systems at hospitals. Meanwhile, many researchers developed a system by using heart rate, ECG, SpO2 and images as properties for heart attack monitoring systems at home. Existing technologies for heart attack monitoring systems in the vehicle used heart rate and ECG as properties in a system. However, there are no review papers yet on heart attack monitoring systems using image processing in vehicles. We believe that researchers and practitioners will embrace this technology by addressing image processing in the heart attack monitoring system in vehicles

Multimodal Features for Detection of Driver Stress and Fatigue: Review

Driver fatigue and stress significantly contribute to higher number of car accidents worldwide. Although, different detection approaches have been already commercialized and used by car producers (and third party companies), research activities in this field are still needed in order to increase the reliability of these alert systems. Also, in the context of automated driving, the driver mental state assessment will be an important part of cars in future. This paper presents state-of-the-art review of different approaches for driver fatigue and stress detection and evaluation. We describe in details various signals (biological, car and video) and derived features used for these tasks and we discuss their relevance and advantages. In order to make this review complete, we also describe different datasets, acquisition systems and experiment scenarios

A Context Aware Classification System for Monitoring Driver’s Distraction Levels

Understanding the safety measures regarding developing self-driving futuristic cars is a concern for decision-makers, civil society, consumer groups, and manufacturers. The researchers are trying to thoroughly test and simulate various driving contexts to make these cars fully secure for road users. Including the vehicle’ surroundings offer an ideal way to monitor context-aware situations and incorporate the various hazards. In this regard, different studies have analysed drivers’ behaviour under different case scenarios and scrutinised the external environment to obtain a holistic view of vehicles and the environment. Studies showed that the primary cause of road accidents is driver distraction, and there is a thin line that separates the transition from careless to dangerous. While there has been a significant improvement in advanced driver assistance systems, the current measures neither detect the severity of the distraction levels nor the context-aware, which can aid in preventing accidents. Also, no compact study provides a complete model for transitioning control from the driver to the vehicle when a high degree of distraction is detected. The current study proposes a context-aware severity model to detect safety issues related to driver’s distractions, considering the physiological attributes, the activities, and context-aware situations such as environment and vehicle. Thereby, a novel three-phase Fast Recurrent Convolutional Neural Network (Fast-RCNN) architecture addresses the physiological attributes. Secondly, a novel two-tier FRCNN-LSTM framework is devised to classify the severity of driver distraction. Thirdly, a Dynamic Bayesian Network (DBN) for the prediction of driver distraction. The study further proposes the Multiclass Driver Distraction Risk Assessment (MDDRA) model, which can be adopted in a context-aware driving distraction scenario. Finally, a 3-way hybrid CNN-DBN-LSTM multiclass degree of driver distraction according to severity level is developed. In addition, a Hidden Markov Driver Distraction Severity Model (HMDDSM) for the transitioning of control from the driver to the vehicle when a high degree of distraction is detected. This work tests and evaluates the proposed models using the multi-view TeleFOT naturalistic driving study data and the American University of Cairo dataset (AUCD). The evaluation of the developed models was performed using cross-correlation, hybrid cross-correlations, K-Folds validation. The results show that the technique effectively learns and adopts safety measures related to the severity of driver distraction. In addition, the results also show that while a driver is in a dangerous distraction state, the control can be shifted from driver to vehicle in a systematic manner

A systematic literature review on the relationship between autonomous vehicle technology and traffic-related mortality.

학위논문(석사) -- 서울대학교대학원 : 행정대학원 글로벌행정전공, 2023. 2. 최태현.The society is anticipated to gain a lot from Autonomous Vehicles (AV), such as improved traffic flow and a decrease in accidents. They heavily rely on improvements in various Artificial Intelligence (AI) processes and strategies. Though some researchers in this field believe AV is the key to enhancing safety, others believe AV creates new challenges when it comes to ensuring the security of these new technology/systems and applications. The article conducts a systematic literature review on the relationship between autonomous vehicle technology and traffic-related mortality. According to inclusion and exclusion criteria, articles from EBSCO, ProQuest, IEEE Explorer, Web of Science were chosen, and they were then sorted. The findings reveal that the most of these publications have been published in advanced transport-related journals. Future improvements in the automobile industry and the development of intelligent transportation systems could help reduce the number of fatal traffic accidents. Technologies for autonomous cars provide effective ways to enhance the driving experience and reduce the number of traffic accidents. A multitude of driving-related problems, such as crashes, traffic, energy usage, and environmental pollution, will be helped by autonomous driving technology. More research is needed for the significant majority of the studies that were assessed. They need to be expanded so that they can be tested in real-world or computer-simulated scenarios, in better and more realistic scenarios, with better and more data, and in experimental designs where the results of the proposed strategy are compared to those of industry standards and competing strategies. Therefore, additional study with improved methods is needed. Another major area that requires additional research is the moral and ethical choices made by AVs. Government, policy makers, manufacturers, and designers all need to do many actions in order to deploy autonomous vehicles on the road effectively. The government should develop laws, rules, and an action plan in particular. It is important to create more effective programs that might encourage the adoption of emerging technology in transportation systems, such as driverless vehicles. In this regard, user perception becomes essential since it may inform designers about current issues and observations made by people. The perceptions of autonomous car users in developing countries like Azerbaijan haven't been thoroughly studied up to this point. The manufacturer has to fix the system flaw and needs a good data set for efficient operation. In the not-too-distant future, the widespread use of highly automated vehicles (AVs) may open up intriguing new possibilities for resolving persistent issues in current safety-related research. Further research is required to better understand and quantify the significant policy implications of Avs, taking into consideration factors like penetration rate, public adoption, technological advancements, traffic patterns, and business models. It only needs to take into account peer-reviewed, full-text journal papers for the investigation, but it's clear that a larger database and more documents would provide more results and a more thorough analysis.자율주행차(AV)를 통해 교통 흐름이 개선되고 사고가 줄어드는 등 사회가 얻는 것이 많을 것으로 예상된다. 그들은 다양한 인공지능(AI) 프로세스와 전략의 개선에 크게 의존한다. 이 분야의 일부 연구자들은 AV가 안전성을 향상시키는 열쇠라고 믿지만, 다른 연구자들은 AV가 이러한 새로운 기술/시스템 및 애플리케이션의 보안을 보장하는 것과 관련하여 새로운 문제를 야기한다고 믿는다. 이 논문은 자율주행차 기술과 교통 관련 사망률 사이의 관계에 대한 체계적인 문헌 검토를 수행한다. 포함 및 제외 기준에 따라 EBSCO, ProQuest, IEEE Explorer 및 Web of Science의 기사를 선택하고 분류했다.연구 결과는 이러한 출판물의 대부분이 고급 운송 관련 저널에 게재되었음을 보여준다. 미래의 자동차 산업의 개선과 지능형 교통 시스템의 개발은 치명적인 교통 사고의 수를 줄이는 데 도움이 될 수 있다. 자율주행 자동차 기술은 운전 경험을 향상시키고 교통 사고의 수를 줄일 수 있는 효과적인 방법을 제공한다. 충돌, 교통, 에너지 사용, 환경 오염과 같은 수많은 운전 관련 문제들은 자율 주행 기술에 의해 도움을 받을 것이다. 평가된 대부분의 연구에 대해 더 많은 연구가 필요하다. 실제 또는 컴퓨터 시뮬레이션 시나리오, 더 좋고 현실적인 시나리오, 더 좋고 더 많은 데이터, 그리고 제안된 전략 결과가 산업 표준 및 경쟁 전략의 결과와 비교되는 실험 설계에서 테스트될 수 있도록 확장되어야 한다. 따라서 개선된 방법에 대한 추가 연구가 필요하다. 추가 연구가 필요한 또 다른 주요 분야는 AV의 도덕적, 윤리적 선택이다. 정부, 정책 입안자, 제조업체 및 설계자는 모두 자율 주행 차량을 효과적으로 도로에 배치하기 위해 많은 조치를 취해야 한다. 정부는 특히 법, 규칙, 실행 계획을 개발해야 한다. 운전자 없는 차량과 같은 운송 시스템에서 새로운 기술의 채택을 장려할 수 있는 보다 효과적인 프로그램을 만드는 것이 중요하다. 이와 관련하여, 설계자에게 현재 이슈와 사람에 의한 관찰을 알려줄 수 있기 때문에 사용자 인식이 필수적이 된다.제조업체는 시스템 결함을 수정해야 하며 효율적인 작동을 위해 좋은 데이터 세트가 필요하다. 멀지 않은 미래에, 고도로 자동화된 차량(AV)의 광범위한 사용은 현재의 안전 관련 연구에서 지속적인 문제를 해결하기 위한 흥미로운 새로운 가능성을 열어줄 수 있다. 보급률, 공공 채택, 기술 발전, 교통 패턴 및 비즈니스 모델과 같은 요소를 고려하여 Avs의 중요한 정책 영향을 더 잘 이해하고 정량화하기 위한 추가 연구가 필요하다. 조사를 위해 동료 검토를 거친 전문 저널 논문만 고려하면 되지만, 데이터베이스가 커지고 문서가 많아지면 더 많은 결과와 더 철저한 분석이 제공될 것이 분명하다.Abstract 3 Table of Contents 6 List of Tables 7 List of Figures 7 List of Appendix 7 CHAPTER 1: INTRODUCTION 8 1.1. Background 8 1.2. Purpose of Research 13 CHAPTER 2: AUTONOMOUS VEHICLES 21 2.1. Intelligent Traffic Systems 21 2.2. System Architecture for Autonomous Vehicles 22 2.3. Key components in AV classification 27 CHAPTER 3: METHODOLOGY AND DATA COLLECTION PROCEDURE 35 CHAPTER 4: FINDINGS AND DISCUSSION 39 4.1. RQ1: Do autonomous vehicles reduce traffic-related deaths 40 4.2. RQ2: Are there any challenges to using autonomous vehicles 63 4.3. RQ3: As a developing country, how effective is the use of autonomous vehicles for reducing traffic mortality 72 CHAPTER 5: CONCLUSION 76 5.1. Summary 76 5.2. Implications and Recommendations 80 5.3. Limitation of the study 91 Bibliography 93 List of Tables Table 1: The 6 Levels of Autonomous Vehicles Table 2: Search strings Table 3: Inclusion and exclusion criteria List of Figures Figure 1: Traffic Death Comparison with Europe Figure 2: Research strategy and study selection process List of Appendix Appendix 1: List of selected articles석

Investigation of low-cost infrared sensing for intelligent deployment of occupant restraints

In automotive transport, airbags and seatbelts are effective at restraining the driver and passenger in the event of a crash, with statistics showing a dramatic reduction in the number of casualties from road crashes. However, statistics also show that a small number of these people have been injured or even killed from striking the airbag, and that the elderly and small children are especially at risk of airbag-related injury. This is the result of the fact that in-car restraint systems were designed for the average male at an average speed of 50 km/hr, and people outside these norms are at risk. Therefore one of the future safety goals of the car manufacturers is to deploy sensors that would gain more information about the driver or passenger of their cars in order to tailor the safety systems specifically for that person, and this is the goal of this project. This thesis describes a novel approach to occupant detection, position measurement and monitoring using a low-cost thermal imaging based system, which is a departure from traditional video camera-based systems, and at an affordable price. Experiments were carried out using a specially designed test rig and a car driving simulator with members of the public. Results have shown that the thermal imager can detect a human in a car cabin mock up and provide crucial real-time position data, which could be used to support intelligent restraint deployment. Other valuable information has been detected such as whether the driver is smoking, drinking a hot or cold drink, using a mobile phone, which can help to infer the level of driver attentiveness or engagement

Face Emotion Recognition Based on Machine Learning: A Review

Computers can now detect, understand, and evaluate emotions thanks to recent developments in machine learning and information fusion. Researchers across various sectors are increasingly intrigued by emotion identification, utilizing facial expressions, words, body language, and posture as means of discerning an individual's emotions. Nevertheless, the effectiveness of the first three methods may be limited, as individuals can consciously or unconsciously suppress their true feelings. This article explores various feature extraction techniques, encompassing the development of machine learning classifiers like k-nearest neighbour, naive Bayesian, support vector machine, and random forest, in accordance with the established standard for emotion recognition. The paper has three primary objectives: firstly, to offer a comprehensive overview of effective computing by outlining essential theoretical concepts; secondly, to describe in detail the state-of-the-art in emotion recognition at the moment; and thirdly, to highlight important findings and conclusions from the literature, with an emphasis on important obstacles and possible future paths, especially in the creation of state-of-the-art machine learning algorithms for the identification of emotions