A VISION-BASED SYSTEM FOR MONITORING DRIVER FATIGUE

Abstract This paper presents a vision-based system for monitoring driver fatigue. The system is divided into three stages: face detection, eye detection, and fatigue detection. Face detection based on the skin color segmentation is used to localize face image from a whole image. To overcome the normalized RGB chromaticity diagram is adopted. After face is localized, eye is detected by PERCLOS (percentage of eye closure over time) is calculated and used to detect a fatigue condition. Keywords : Driver fatigue, machine vision, face detection, eye detecton, fatigue detection

Car that Knows Before You Do: Anticipating Maneuvers via Learning Temporal Driving Models

Advanced Driver Assistance Systems (ADAS) have made driving safer over the last decade. They prepare vehicles for unsafe road conditions and alert drivers if they perform a dangerous maneuver. However, many accidents are unavoidable because by the time drivers are alerted, it is already too late. Anticipating maneuvers beforehand can alert drivers before they perform the maneuver and also give ADAS more time to avoid or prepare for the danger. In this work we anticipate driving maneuvers a few seconds before they occur. For this purpose we equip a car with cameras and a computing device to capture the driving context from both inside and outside of the car. We propose an Autoregressive Input-Output HMM to model the contextual information alongwith the maneuvers. We evaluate our approach on a diverse data set with 1180 miles of natural freeway and city driving and show that we can anticipate maneuvers 3.5 seconds before they occur with over 80\% F1-score in real-time.Comment: ICCV 2015, http://brain4cars.co

A VISION-BASED SYSTEM FOR MONITORING DRIVER FATIGUE

A VISION-BASED SYSTEM FOR MONITORING DRIVER FATIGUE Aryuanto1) F. Yudi Limpraptono2) 1,2 Department of Electrical Engineering, Institut Teknologi Nasional (ITN) Malang Jalan Raya Karanglo Km. 2 Malang 1 [email protected], 2 [email protected] Abstract This paper presents a vision-based system for monitoring driver fatigue. The system is divided into three stages: face detection, eye detection, and fatigue detection. Face detection based on the skin color segmentation is used to localize face image from a whole image. To overcome the normalized RGB chromaticity diagram is adopted. After face is localized, eye is detected by PERCLOS (percentage of eye closure over time) is calculated and used to detect a fatigue condition. Keywords : Driver fatigue, machine vision, face detection, eye detecton, fatigue detection

A Review of Driver Gaze Estimation and Application in Gaze Behavior Understanding

Driver gaze plays an important role in different gaze-based applications such as driver attentiveness detection, visual distraction detection, gaze behavior understanding, and building driver assistance system. The main objective of this study is to perform a comprehensive summary of driver gaze fundamentals, methods to estimate driver gaze, and it's applications in real world driving scenarios. We first discuss the fundamentals related to driver gaze, involving head-mounted and remote setup based gaze estimation and the terminologies used for each of these data collection methods. Next, we list out the existing benchmark driver gaze datasets, highlighting the collection methodology and the equipment used for such data collection. This is followed by a discussion of the algorithms used for driver gaze estimation, which primarily involves traditional machine learning and deep learning based techniques. The estimated driver gaze is then used for understanding gaze behavior while maneuvering through intersections, on-ramps, off-ramps, lane changing, and determining the effect of roadside advertising structures. Finally, we have discussed the limitations in the existing literature, challenges, and the future scope in driver gaze estimation and gaze-based applications

Human-Centric Detection and Mitigation Approach for Various Levels of Cell Phone-Based Driver Distractions

abstract: Driving a vehicle is a complex task that typically requires several physical interactions and mental tasks. Inattentive driving takes a driver’s attention away from the primary task of driving, which can endanger the safety of driver, passenger(s), as well as pedestrians. According to several traffic safety administration organizations, distracted and inattentive driving are the primary causes of vehicle crashes or near crashes. In this research, a novel approach to detect and mitigate various levels of driving distractions is proposed. This novel approach consists of two main phases: i.) Proposing a system to detect various levels of driver distractions (low, medium, and high) using a machine learning techniques. ii.) Mitigating the effects of driver distractions through the integration of the distracted driving detection algorithm and the existing vehicle safety systems. In phase- 1, vehicle data were collected from an advanced driving simulator and a visual based sensor (webcam) for face monitoring. In addition, data were processed using a machine learning algorithm and a head pose analysis package in MATLAB. Then the model was trained and validated to detect different human operator distraction levels. In phase 2, the detected level of distraction, time to collision (TTC), lane position (LP), and steering entropy (SE) were used as an input to feed the vehicle safety controller that provides an appropriate action to maintain and/or mitigate vehicle safety status. The integrated detection algorithm and vehicle safety controller were then prototyped using MATLAB/SIMULINK for validation. A complete vehicle power train model including the driver’s interaction was replicated, and the outcome from the detection algorithm was fed into the vehicle safety controller. The results show that the vehicle safety system controller reacted and mitigated the vehicle safety status-in closed loop real-time fashion. The simulation results show that the proposed approach is efficient, accurate, and adaptable to dynamic changes resulting from the driver, as well as the vehicle system. This novel approach was applied in order to mitigate the impact of visual and cognitive distractions on the driver performance.Dissertation/ThesisDoctoral Dissertation Applied Psychology 201

Correlating driver gaze with the road scene for driver assistance systems

A driver assistance system (DAS) should support the driver by monitoring road and vehicle events and presenting relevant and timely information to the driver. It is impossible to know what a driver is thinking, but we can monitor the driver's gaze direction and compare it with the position of information in the driver's viewfield to make inferences. In this way, not only do we monitor the driver's actions, we monitor the driver's observations as well. In this paper we present the automated detection and recognition of road signs, combined with the monitoring of the driver's response. We present a complete system that reads speed signs in real-time, compares the driver's gaze, and provides immediate feedback if it appears the sign has been missed by the driver

Assessment of Driver\u27s Attention to Traffic Signs through Analysis of Gaze and Driving Sequences

A driver’s behavior is one of the most significant factors in Advance Driver Assistance Systems. One area that has received little study is just how observant drivers are in seeing and recognizing traffic signs. In this contribution, we present a system considering the location where a driver is looking (points of gaze) as a factor to determine that whether the driver has seen a sign. Our system detects and classifies traffic signs inside the driver’s attentional visual field to identify whether the driver has seen the traffic signs or not. Based on the results obtained from this stage which provides quantitative information, our system is able to determine how observant of traffic signs that drivers are. We take advantage of the combination of Maximally Stable Extremal Regions algorithm and Color information in addition to a binary linear Support Vector Machine classifier and Histogram of Oriented Gradients as features detector for detection. In classification stage, we use a multi class Support Vector Machine for classifier also Histogram of Oriented Gradients for features. In addition to the detection and recognition of traffic signs, our system is capable of determining if the sign is inside the attentional visual field of the drivers. It means the driver has kept his gaze on traffic signs and sees the sign, while if the sign is not inside this area, the driver did not look at the sign and sign has been missed

A framework for context-aware driver status assessment systems

The automotive industry is actively supporting research and innovation to meet manufacturers' requirements related to safety issues, performance and environment. The Green ITS project is among the efforts in that regard. Safety is a major customer and manufacturer concern. Therefore, much effort have been directed to developing cutting-edge technologies able to assess driver status in term of alertness and suitability. In that regard, we aim to create with this thesis a framework for a context-aware driver status assessment system. Context-aware means that the machine uses background information about the driver and environmental conditions to better ascertain and understand driver status. The system also relies on multiple sensors, mainly video and audio. Using context and multi-sensor data, we need to perform multi-modal analysis and data fusion in order to infer as much knowledge as possible about the driver. Last, the project is to be continued by other students, so the system should be modular and well-documented. With this in mind, a driving simulator integrating multiple sensors was built. This simulator is a starting point for experimentation related to driver status assessment, and a prototype of software for real-time driver status assessment is integrated to the platform. To make the system context-aware, we designed a driver identification module based on audio-visual data fusion. Thus, at the beginning of driving sessions, the users are identified and background knowledge about them is loaded to better understand and analyze their behavior. A driver status assessment system was then constructed based on two different modules. The first one is for driver fatigue detection, based on an infrared camera. Fatigue is inferred via percentage of eye closure, which is the best indicator of fatigue for vision systems. The second one is a driver distraction recognition system, based on a Kinect sensor. Using body, head, and facial expressions, a fusion strategy is employed to deduce the type of distraction a driver is subject to. Of course, fatigue and distraction are only a fraction of all possible drivers' states, but these two aspects have been studied here primarily because of their dramatic impact on traffic safety. Through experimental results, we show that our system is efficient for driver identification and driver inattention detection tasks. Nevertheless, it is also very modular and could be further complemented by driver status analysis, context or additional sensor acquisition

Fahrerzustandserkennung zur Optimierung von Spurhalteassistenzsystemen

Aktuelle Fahrerassistenzsysteme der Längs- und Querführung bleiben immer noch hinter Ihren potentiellen Leistungen zurück. Die Regelung erfolgt derzeit nur über Zustandsgrößen des Fahrzeugs und Sensordaten der Umgebung, lässt den Zustand und die Intentionen des Fahrers aber unberücksichtigt. So zeigen beispielsweise Untersuchungen zur Wirksamkeit von Spurhalteassistenzsystemen, dass in einer Vielzahl der Fälle Warnungen unnötig sind, da der Fahrer ohnehin aufmerksam fährt. Nur durch die Berücksichtigung des Fahrers im Regelkreis Fahrer-Fahrzeug-Umwelt können Warnungen gezielt erfolgen und kann die Wirksamkeit von Assistenzsystemen erhöht werden. Der erste Teil der Dissertation beschäftigt sich generell mit verschiedenen Möglichkeiten einer Echtzeit-Erkennung des Fahrerzustands im Fahrzeug. Im zweiten Teil wird aufbauend darauf speziell das Potential der Adaption von Spurhalteassistenzsystemen an den Fahrerzustand analysiert. Die Fahrerzustandsschätzung wird über drei Ansätze näher betrachtet. Zunächst wird eine Schätzung der Aufmerksamkeitsausrichtung des Fahrers über die Erfassung seiner Augenbewegungen und der Kopforientierung untersucht. Dabei zeigt sich, dass Systeme zur Blickbewegungserfassung viel Potential bieten, dass diese aber noch nicht im automobilen Bereich nutzbar sind. Die Aussagekraft der Kopforientierung hinsichtlich einer Ablenkung des Fahrers wurde in einer eigenen experimentellen Studie im realen Verkehr über eigens entwickelte Algorithmen analysiert. Zwar ist eine solche Erfassung schon heute in Serienfahrzeugen möglich, die Aussagekraft der Daten ist jedoch begrenzt. Eine weitere Möglichkeit zur Erfassung des Fahrerzustands besteht darin, direkt Nebentätigkeiten des Fahrers zu detektieren. Eine eigene Probandenstudie verdeutlicht dabei, dass es schon heute möglich ist, die Ablenkung des Fahrers aufgrund von Bedienhandlungen am Infotainment-System eines Fahrzeugs zu erkennen. In einem letzten Schritt wird die Erfassbarkeit des Fahrerzustands aufgrund der schon heute in Fahrzeugen verfügbaren Sensorik untersucht. Verschiedene Studien aus der Literatur zeigen auffälliges Lenk- und Spurhalteverhalten abgelenkter Fahrer. Darauf basierend wird mit Hilfe maschinellen Lernens ein Algorithmus zur Fahrerzustandserkennung entwickelt, der eine prinzipielle Verwendbarkeit dieses Ansatzes aufzeigt. Die beschrieben Methoden zur Fahrerzustandserkennung werden im zweiten Teil der Arbeit hinsichtlich einer konkreten Anwendung im Bereich der Spurhalteassistenzsysteme untersucht. Fremde und eigene experimentelle Studien zur adaptiven Parametrierung von Spurhalteassistenzsystemen zeigen, dass diese Anpassungen im realen Straßenverkehr nicht nur zu einer objektiven Verbesserung der Spurhaltung und damit der Verkehrssicherheit führen, sondern dass diese von den Probanden auch positiv beurteilt werden. Für eine Fahrerzustands-adaptive Auslegung von Spurhalteassistenten eignen sich dabei vorrangig Blickbewegungsdaten und die Erfassung von Bedienhandlungen. Schätzungen des Fahrerzustands über Lenkverhalten und die Kopforientierung des Fahrers sind weniger geeignet. Zusammenfassend gibt diese Arbeit somit einen Einblick in Methoden einer Fahrerzustandserfassung im Fahrzeug und zeigt basierend darauf den Akzeptanz- und Sicherheitsgewinn einer Fahrerzustands-adaptiven Auslegung von Spurhalteassistenzsystemen