Drowsy Eyes and Face Mask Detection for Car Drivers using the Embedded System

Efforts to prevent the spread of the COVID-19 virus have underscored the critical importance of mask-wearing as a preventive measure. Concurrently, road traffic accidents, often resulting from human error, have emerged as a significant contributor to global mortality rates. This study endeavors to address these pressing issues by employing advanced Deep Learning techniques to detect mask usage and identify drowsy eyes, thus contributing to the prevention of COVID-19 and accidents due to driver fatigue. To achieve this objective, an embedded system was developed, utilizing the integration of hardware and software components. The system effectively utilizes MobileNetV2 for face mask detection and employs HOG and SVM algorithms for drowsy eye detection. By seamlessly integrating these detection systems into a single embedded device, the simultaneous detection of both mask usage and drowsy eyes is made possible. The results demonstrates a commendable accuracy rate of 80% for face mask detection and 75% for drowsy eye detection. Furthermore, the mask detection component exhibits a remarkable training accuracy of 99%, while the drowsy eye detection component demonstrates an 80% training accuracy, affirming the system's efficacy in precisely identifying masks and drowsy eyes. The proposed embedded system offers potential applications in enhancing road safety. Its capability to effectively detect drowsy eyes and mask usage in car drivers contributes significantly to preventing accidents due to driver fatigue. Additionally, it plays a vital role in mitigating COVID-19 transmission by promoting widespread mask-wearing among individuals. This study exemplifies the potential of integrating Deep Learning methodologies with embedded systems, thus paving the way for future research and development in the realm of driver safety and virus prevention

Analysis and detection of driver fatigue caused by sleep deprivation

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Aeronautics and Astronautics, 2007.Includes bibliographical references (leaves 167-181).Human errors in attention and vigilance are among the most common causes of transportation accidents. Thus, effective countermeasures are crucial for enhancing road safety. By pursuing a practical and reliable design of an Active Safety system which aims to predict and avoid road accidents, we identify the characteristics of drowsy driving and devise a systematic way to infer the state of driver alertness based on driver-vehicle data. Although sleep and fatigue are major causes of impaired driving, neither effective regulations nor acceptable countermeasures are available yet. The first part of this thesis analyzes driver-vehicle systems with discrete sleep-deprivation levels, and reveals differences in the performance characteristics of drivers. Inspired by the human sleep-wake cycle mechanism and attributes of driver-vehicle systems, we design and perform human-in-the-loop experiments in a test bed built with STISIM Drive, an interactive fixed-based driving simulator. In the simulated driving, participants were given various driving tasks and secondary tasks for both non and partially sleep-deprived conditions. This experiment demonstrates that sleep deprivation has a greater effect on rule-based tasks than on skill-based tasks; when drivers are sleep-deprived, their performance of responding to unexpected disturbances degrades while they are robust enough to continue such routine driving tasks as straight lane tracking, following a lead vehicle, lane changes, etc. In the second part of the thesis we present both qualitative and quantitative guidelines for designing drowsy driver detection systems in a probabilistic framework based on the Bayesian network paradigm and experimental data.(cont.) We consider two major causes of sleep, i.e., sleep debt and circadian rhythm, in the framework with various driver-vehicle parameters, and also address temporal aspects of drowsiness and individual differences of subjects. The thesis concludes that detection of drowsy driving based on driver-vehicle data is a feasible but difficult problem which has diverse issues to be addressed; the ultimate challenge lies in the human operator.by Ji Hyun Yang.Ph.D

A Context Aware Classification System for Monitoring Driver’s Distraction Levels

Understanding the safety measures regarding developing self-driving futuristic cars is a concern for decision-makers, civil society, consumer groups, and manufacturers. The researchers are trying to thoroughly test and simulate various driving contexts to make these cars fully secure for road users. Including the vehicle’ surroundings offer an ideal way to monitor context-aware situations and incorporate the various hazards. In this regard, different studies have analysed drivers’ behaviour under different case scenarios and scrutinised the external environment to obtain a holistic view of vehicles and the environment. Studies showed that the primary cause of road accidents is driver distraction, and there is a thin line that separates the transition from careless to dangerous. While there has been a significant improvement in advanced driver assistance systems, the current measures neither detect the severity of the distraction levels nor the context-aware, which can aid in preventing accidents. Also, no compact study provides a complete model for transitioning control from the driver to the vehicle when a high degree of distraction is detected. The current study proposes a context-aware severity model to detect safety issues related to driver’s distractions, considering the physiological attributes, the activities, and context-aware situations such as environment and vehicle. Thereby, a novel three-phase Fast Recurrent Convolutional Neural Network (Fast-RCNN) architecture addresses the physiological attributes. Secondly, a novel two-tier FRCNN-LSTM framework is devised to classify the severity of driver distraction. Thirdly, a Dynamic Bayesian Network (DBN) for the prediction of driver distraction. The study further proposes the Multiclass Driver Distraction Risk Assessment (MDDRA) model, which can be adopted in a context-aware driving distraction scenario. Finally, a 3-way hybrid CNN-DBN-LSTM multiclass degree of driver distraction according to severity level is developed. In addition, a Hidden Markov Driver Distraction Severity Model (HMDDSM) for the transitioning of control from the driver to the vehicle when a high degree of distraction is detected. This work tests and evaluates the proposed models using the multi-view TeleFOT naturalistic driving study data and the American University of Cairo dataset (AUCD). The evaluation of the developed models was performed using cross-correlation, hybrid cross-correlations, K-Folds validation. The results show that the technique effectively learns and adopts safety measures related to the severity of driver distraction. In addition, the results also show that while a driver is in a dangerous distraction state, the control can be shifted from driver to vehicle in a systematic manner

Particle Filters for Colour-Based Face Tracking Under Varying Illumination

Automatic human face tracking is the basis of robotic and active vision systems used for facial feature analysis, automatic surveillance, video conferencing, intelligent transportation, human-computer interaction and many other applications. Superior human face tracking will allow future safety surveillance systems which monitor drowsy drivers, or patients and elderly people at the risk of seizure or sudden falls and will perform with lower risk of failure in unexpected situations. This area has actively been researched in the current literature in an attempt to make automatic face trackers more stable in challenging real-world environments. To detect faces in video sequences, features like colour, texture, intensity, shape or motion is used. Among these feature colour has been the most popular, because of its insensitivity to orientation and size changes and fast process-ability. The challenge of colour-based face trackers, however, has been dealing with the instability of trackers in case of colour changes due to the drastic variation in environmental illumination. Probabilistic tracking and the employment of particle filters as powerful Bayesian stochastic estimators, on the other hand, is increasing in the visual tracking field thanks to their ability to handle multi-modal distributions in cluttered scenes. Traditional particle filters utilize transition prior as importance sampling function, but this can result in poor posterior sampling. The objective of this research is to investigate and propose stable face tracker capable of dealing with challenges like rapid and random motion of head, scale changes when people are moving closer or further from the camera, motion of multiple people with close skin tones in the vicinity of the model person, presence of clutter and occlusion of face. The main focus has been on investigating an efficient method to address the sensitivity of the colour-based trackers in case of gradual or drastic illumination variations. The particle filter is used to overcome the instability of face trackers due to nonlinear and random head motions. To increase the traditional particle filter\u27s sampling efficiency an improved version of the particle filter is introduced that considers the latest measurements. This improved particle filter employs a new colour-based bottom-up approach that leads particles to generate an effective proposal distribution. The colour-based bottom-up approach is a classification technique for fast skin colour segmentation. This method is independent to distribution shape and does not require excessive memory storage or exhaustive prior training. Finally, to address the adaptability of the colour-based face tracker to illumination changes, an original likelihood model is proposed based of spatial rank information that considers both the illumination invariant colour ordering of a face\u27s pixels in an image or video frame and the spatial interaction between them. The original contribution of this work lies in the unique mixture of existing and proposed components to improve colour-base recognition and tracking of faces in complex scenes, especially where drastic illumination changes occur. Experimental results of the final version of the proposed face tracker, which combines the methods developed, are provided in the last chapter of this manuscript

Methods and techniques for analyzing human factors facets on drivers

Mención Internacional en el título de doctorWith millions of cars moving daily, driving is the most performed activity worldwide. Unfortunately, according to the World Health Organization (WHO), every year, around 1.35 million people worldwide die from road traffic accidents and, in addition, between 20 and 50 million people are injured, placing road traffic accidents as the second leading cause of death among people between the ages of 5 and 29. According to WHO, human errors, such as speeding, driving under the influence of drugs, fatigue, or distractions at the wheel, are the underlying cause of most road accidents. Global reports on road safety such as "Road safety in the European Union. Trends, statistics, and main challenges" prepared by the European Commission in 2018 presented a statistical analysis that related road accident mortality rates and periods segmented by hours and days of the week. This report revealed that the highest incidence of mortality occurs regularly in the afternoons during working days, coinciding with the period when the volume of traffic increases and when any human error is much more likely to cause a traffic accident. Accordingly, mitigating human errors in driving is a challenge, and there is currently a growing trend in the proposal for technological solutions intended to integrate driver information into advanced driving systems to improve driver performance and ergonomics. The study of human factors in the field of driving is a multidisciplinary field in which several areas of knowledge converge, among which stand out psychology, physiology, instrumentation, signal treatment, machine learning, the integration of information and communication technologies (ICTs), and the design of human-machine communication interfaces. The main objective of this thesis is to exploit knowledge related to the different facets of human factors in the field of driving. Specific objectives include identifying tasks related to driving, the detection of unfavorable cognitive states in the driver, such as stress, and, transversely, the proposal for an architecture for the integration and coordination of driver monitoring systems with other active safety systems. It should be noted that the specific objectives address the critical aspects in each of the issues to be addressed. Identifying driving-related tasks is one of the primary aspects of the conceptual framework of driver modeling. Identifying maneuvers that a driver performs requires training beforehand a model with examples of each maneuver to be identified. To this end, a methodology was established to form a data set in which a relationship is established between the handling of the driving controls (steering wheel, pedals, gear lever, and turn indicators) and a series of adequately identified maneuvers. This methodology consisted of designing different driving scenarios in a realistic driving simulator for each type of maneuver, including stop, overtaking, turns, and specific maneuvers such as U-turn and three-point turn. From the perspective of detecting unfavorable cognitive states in the driver, stress can damage cognitive faculties, causing failures in the decision-making process. Physiological signals such as measurements derived from the heart rhythm or the change of electrical properties of the skin are reliable indicators when assessing whether a person is going through an episode of acute stress. However, the detection of stress patterns is still an open problem. Despite advances in sensor design for the non-invasive collection of physiological signals, certain factors prevent reaching models capable of detecting stress patterns in any subject. This thesis addresses two aspects of stress detection: the collection of physiological values during stress elicitation through laboratory techniques such as the Stroop effect and driving tests; and the detection of stress by designing a process flow based on unsupervised learning techniques, delving into the problems associated with the variability of intra- and inter-individual physiological measures that prevent the achievement of generalist models. Finally, in addition to developing models that address the different aspects of monitoring, the orchestration of monitoring systems and active safety systems is a transversal and essential aspect in improving safety, ergonomics, and driving experience. Both from the perspective of integration into test platforms and integration into final systems, the problem of deploying multiple active safety systems lies in the adoption of monolithic models where the system-specific functionality is run in isolation, without considering aspects such as cooperation and interoperability with other safety systems. This thesis addresses the problem of the development of more complex systems where monitoring systems condition the operability of multiple active safety systems. To this end, a mediation architecture is proposed to coordinate the reception and delivery of data flows generated by the various systems involved, including external sensors (lasers, external cameras), cabin sensors (cameras, smartwatches), detection models, deliberative models, delivery systems and machine-human communication interfaces. Ontology-based data modeling plays a crucial role in structuring all this information and consolidating the semantic representation of the driving scene, thus allowing the development of models based on data fusion.I would like to thank the Ministry of Economy and Competitiveness for granting me the predoctoral fellowship BES-2016-078143 corresponding to the project TRA2015-63708-R, which provided me the opportunity of conducting all my Ph. D activities, including completing an international internship.Programa de Doctorado en Ciencia y Tecnología Informática por la Universidad Carlos III de MadridPresidente: José María Armingol Moreno.- Secretario: Felipe Jiménez Alonso.- Vocal: Luis Mart

the admissibility of AI- generated evidence

Durante as duas últimas décadas, a Inteligência Artificial tornou-se uma presença constante nas nossas vidas. Ao impactar setores relevantes da sociedade, tem relevando o seu caráter disruptivo, sendo um dos motores impulsionadores da Quarta Revolução Industrial. A Inteligência Artificial além dos seus presentes benefícios para a humanidade, promete soluções inovadoras para os problemas que afligem a sociedade contemporânea, porém a mesma comporta uma duplicidade de efeitos. Os sistemas de Inteligência Artificial pela sua capacidade de monitorizar o seu ambiente circundante, e autonomamente recolher, processar dados, aprender e agir, podem concretizar riscos para os direitos fundamentais, principalmente no contexto da justiça criminal. Esta análise irá focar-se nas especificidades dos sistemas dotados de Inteligência Artificial, aprofundando a temática da admissibilidade da prova gerada por Inteligência Artificial no quadro probatório do Direito Processual Penal Português à luz dos direitos de defesa do arguido e dos seus princípios que norteadores.During the last two decades Artificial Intelligence became ubiquitous in our lives. Revealing itself as a disruptive technology, it is already impacting important sectors of society, being a driver of the Fourth Industrial Revolution. Artificial Intelligence is benefiting humanity, and promises innovative solutions to modern-life problems, nevertheless it has a twofold effect. Artificial Intelligence as systems that are capable to monitor their surrounding environment, autonomously collect and process data, learn and act, may constitute harm to fundamental rights, mainly when deployed to criminal justice. This analysis will focus on the specificities of Artificial Intelligence systems, delving into the admissibility of AI-generated evidence in the Portuguese criminal evidentiary framework in light of the defence rights and structuring principles of Portuguese criminal procedure

Improving Vehicular ad hoc Network Protocols to Support Safety Applications in Realistic Scenarios

La convergencia de las telecomunicaciones, la informática, la tecnología inalámbrica y los sistemas de transporte, va a facilitar que nuestras carreteras y autopistas nos sirvan tanto como plataforma de transporte, como de comunicaciones. Estos cambios van a revolucionar completamente cómo y cuándo vamos a acceder a determinados servicios, comunicarnos, viajar, entretenernos, y navegar, en un futuro muy cercano. Las redes vehiculares ad hoc (vehicular ad hoc networks VANETs) son redes de comunicación inalámbricas que no requieren de ningún tipo de infraestructura, y que permiten la comunicación y conducción cooperativa entre los vehículos en la carretera. Los vehículos actúan como nodos de comunicación y transmisores, formando redes dinámicas junto a otros vehículos cercanos en entornos urbanos y autopistas. Las características especiales de las redes vehiculares favorecen el desarrollo de servicios y aplicaciones atractivas y desafiantes. En esta tesis nos centramos en las aplicaciones relacionadas con la seguridad. Específicamente, desarrollamos y evaluamos un novedoso protocol que mejora la seguridad en las carreteras. Nuestra propuesta combina el uso de información de la localización de los vehículos y las características del mapa del escenario, para mejorar la diseminación de los mensajes de alerta. En las aplicaciones de seguridad para redes vehiculares, nuestra propuesta permite reducir el problema de las tormentas de difusión, mientras que se mantiene una alta efectividad en la diseminación de los mensajes hacia los vehículos cercanos. Debido a que desplegar y evaluar redes VANET supone un gran coste y una tarea dura, la metodología basada en la simulación se muestra como una metodología alternativa a la implementación real. A diferencia de otros trabajos previos, con el fin de evaluar nuestra propuesta en un entorno realista, en nuestras simulaciones tenemos muy en cuenta tanto la movilidad de los vehículos, como la transmisión de radio en entornos urbanos, especialmente cuando los edificios interfieren en la propagación de la señal de radio. Con este propósito, desarrollamos herramientas para la simulación de VANETs más precisas y realistas, mejorando tanto la modelización de la propagación de radio, como la movilidad de los vehículos, obteniendo una solución que permite integrar mapas reales en el entorno de simulación. Finalmente, evaluamos las prestaciones de nuestro protocolo propuesto haciendo uso de nuestra plataforma de simulación mejorada, evidenciando la importancia del uso de un entorno de simulación adecuado para conseguir resultados más realistas y poder obtener conclusiones más significativas.Martínez Domínguez, FJ. (2010). Improving Vehicular ad hoc Network Protocols to Support Safety Applications in Realistic Scenarios [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/9195Palanci

Driver lane change intention inference using machine learning methods.

Lane changing manoeuvre on highway is a highly interactive task for human drivers. The intelligent vehicles and the advanced driver assistance systems (ADAS) need to have proper awareness of the traffic context as well as the driver. The ADAS also need to understand the driver potential intent correctly since it shares the control authority with the human driver. This study provides a research on the driver intention inference, particular focus on the lane change manoeuvre on highways. This report is organised in a paper basis, where each chapter corresponding to a publication, which is submitted or to be submitted. Part Ⅰ introduce the motivation and general methodology framework for this thesis. Part Ⅱ includes the literature survey and the state-of-art of driver intention inference. Part Ⅲ contains the techniques for traffic context perception that focus on the lane detection. A literature review on lane detection techniques and its integration with parallel driving framework is proposed. Next, a novel integrated lane detection system is designed. Part Ⅳ contains two parts, which provides the driver behaviour monitoring system for normal driving and secondary tasks detection. The first part is based on the conventional feature selection methods while the second part introduces an end-to-end deep learning framework. The design and analysis of driver lane change intention inference system for the lane change manoeuvre is proposed in Part Ⅴ. Finally, discussions and conclusions are made in Part Ⅵ. A major contribution of this project is to propose novel algorithms which accurately model the driver intention inference process. Lane change intention will be recognised based on machine learning (ML) methods due to its good reasoning and generalizing characteristics. Sensors in the vehicle are used to capture context traffic information, vehicle dynamics, and driver behaviours information. Machine learning and image processing are the techniques to recognise human driver behaviour.PhD in Transpor

Essays on Creative Ideation and New Product Design

Creative ideation, i.e., the generation of novel ideas, represents the terminus-a-quo in the design and development of innovative products. In my dissertation essays, I examine two approaches employed by firms for creative ideation, (1) channeled ideation, a closed approach, which involves applying replicable patterns or properties observed in historical innovations and (2) idea crowdsourcing, an open approach where firms invite crowds to contribute ideas to solve a specific challenge. In my studies, I clarify how firms can incorporate market-related information in the channeled ideation process and examine how the selection of ideas in crowdsourcing challenges relates to local and global novelty. In Essay 1, “Attribute Auto-dynamics and New Product Ideation,” I introduce a replicable property – attribute auto-dynamics, observed in several novel products, where a product possesses the ability to modify its attributes automatically in response to changing customer, product-system, or environmental conditions. I propose a typology of attribute auto-dynamics, based on an analysis of U.S. utility patents. Based on this typology, I specify a procedural framework for new product ideation that integrates market-pull relevant knowledge and technology-push relevant knowledge. I also illustrate how managers and product designers can apply the framework to identify new product ideas for specific target markets using a channeled ideation approach. In Essay 2, “Selection in Crowdsourced Ideation: Role of Local and Global Novelty,” I examine how the selection of ideas in crowdsourced challenges depends on the form of novelty – local or global. Firms often turn to idea crowdsourcing challenges to obtain novel ideas. Yet prior research cautions that ideators and seeker firms may not select novel ideas. To reexamine the links between idea novelty and selection, I propose a bi-faceted notion of idea novelty that may be local or global. Examining data on OpenIDEO, I find that the selection of novel ideas differs according to the selector, the form of novelty, and the challenge task structure. I also specify a predictive model that seeker firms can leverage when ideator selection metrics such as likes are unavailable.Doctor of Philosoph

Development of rear-end collision avoidance in automobiles

The goal of this work is to develop a Rear-End Collision Avoidance System for automobiles. In order to develop the Rear-end Collision Avoidance System, it is stated that the most important difference from the old practice is the fact that new design approach attempts to completely avoid collision instead of minimizing the damage by over-designing cars. Rear-end collisions are the third highest cause of multiple vehicle fatalities in the U.S. Their cause seems to be a result of poor driver awareness and communication. For example, car brake lights illuminate exactly the same whether the car is slowing, stopping or the driver is simply resting his foot on the pedal. In the development of Rear-End Collision Avoidance System (RECAS), a thorough review of hardware, software, driver/human factors, and current rear-end collision avoidance systems are included. Key sensor technologies are identified and reviewed in an attempt to ease the design effort. The characteristics and capabilities of alternative and emerging sensor technologies are also described and their performance compared. In designing a RECAS the first component is to monitor the distance and speed of the car ahead. If an unsafe condition is detected a warning is issued and the vehicle is decelerated (if necessary). The second component in the design effort utilizes the illumination of independent segments of brake lights corresponding to the stopping condition of the car. This communicates the stopping intensity to the following driver. The RECAS is designed the using the LabVIEW software. The simulation is designed to meet several criteria: System warnings should result in a minimum load on driver attention, and the system should also perform well in a variety of driving conditions. In order to illustrate and test the proposed RECAS methods, a Java program has been developed. This simulation animates a multi-car, multi-lane highway environment where car speeds are assigned randomly, and the proposed RECAS approaches demonstrate rear-end collision avoidance successfully. The Java simulation is an applet, which is easily accessible through the World Wide Web and also can be tested for different angles of the sensor