Monitoring fatigue and drowsiness in motor vehicle occupants using electrocardiogram and heart rate - A systematic review

Introdução: A fadiga é um estado complexo que pode resultar em diminuição da vigilância, frequentemente acompanhada de sonolência. A fadiga durante a condução contribui significativamente para acidentes de trânsito em todo o mundo, destacando-se a necessidade de técnicas de monitorização eficazes. Existem várias tecnologias para aumentar a segurança do condutor e reduzir os riscos de acidentes, como sistemas de deteção de fadiga que podem alertar os condutores à medida que a sonolência se instala. Em particular, a análise dos padrões de frequência cardíaca pode oferecer informações valiosas sobre a condição fisiológica e o nível de vigilância do condutor, permitindo-lhe compreender os seus níveis de fadiga. Esta revisão tem como objetivo estabelecer o estado atual das estratégias de monitorização para ocupantes de veículos, com foco específico na avaliação da fadiga pela frequência cardíaca e variabilidade da frequência cardíaca. Métodos: Realizamos uma pesquisa sistemática da literatura nas bases de dados Web of Science, SCOPUS e Pubmed, utilizando os termos veículo, condutor, monitoração fisiológica, fadiga, sono, eletrocardiograma, frequência cardíaca e variabilidade da frequência cardíaca. Examinamos artigos publicados entre 1 de janeiro de 2018 e 31 de janeiro de 2023. Resultados: Um total de 371 artigos foram identificados, dos quais 71 foram incluídos neste estudo. Entre os artigos incluídos, 57 utilizam o eletrocardiograma (ECG) como sinal adquirido para medir a frequência cardíaca, sendo que a maioria das leituras de ECG foi obtida através de sensores de contacto (n=41), seguidos por sensores vestíveis não invasivos (n=11). Relativamente à validação, 23 artigos não mencionam qualquer tipo de validação, enquanto a maioria se baseia em avaliações subjetivas de fadiga relatadas pelos próprios participantes (n=27) e avaliações feitas por observadores com base em vídeos (n=11). Dos artigos incluídos, apenas 14 englobam um sistema de estimativa de fadiga e sonolência. Alguns relatam um desempenho satisfatórios, no entanto, o tamanho reduzido da amostra limita a abrangência de quaisquer conclusões. Conclusão: Esta revisão destaca o potencial da análise da frequência cardíaca e da instrumentação não invasiva para a monitorização contínua do estado do condutor e deteção de sonolência. Uma das principais questões é a falta de métodos suficientes de validação e estimativa para a fadiga, o que contribui para a insuficiência dos métodos na criação de sistemas de alarme proativos. Esta área apresenta grandes perspetivas, mas ainda está longe de ser implementada de forma fiável.Background: Fatigue is a complex state that can result in decreased alertness, often accompanied by drowsiness. Driving fatigue has become a significant contributor to traffic accidents globally, highlighting the need for effective monitoring techniques. Various technologies exist to enhance driver safety and minimize accident risks, such as fatigue detection systems that can alert drivers as drowsiness sets in. In particular, measuring heart rate patterns may offer valuable insights into the occupant's physiological condition and level of alertness, and may allow them to understand their fatigue levels. This review aims to establish the current state of the art of monitoring strategies for vehicle occupants, specifically focusing on fatigue assessed by heart rate and heart rate variability. Methods: We performed a systematic literature search in the databases of Web Of Science, SCOPUS and Pubmed, using the terms vehicle, driver, physiologic monitoring, fatigue, sleep, electrocardiogram, heart rate and heart rate variability. We examine articles published between 1st of january 2018 and 31st of January 2023. Results: A total of 371 papers were identified from which 71 articles were included in this study. Among the included papers, 57 utilized electrocardiogram (ECG) as the acquired signal for heart rate (HR) measures, with most ECG readings obtained through contact sensors (n=41), followed by non-intrusive wearable sensors (n=11). Regarding validation, 23 papers do not report validation, while the majority rely on subjective self-reported fatigue ratings (n=27) and video-based observer ratings(n=11). From the included papers, only 14 comprise a fatigue and drowsiness estimation system. Some report acceptable performances, but reduced sample size limits the reach of any conclusions. Conclusions: This review highlights the potential of HR analysis and non-intrusive instrumentation for continuous monitoring of driver's status and detecting sleepiness. One major issue is the lack of sufficient validation and estimation methods for fatigue, contributing to the insufficiency of methods in providing proactive alarm systems. This area shows great promise but is still far from being reliably implemented

Real-time drowsiness detection using wearable, lightweight EEG sensors

Driver drowsiness has always been a major concern for researchers and road use administrators. It has led to countless deaths accounting to significant percentile of deaths world over. Researchers have attempted to determine driver drowsiness using the following measures: (1) subjective measures (2) vehicle-based measures; (3) behavioral measures and (4) physiological measures.;Studies carried out to assess the efficacy of all the four measures, have brought out significant weaknesses in each of these measures. However detailed and comprehensive review has indicated that Physiological Measure namely EEG signal analysis provides most reliable and accurate information on driver drowsiness. In this paper a brief review of systems, and issues associated with them has been discussed with a view to evolve a novel system based on EEG signals especially for use in mine vehicles.;The feasibility of real-time drowsiness detection using commercially available, off-the-shelf, lightweight, wearable EEG sensors is explored. While EEG signals are known to be reliable indicators of fatigue and drowsiness, they have not been used widely due to their size and form factor. But the use of light-weight wearable EEGs alleviates this concern. Spectral analysis of EEG signals from these sensors using support vector machines is shown to classify drowsy states with high accuracy.;The system is validated using data collected on 23 subjects in fresh and drowsy states. The EEG signals are also used to characterize the blink duration and frequency of subjects. However, classification of drowsy states using blink analysis is shown to have lower accuracy than that using spectral analysis

Eye-tracking assistive technologies for individuals with amyotrophic lateral sclerosis

Amyotrophic lateral sclerosis, also known as ALS, is a progressive nervous system disorder that affects nerve cells in the brain and spinal cord, resulting in the loss of muscle control. For individuals with ALS, where mobility is limited to the movement of the eyes, the use of eye-tracking-based applications can be applied to achieve some basic tasks with certain digital interfaces. This paper presents a review of existing eye-tracking software and hardware through which eye-tracking their application is sketched as an assistive technology to cope with ALS. Eye-tracking also provides a suitable alternative as control of game elements. Furthermore, artificial intelligence has been utilized to improve eye-tracking technology with significant improvement in calibration and accuracy. Gaps in literature are highlighted in the study to offer a direction for future research

Embedded System Performance Analysis for Implementing a Portable Drowsiness Detection System for Drivers

Drowsiness on the road is a widespread problem with fatal consequences; thus, a multitude of systems and techniques have been proposed. Among existing methods, Ghoddoosian et al. utilized temporal blinking patterns to detect early signs of drowsiness, but their algorithm was tested only on a powerful desktop computer, which is not practical to apply in a moving vehicle setting. In this paper, we propose an efficient platform to run Ghoddosian's algorithm, detail the performance tests we ran to determine this platform, and explain our threshold optimization logic. After considering the Jetson Nano and Beelink (Mini PC), we concluded that the Mini PC is the most efficient and practical to run our embedded system in a vehicle. To determine this, we ran communication speed tests and evaluated total processing times for inference operations. Based on our experiments, the average total processing time to run the drowsiness detection model was 94.27 ms for Jetson Nano and 22.73 ms for the Beelink (Mini PC). Considering the portability and power efficiency of each device, along with the processing time results, the Beelink (Mini PC) was determined to be most suitable. Also, we propose a threshold optimization algorithm, which determines whether the driver is drowsy or alert based on the trade-off between the sensitivity and specificity of the drowsiness detection model. Our study will serve as a crucial next step for drowsiness detection research and its application in vehicles. Through our experiment, we have determinend a favorable platform that can run drowsiness detection algorithms in real-time and can be used as a foundation to further advance drowsiness detection research. In doing so, we have bridged the gap between an existing embedded system and its actual implementation in vehicles to bring drowsiness technology a step closer to prevalent real-life implementation.Comment: 26 pages, 13 figures, 4 table

Modelling driving performance using implicit interaction

The current project has been realized in collaboration with EIT Digital and Philips Research as part of the high impact initiative in the health & well-being action line. The challenge we are facing in the present work is to design a system for professional truck drivers that monitors driving behavior and predicts vigilance degradation. The research ended with defining parameters that can model drowsiness, fatigue, stress, aggressiveness and driver inattentiveness. The final proposal includes an in-vehicle system that does not impede the drivers' primary or secondary tasks, requires no explicit user input and provides feedback that promotes driving awareness and safer on-road behavior. The system is being designed to support user identification, personal profiles, driving performance monitoring and context-aware interaction for providing personalized and relevant to the circumstances feedback. In order to reach the desired conclusion, we initially conducted a literature review on advanced human-computer interaction and intelligent systems models and we present a model-based interface that supports the desired functionalities. The work also included comparison of cutting edge technologies for affective computing and driver modelling. Due to the nature of the agreement with Philips, we are not authorized to disclose any information that relate to user studies, thus the reader is presented with hypothetical scenarios for system output and user feedback that remain to be verified. These scenarios have been shaped with the help of technology acceptance and data privacy academic papers as well as deep understanding of the driving related context

Decoding Neural Correlates of Cognitive States to Enhance Driving Experience

Modern cars can support their drivers by assessing and autonomously performing different driving maneuvers based on information gathered by in-car sensors. We propose that brain–machine interfaces (BMIs) can provide complementary information that can ease the interaction with intelligent cars in order to enhance the driving experience. In our approach, the human remains in control, while a BMI is used to monitor the driver's cognitive state and use that information to modulate the assistance provided by the intelligent car. In this paper, we gather our proof-of-concept studies demonstrating the feasibility of decoding electroencephalography correlates of upcoming actions and those reflecting whether the decisions of driving assistant systems are in-line with the drivers' intentions. Experimental results while driving both simulated and real cars consistently showed neural signatures of anticipation, movement preparation, and error processing. Remarkably, despite the increased noise inherent to real scenarios, these signals can be decoded on a single-trial basis, reflecting some of the cognitive processes that take place while driving. However, moderate decoding performance compared to the controlled experimental BMI paradigms indicate there exists room for improvement of the machine learning methods typically used in the state-of-the-art BMIs. We foresee that neural fusion correlates with information extracted from other physiological measures, e.g., eye movements or electromyography as well as contextual information gathered by in-car sensors will allow intelligent cars to provide timely and tailored assistance only if it is required; thus, keeping the user in the loop and allowing him to fully enjoy the driving experience

Eye-Tracking Assistive Technologies for Individuals with Amyotrophic Lateral Sclerosis

Amyotrophic lateral sclerosis, also known as ALS, is a progressive nervous system disorder that affects nerve cells in the brain and spinal cord, resulting in the loss of muscle control. For individuals with ALS, where mobility is limited to the movement of the eyes, the use of eye-tracking-based applications can be applied to achieve some basic tasks with certain digital interfaces. This paper presents a review of existing eye-tracking software and hardware through which eye-tracking their application is sketched as an assistive technology to cope with ALS. Eye-tracking also provides a suitable alternative as control of game elements. Furthermore, artificial intelligence has been utilized to improve eye-tracking technology with significant improvement in calibration and accuracy. Gaps in literature are highlighted in the study to offer a direction for future research

Driver monitoring system based on eye tracking

Dissertação de mestrado integrado em Engenharia Electrónica Industrial e ComputadoresRecent statistics indicate that driver drowsiness is one of the major causes of road accidents and deaths behind the wheel. This reveals the need of reliable systems capable of predict when drivers are in this state and warn them in order to avoid crashes with other vehicles or stationary objects. Therefore, the purpose of this dissertation is to develop a driver’s monitoring system based on eye tracking that will be able to detect driver’s drowsiness level and actuate accordingly. The alert to the driver may vary from a message on the cluster to a vibration on the seat. The proposed algorithm to estimate driver’s state only requires one variable: eyelid opening. Through this variable the algorithm computes several eye parameters used to decide if the driver is drowsy or not, namely: PERCLOS, blink frequency and blink duration. Eyelid opening is obtained over a software and hardware platform called SmartEye Pro. This eye tracking system uses infrared cameras and computer vision software to gather eye’s state information. Additionally, since this dissertation is part of the project "INNOVATIVE CAR HMI", from Bosch and University of Minho partnership, the driver monitoring system will be integrated in the Bosch DSM (Driver Simulator Mockup).Estatísticas recentes indicam que a sonolência do condutor é uma das principais causas de acidentes e mortes nas estradas. Isto revela a necessidade de sistemas fiáveis capazes de prever quando um condutor está sonolento e avisá-lo, de modo a evitar colisões com outros veículos ou objetos estacionários. Portanto, o propósito desta dissertação é desenvolver um sistema de monitorização do condutor baseado em eye tracking que será capaz de detetar o nível de sonolência do condutor e atuar em conformidade. O alerta para o condutor pode variar entre uma mensagem no painel de instrumentos ou uma vibração no assento. O algoritmo proposto para estimar o estado do condutor apenas requer a aquisição de uma variável: abertura da pálpebra. Através desta variável, o algoritmo computa alguns parâmetros utilizados para verificar se o condutor está sonolento ou não, nomeadamente: PERCLOS, frequência do pestanejar e duração do pestanejar. A abertura da pálpebra é obtida através de uma plataforma de hardware e software chamada SmartEye Pro. Esta plataforma de eye tracking utiliza câmaras infravermelho e software de visão por computador para obter informação sobre o estado dos olhos. Adicionalmente, uma vez que esta dissertação está inserida projeto: "INNOVATIVE CAR HMI", da parceria entre a Bosch e a Universidade do Minho, o sistema desenvolvido será futuramente integrado no Bosch DSM (Driver Simulator Mockup)