Monitoring fatigue and drowsiness in motor vehicle occupants using electrocardiogram and heart rate - A systematic review

Introdução: A fadiga é um estado complexo que pode resultar em diminuição da vigilância, frequentemente acompanhada de sonolência. A fadiga durante a condução contribui significativamente para acidentes de trânsito em todo o mundo, destacando-se a necessidade de técnicas de monitorização eficazes. Existem várias tecnologias para aumentar a segurança do condutor e reduzir os riscos de acidentes, como sistemas de deteção de fadiga que podem alertar os condutores à medida que a sonolência se instala. Em particular, a análise dos padrões de frequência cardíaca pode oferecer informações valiosas sobre a condição fisiológica e o nível de vigilância do condutor, permitindo-lhe compreender os seus níveis de fadiga. Esta revisão tem como objetivo estabelecer o estado atual das estratégias de monitorização para ocupantes de veículos, com foco específico na avaliação da fadiga pela frequência cardíaca e variabilidade da frequência cardíaca. Métodos: Realizamos uma pesquisa sistemática da literatura nas bases de dados Web of Science, SCOPUS e Pubmed, utilizando os termos veículo, condutor, monitoração fisiológica, fadiga, sono, eletrocardiograma, frequência cardíaca e variabilidade da frequência cardíaca. Examinamos artigos publicados entre 1 de janeiro de 2018 e 31 de janeiro de 2023. Resultados: Um total de 371 artigos foram identificados, dos quais 71 foram incluídos neste estudo. Entre os artigos incluídos, 57 utilizam o eletrocardiograma (ECG) como sinal adquirido para medir a frequência cardíaca, sendo que a maioria das leituras de ECG foi obtida através de sensores de contacto (n=41), seguidos por sensores vestíveis não invasivos (n=11). Relativamente à validação, 23 artigos não mencionam qualquer tipo de validação, enquanto a maioria se baseia em avaliações subjetivas de fadiga relatadas pelos próprios participantes (n=27) e avaliações feitas por observadores com base em vídeos (n=11). Dos artigos incluídos, apenas 14 englobam um sistema de estimativa de fadiga e sonolência. Alguns relatam um desempenho satisfatórios, no entanto, o tamanho reduzido da amostra limita a abrangência de quaisquer conclusões. Conclusão: Esta revisão destaca o potencial da análise da frequência cardíaca e da instrumentação não invasiva para a monitorização contínua do estado do condutor e deteção de sonolência. Uma das principais questões é a falta de métodos suficientes de validação e estimativa para a fadiga, o que contribui para a insuficiência dos métodos na criação de sistemas de alarme proativos. Esta área apresenta grandes perspetivas, mas ainda está longe de ser implementada de forma fiável.Background: Fatigue is a complex state that can result in decreased alertness, often accompanied by drowsiness. Driving fatigue has become a significant contributor to traffic accidents globally, highlighting the need for effective monitoring techniques. Various technologies exist to enhance driver safety and minimize accident risks, such as fatigue detection systems that can alert drivers as drowsiness sets in. In particular, measuring heart rate patterns may offer valuable insights into the occupant's physiological condition and level of alertness, and may allow them to understand their fatigue levels. This review aims to establish the current state of the art of monitoring strategies for vehicle occupants, specifically focusing on fatigue assessed by heart rate and heart rate variability. Methods: We performed a systematic literature search in the databases of Web Of Science, SCOPUS and Pubmed, using the terms vehicle, driver, physiologic monitoring, fatigue, sleep, electrocardiogram, heart rate and heart rate variability. We examine articles published between 1st of january 2018 and 31st of January 2023. Results: A total of 371 papers were identified from which 71 articles were included in this study. Among the included papers, 57 utilized electrocardiogram (ECG) as the acquired signal for heart rate (HR) measures, with most ECG readings obtained through contact sensors (n=41), followed by non-intrusive wearable sensors (n=11). Regarding validation, 23 papers do not report validation, while the majority rely on subjective self-reported fatigue ratings (n=27) and video-based observer ratings(n=11). From the included papers, only 14 comprise a fatigue and drowsiness estimation system. Some report acceptable performances, but reduced sample size limits the reach of any conclusions. Conclusions: This review highlights the potential of HR analysis and non-intrusive instrumentation for continuous monitoring of driver's status and detecting sleepiness. One major issue is the lack of sufficient validation and estimation methods for fatigue, contributing to the insufficiency of methods in providing proactive alarm systems. This area shows great promise but is still far from being reliably implemented

Kinesthesia in a sustained-attention driving task

This study investigated the effects of kinesthetic stimuli on brain activities during a sustained-attention task in an immersive driving simulator. Tonic and phasic brain responses on multiple timescales were analyzed using time-frequency analysis of electroencephalographic (EEG) sources identified by independent component analysis (ICA). Sorting EEG spectra with respect to reaction times (RT) to randomly introduced lane-departure events revealed distinct effects of kinesthetic stimuli on the brain under different performance levels. Experimental results indicated that EEG spectral dynamics highly correlated with performance lapses when driving involved kinesthetic feedback. Furthermore, in the realistic environment involving both visual and kinesthetic feedback, a transitive relationship of power spectra between optimal-, suboptimal-, and poor-performance groups was found predominately across most of the independent components. In contrast to the static environment with visual input only, kinesthetic feedback reduced theta-power augmentation in the central and frontal components when preparing for action and error monitoring, while strengthening alpha suppression in the central component while steering the wheel. In terms of behavior, subjects tended to have a short response time to process unexpected events with the assistance of kinesthesia, yet only when their performance was optimal. Decrease in attentional demand, facilitated by kinesthetic feedback, eventually significantly increased the reaction time in the suboptimal-performance state. Neurophysiological evidence of mutual relationships between behavioral performance and neurocognition in complex task paradigms and experimental environments, presented in this study, might elucidate our understanding of distributed brain dynamics, supporting natural human cognition and complex coordinated, multi-joint naturalistic behavior, and lead to improved understanding of brain-behavior relations in operating environments. © 2014 Elsevier Inc

Novel technologies for the detection and mitigation of drowsy driving

In the human control of motor vehicles, there are situations regularly encountered wherein the vehicle operator becomes drowsy and fatigued due to the influence of long work days, long driving hours, or low amounts of sleep. Although various methods are currently proposed to detect drowsiness in the operator, they are either obtrusive, expensive, or otherwise impractical. The method of drowsy driving detection through the collection of Steering Wheel Movement (SWM) signals has become an important measure as it lends itself to accurate, effective, and cost-effective drowsiness detection. In this dissertation, novel technologies for drowsiness detection using Inertial Measurement Units (IMUs) are investigated and described. IMUs are an umbrella group of kinetic sensors (including accelerometers and gyroscopes) which transduce physical motions into data. Driving performances were recorded using IMUs as the primary sensors, and the resulting data were used by artificial intelligence algorithms, specifically Support Vector Machines (SVMs) to determine whether or not the individual was still fit to operate a motor vehicle. Results demonstrated high accuracy of the method in classifying drowsiness. It was also shown that the use of a smartphone-based approach to IMU monitoring of drowsiness will result in the initiation of feedback mechanisms upon a positive detection of drowsiness. These feedback mechanisms are intended to notify the driver of their drowsy state, and to dissuade further driving which could lead to crashes and/or fatalities. The novel methods not only demonstrated the ability to qualitatively determine a drivers drowsy state, but they were also low-cost, easy to implement, and unobtrusive to drivers. The efficacy, ease of use, and ease of access to these methods could potentially eliminate many barriers to the implementation of the technologies. Ultimately, it is hoped that these findings will help enhance traveler safety and prevent deaths and injuries to users

Sitting behaviour-based pattern recognition for predicting driver fatigue

The proposed approach based on physiological characteristics of sitting behaviours and sophisticated machine learning techniques would enable an effective and practical solution to driver fatigue prognosis since it is insensitive to the illumination of driving environment, non-obtrusive to driver, without violating driver’s privacy, more acceptable by drivers

Predicting Sleepiness from Driving Behaviour

This research investigates the use of objective EEG analysis to determine multiple levels of sleepiness in drivers. In the literature, current methods propose a binary (awake or sleep) or ternary (awake, drowsy or sleep) classification of sleepiness. Having few classification of sleepiness increases the risk of the driver reaching dangerous levels of sleepiness before a safety system can prevent it. Also, these methods are based on subjective analysis of physiological variables, which leads to lack of reproducibility and loss of data, when a lack of consensus is reached amongst the EEG experts. Therefore, the doctoral challenge was to determine whether multiple levels of sleepiness could be defined with high accuracy, using an objective analysis of EEG, a reliable indicator of sleepiness. The study identified awake, post-awake, pre-sleep and sleep as the multiple levels of sleepiness through the objective analysis of EEG. The research used Neural Networks, a type of Machine Learning algorithm, to determine the accuracy of the proposed multiple levels of sleepiness. The Neural Networks were trained using driving and physiological behaviour. The EEG data and the driving and physiological variables were obtained through a series of experiments aimed to induce sleepiness, conducted in the driving simulator at the University of Leeds. As the Neural Network obtained high accuracy when differentiating between awake and sleep and between post-awake and pre-sleep, it led to the conclusion that the proposed objective classification based on objective EEG analysis was suitable. However, this study did not reach the highest levels of accuracy when the 4 levels of sleepiness are combined, nevertheless the solutions proposed by the researcher to be carried in future work can contribute towards increasing the accuracy of the proposed method

Étude de corrélats électrophysiologiques pour la discrimination d'états de fatigue et de charge mentale : apports pour les interfaces cerveau-machine passives

Mental state estimation on the basis of cerebral activity and its resulting physiological activities has become a challenge for passive Brain-Computer Interfaces (BCI), in particular to address a need in neuroergonomics. This thesis work focuses on mental fatigue and workload estimation. Its purpose is to provide efficient and realistic processing chains. Thus, one issue was the modulation of workload markers as well as classification performance robustness depending on time-on-task (TOT). The impact of workload and TOT on attentional state markers was also assessed. For those purposes, an experimental protocol was implemented to collect the electroencephalographic (EEG), cardiac (ECG) and ocular (EOG) signals from healthy volunteers as they performed for a prolonged period of time a task that mixes working memory load and selective attention. Efficient signal processing chains that include spatial filtering and classification steps were designed in order to better estimate these mental states. The relevance of several electrophysiological markers was compared, among which spontaneous EEG activity and event-related potentials (ERPs), as well as various preprocessing steps such as spatial filtering methods for ERPs. Interaction effects between mental states were brought to light. In particular, TOT negatively impacted mental workload estimation when using power features. However, the chain based on ERPs was robust to this effect. A comparison of the type of stimuli that can be used to elicit the ERPs revealed that task-independent probes still allow very high performance, which shows their relevance for real-life implementation. Lastly, ongoing work that aims at assessing task-robust workload markers, as well as the usefulness of auditory ERPs in a single-stimulus paradigm will be presented as prospects.L'estimation de l'état mental d'un individu sur la base de son activité cérébrale et de ses activités physiologiques résultantes est devenue l'un des challenges des interfaces cerveau-machine (ICM) dites passives, dans le but notamment de répondre à un besoin en neuroergonomie. Ce travail de thèse se focalise sur l'estimation des états de fatigue et de charge mentale. Son objectif est de proposer des chaines de traitement efficaces et réalistes dans leur mise en œuvre. Ainsi, un des points à l'étude a été la modulation des indicateurs de charge ainsi que la robustesse des performances de classification en fonction du temps passé sur une tâche (TPT). L'impact de la charge et du TPT sur les marqueurs d'état attentionnel a aussi été évalué. Pour ce faire, un protocole expérimental a été mis en œuvre afin de recueillir les signaux électro-encéphalographiques (EEG), cardiaques (ECG) et oculaires (EOG) de participants volontaires sains lors de la réalisation prolongée d'une tâche combinant charge en mémoire de travail et attention sélective. Des chaînes de traitement performantes incluant une étape de filtrage spatial et une classification supervisée ont été mises en place afin de classer au mieux ces états. La pertinence de plusieurs marqueurs électrophysiologiques a été comparée, notamment l'activité EEG spontanée et les potentiels évoqués (PEs), ainsi que différentes étapes de prétraitement dont les méthodes de filtrage spatial pour PEs. Des effets d'interactions ont été mis au jour entre les différents états mentaux, dont un effet négatif du TPT sur les performances en classification de la charge mentale lorsque l'on utilise des marqueurs mesurant la puissance moyenne de l'EEG dans des bandes de fréquence d'intérêt. La chaîne basée sur les PEs est en revanche robuste à cet effet. Une comparaison du type de stimuli utilisables pour éliciter les PEs a révélé que des stimuli tâche-indépendants permettent tout de même d'obtenir des performances très élevées, ce qui montre leur pertinence pour une implémentation en situation réelle. En perspective seront présentés des travaux en cours visant à mettre en évidence des marqueurs de charge mentale robustes à la tâche, ainsi que l'utilité des potentiels évoqués auditifs en paradigme de simple stimulus

Foundations and applications of human-machine-interaction

Der vorliegende Tagungsband zur 10. Berliner Werkstatt Mensch-Maschine-Systeme gibt einen Einblick in die aktuelle Forschung im Bereich der Mensch-Maschine- Interaktion. Einen besonderen Fokus stellt das Wechselspiel von Grundlagenforschung und anwendungsbezogener Forschung dar, was sich im breiten Themenspektrum widerspiegelt, welches von theoretischen und methodischen Betrachtungen bis hin zu anwendungsnahen Fragestellungen reicht. Dabei finden Inhalte aus allen Phasen des Forschungsprozesses Beachtung, sodass auch im Rahmen der 10. Berliner Werkstatt MMS wieder sowohl neue Untersuchungskonzepte als auch abschließende Befunde diskutiert werden. Zentrale Themengebiete sind u. a. Fahrer-Fahrzeug-Interaktion, Assistenzsysteme, User Experience, Usability, Ubiquitous Computing, Mixed & Virtual Reality, Robotics & Automation, Wahrnehmungsspezifika sowie Psychophysiologie und Beanspruchung in der Mensch-Maschine-Interaktion.The proceedings of the 10th Berlin Workshop Human-Machine-Systems provide an insight into the current research in the field of human-machine-interaction. The main focus lies on the interplay between basic and applied research, which is reflected in the wide range of subjects: from theoretical and methodological issues to application oriented considerations. Again all stages of the research process are represented in the contributions of the 10th Berlin Workshop HMS. This means new research concepts as well as final results are subject of this volume. Central topics include driver-vehicleinteraction, assistance systems, user experience, usability, ubiquitous computing, mixed and virtual reality, robotics & automation, perception specifics, as well as psychophysiology and workload in human-machine-interaction

Visual complexity in human-machine interaction = Visuelle Komplexität in der Mensch-Maschine Interaktion

Visuelle Komplexität wird oft als der Grad an Detail oder Verworrenheit in einem Bild definiert (Snodgrass & Vanderwart, 1980). Diese hat Einfluss auf viele Bereiche des menschlichen Lebens, darunter auch solche, die die Interaktion mit Technologie invol-vieren. So wurden Effekte visueller Komplexität etwa im Straßenverkehr (Edquist et al., 2012; Mace & Pollack, 1983) oder bei der Interaktion mit Software (Alemerien & Magel, 2014) oder Webseiten (Deng & Poole, 2010; Tuch et al., 2011) nachgewie-sen. Obwohl die Erforschung visueller Komplexität bereits bis auf die Gestaltpsycho-logen zurückgeht, welche etwa mit dem Gestaltprinzip der Prägnanz die Bedeutung von Simplizität und Komplexität im Wahrnehmungsprozess verankerten (Koffka, 1935; Wertheimer, 1923), sind weder die Einflussfaktoren visueller Komplexität, noch die Zusammenhänge mit Blickbewegungen oder mentaler Beanspruchung bisher ab-schließend erforscht. Diese Punkte adressiert die vorliegende Arbeit mithilfe von vier empirischen Forschungsarbeiten. In Studie 1 wird anhand der Komplexität von Videos in Leitwarten sowie der Effekte auf subjektive, physiologische und Leistungsparameter mentaler Beanspruchung die Bedeutung des Konstruktes im Bereich der Mensch-Maschine Interaktion untersucht. Studie 2 betrachtet die dimensionale Struktur und die Bedeutung verschiedener Ein-flussfaktoren visueller Komplexität genauer, wobei unterschiedliches Stimulusmaterial genutzt wird. In Studie 3 werden mithilfe eines experimentellen Ansatzes die Auswir-kungen von Einflussfaktoren visueller Komplexität auf subjektive Bewertungen sowie eine Auswahl okularer Parameter untersucht. Als Stimuli dienen dabei einfache, schwarz-weiße Formenmuster. Zudem werden verschiedene computationale und oku-lare Parameter genutzt, um anhand dieser Komplexitätsbewertungen vorherzusagen. Dieser Ansatz wird in Studie 4 auf Screenshots von Webseiten übertragen, um die Aussagekraft in einem anwendungsnahen Bereich zu untersuchen. Neben vorangegangenen Forschungsarbeiten legen insbesondere die gefundenen Zusammenhänge mit mentaler Beanspruchung nahe, dass visuelle Komplexität ein relevantes Konstrukt im Bereich der Mensch-Maschine Interaktion darstellt. Dabei haben insbesondere quantitative und strukturelle, aber potentiell auch weitere Aspekte Einfluss auf die Bewertung visueller Komplexität sowie auf das Blickverhalten der Be-trachter. Die gewonnenen Ergebnisse erlauben darüber hinaus Rückschlüsse auf die Zusammenhänge mit computationalen Maßen, welche in Kombination mit okularen Parametern gut für die Vorhersage von Komplexitätsbewertungen geeignet sind. Die Erkenntnisse aus den durchgeführten Studien werden im Kontext vorheriger For-schungsarbeiten diskutiert. Daraus wird ein integratives Forschungsmodell visueller Komplexität in der Mensch-Maschine-Interaktion abgeleitet

Bioinspired metaheuristic algorithms for global optimization

This paper presents concise comparison study of newly developed bioinspired algorithms for global optimization problems. Three different metaheuristic techniques, namely Accelerated Particle Swarm Optimization (APSO), Firefly Algorithm (FA), and Grey Wolf Optimizer (GWO) are investigated and implemented in Matlab environment. These methods are compared on four unimodal and multimodal nonlinear functions in order to find global optimum values. Computational results indicate that GWO outperforms other intelligent techniques, and that all aforementioned algorithms can be successfully used for optimization of continuous functions