SUBJECTIVE METHODS FOR ASSESSMENT OF DRIVER DROWSINESS

The paper deals with the issue of fatigue and sleepiness behind the wheel, which for a long time has been of vital importance for the research in the area of driver-car interaction safety. Numerous experiments on car simulators with diverse measurements to observe human behavior have been performed at the laboratories of the faculty of the authors. The paper provides analysis and an overview and assessment of the subjective (self-rating and observer rating) methods for observation of driver behavior and the detection of critical behavior in sleep deprived drivers using the developed subjective rating scales

Analysis and detection of driver fatigue caused by sleep deprivation

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Aeronautics and Astronautics, 2007.Includes bibliographical references (leaves 167-181).Human errors in attention and vigilance are among the most common causes of transportation accidents. Thus, effective countermeasures are crucial for enhancing road safety. By pursuing a practical and reliable design of an Active Safety system which aims to predict and avoid road accidents, we identify the characteristics of drowsy driving and devise a systematic way to infer the state of driver alertness based on driver-vehicle data. Although sleep and fatigue are major causes of impaired driving, neither effective regulations nor acceptable countermeasures are available yet. The first part of this thesis analyzes driver-vehicle systems with discrete sleep-deprivation levels, and reveals differences in the performance characteristics of drivers. Inspired by the human sleep-wake cycle mechanism and attributes of driver-vehicle systems, we design and perform human-in-the-loop experiments in a test bed built with STISIM Drive, an interactive fixed-based driving simulator. In the simulated driving, participants were given various driving tasks and secondary tasks for both non and partially sleep-deprived conditions. This experiment demonstrates that sleep deprivation has a greater effect on rule-based tasks than on skill-based tasks; when drivers are sleep-deprived, their performance of responding to unexpected disturbances degrades while they are robust enough to continue such routine driving tasks as straight lane tracking, following a lead vehicle, lane changes, etc. In the second part of the thesis we present both qualitative and quantitative guidelines for designing drowsy driver detection systems in a probabilistic framework based on the Bayesian network paradigm and experimental data.(cont.) We consider two major causes of sleep, i.e., sleep debt and circadian rhythm, in the framework with various driver-vehicle parameters, and also address temporal aspects of drowsiness and individual differences of subjects. The thesis concludes that detection of drowsy driving based on driver-vehicle data is a feasible but difficult problem which has diverse issues to be addressed; the ultimate challenge lies in the human operator.by Ji Hyun Yang.Ph.D

Advanced Signal Processing in Wearable Sensors for Health Monitoring

Smart, wearables devices on a miniature scale are becoming increasingly widely available, typically in the form of smart watches and other connected devices. Consequently, devices to assist in measurements such as electroencephalography (EEG), electrocardiogram (ECG), electromyography (EMG), blood pressure (BP), photoplethysmography (PPG), heart rhythm, respiration rate, apnoea, and motion detection are becoming more available, and play a significant role in healthcare monitoring. The industry is placing great emphasis on making these devices and technologies available on smart devices such as phones and watches. Such measurements are clinically and scientifically useful for real-time monitoring, long-term care, and diagnosis and therapeutic techniques. However, a pertaining issue is that recorded data are usually noisy, contain many artefacts, and are affected by external factors such as movements and physical conditions. In order to obtain accurate and meaningful indicators, the signal has to be processed and conditioned such that the measurements are accurate and free from noise and disturbances. In this context, many researchers have utilized recent technological advances in wearable sensors and signal processing to develop smart and accurate wearable devices for clinical applications. The processing and analysis of physiological signals is a key issue for these smart wearable devices. Consequently, ongoing work in this field of study includes research on filtration, quality checking, signal transformation and decomposition, feature extraction and, most recently, machine learning-based methods

Analytical Methods for High Dimensional Physiological Sensors

abstract: This dissertation proposes a new set of analytical methods for high dimensional physiological sensors. The methodologies developed in this work were motivated by problems in learning science, but also apply to numerous disciplines where high dimensional signals are present. In the education field, more data is now available from traditional sources and there is an important need for analytical methods to translate this data into improved learning. Affecting Computing which is the study of new techniques that develop systems to recognize and model human emotions is integrating different physiological signals such as electroencephalogram (EEG) and electromyogram (EMG) to detect and model emotions which later can be used to improve these learning systems. The first contribution proposes an event-crossover (ECO) methodology to analyze performance in learning environments. The methodology is relevant to studies where it is desired to evaluate the relationships between sentinel events in a learning environment and a physiological measurement which is provided in real time. The second contribution introduces analytical methods to study relationships between multi-dimensional physiological signals and sentinel events in a learning environment. The methodology proposed learns physiological patterns in the form of node activations near time of events using different statistical techniques. The third contribution addresses the challenge of performance prediction from physiological signals. Features from the sensors which could be computed early in the learning activity were developed for input to a machine learning model. The objective is to predict success or failure of the student in the learning environment early in the activity. EEG was used as the physiological signal to train a pattern recognition algorithm in order to derive meta affective states. The last contribution introduced a methodology to predict a learner's performance using Bayes Belief Networks (BBNs). Posterior probabilities of latent nodes were used as inputs to a predictive model in real-time as evidence was accumulated in the BBN. The methodology was applied to data streams from a video game and from a Damage Control Simulator which were used to predict and quantify performance. The proposed methods provide cognitive scientists with new tools to analyze subjects in learning environments.Dissertation/ThesisDoctoral Dissertation Industrial Engineering 201

Analysis and Quantification of Physical Fatigue in Automobile Drivers: A Biomedical Approach

Vehicular accidents from fatigue due to sleep-deprived driving are rapidly increasing among heavy vehicle drivers. Consequently, a critical analysis of drivers’ fatigue in real time, using established clinical parameters, and subsequent scoring is of dire need in automobile sector. Such a scoring system would be helpful in validating the fatigue detecting devices based on non-contact features that can be installed onboard. Therefore, this study aimed to analyze and quantify physical fatigue during sleep-deprived simulated driving and their utility in developing objective score for fatigue assessment. The genesis and progression of physical fatigue was also analyzed in apparently healthy and pathological condition such as tauopathy. In the first set of experiments, the behavioral changes, cognition and motor performance in response to induced fatigue was analyzed in αCAMK-II-4R tau (transgenic/Tg) mice model of tautopathy (n=24) and were compared with those of wild-type mice (n =24). The mice were subjected to Accelerated Rotarod Test (ART), Open Field Test (OFT), Elevated Plus Maze Test (EPMT), Light and Dark Transition Test (LDT) and Forced Swimming Test (FST) for a comprehensive motor and cognitive performance analysis. Results showed that, genesis and progress of fatigue followed similar trend under physiological condition and pathological condition like tauopathy although the signs and symptoms of physical fatigue in mice models of tauopathy were more pronounced compared to healthy ones. Thus, a fatigue score system developed on healthy individuals may also be applied on pathological conditions that make the subjects vulnerable to fatigue. Subsequently, to quantify the manifestations of physical fatigue, twelve seasoned drivers were subjected to simulated driving session for 30 h and electroencephalogram (EEG),electrocardiogram (ECG) and spirometer recordings were taken for each individual at 3h intervals. In addition, peripheral blood samples were collected and analyzed at 8 h intervals for random blood sugar (RBS), blood urea (BUN) and serum creatinine. Results revealed that, energy and entropy features of EEG showed significant discrimination across time points in α and θ-bands at Cz electrode. The power spectrum density of HF (high frequency) components of ECG decreased with advancing stress and fatigue indicating sympathetic predominance with severity in fatigue. Spirometer recording confirmed gradual decrease in FEV1/FVC ratio (Forced Expiratory Volume at 1stsec / Forced Vital Capacity) as fatigue progressed. On the other hand, all blood biomarkers increased with the progress of fatigue but RBS and creatinine showed better discrimination across time-points....

Saccadic eye movements estimate prolonged time awake

Prolonged time awake increases sleep drive and causes sleepiness. Increasing sleep drive induces rapid and uncontrolled sleep initiation leading to unstable cognitive performance which is comparable to alcohol intoxication. Sleepiness causes 10 – 20 % of traffic accidents hence being a major identifiable and preventable cause of accidents. Even though the severeness of sleepiness -related accidents and hazards have been recognized and the state of New Jersey (USA) even has a law that forbids driving after being awake for more than 24 h, there is no reliable on-site test for estimating total time awake of a person. A reliable, objective, and practical metrics for measuring sleepiness outside the laboratory would be valuable. This thesis presents a novel approach and examines whether an eye movement based metric could serve as an on-site test metric for time awake. The rationale for the studying the use of eye movements to estimate overall time awake is as follows: Different cognitive functions, especially attentional ones are vulnerable to sleepiness. The attentional and oculomotor processes share neuroanatomical networks in the brain and saccadic eye movements have been used to study attentional functions. Moreover, saccadic eye movements are sensitive to sleepiness. The thesis consists of two parts: 1) Algorithm development for electro-oculographic (EOG) feature extraction to enable effective and practical analyses of measurements conducted outside the laboratory, and 2) Development of an eye movement based metric to estimate prolonged time awake. Saccadic eye movements were measured from eleven healthy adults every sixth hour with EOG in a 8-minute saccade task during 60 h of prolonged time awake. The saccade task performance, estimated as the number of saccades, decreased as a function of time awake on an individual level. The saccadic performance differed between the participants but was stable within participants (tested with 5 participants). The circadian rhythm affected the saccade task performance. Thus, the three-process model of alertness (TPMA) was fitted to, and the circadian component (C-component) was removed from, the measured data. After removing the C-component, the linear model revealed a significant trend for six out of eleven participants. The results imply that saccades measured with EOG could be used as a time awake metric outside the laboratory. The metric needs individual calibration before the time awake of a person can be estimated. More research is needed to study individual differences, optimize the measurement duration, and stimulus parameters.Pitkittynyt hereilläoloaika lisää unipainetta ja siten väsymystä. Kasvava unen tarve aiheuttaa kontrolloimattomia torkahduksia, jotka heikentävät merkittävästi ihmisen tarkkaavuutta ja siten kognitiivisia toimintoja. Univajeen aiheuttama epävakaa tila on verrattavissa humalatilaan. Liikenneonnettomuuksista 10 – 20 % on väsymyksen aiheuttamia. Väsymys on näin ollen yksi suurimmista tunnetuista, estettävissä olevista onnettomuuksien syistä. Väsymyksestä johtuvien onnettomuuksien ja katastrofien vakavuus on tunnistettu; mm. New Jerseyssä (Yhdysvallat) on säädetty laki, joka kieltää ajamisen yli 24 tunnin hereilläoloajan jälkeen. Mittalaitetta, jolla kenttäolosuhteissa pystytään mittaamaan luotettavasti, objektiivisesti ja käytännöllisesti kuljettajan hereilläolon kokonaisaikaa ei kuitenkaan ole tällä hetkellä saatavilla. Tässä väitöskirjassa on kehitetty silmänliikkeisiin perustuva mittausmenetelmä, jonka avulla voidaan mitata hereilläoloaikaa laboratorion kenttäolosuhteissa, laboratorion ulkopuolella. Univajeessa kognitiiviset toiminnot heikkenevät, erityisesti tarkkaavuus sekä visuaalinen, silmänliikkeiden avulla tapahtuva ympäristön havainnointi. Tarkkaavuutta ja okulomotorisia toimintoja säätelevät osittain samat aivojen otsalohkoalueiden hermoverkot. Tästä syystä sakkadisia silmänliikkeitä voidaan käyttää sekä tarkkaavuuden että univajeen ja väsymyksen tutkimiseen. Väitöskirja koostuu kahdesta osiosta: 1) Algoritmikehitystyöstä silmänliikkeiden tunnistamiseksi luotettavasti kenttäolosuhteissa silmänliikesignaalista, 2) Silmänliikepohjaisen menetelmän kehittäminen hereilläoloajan estimointiin. Sakkadisia silmänliikkeitä mitattiin yhdeltätoista terveeltä aikuiselta kuuden tunnin välein 60 tunnin yhtäjaksoisen univajeen aikana. Silmänliikkeet rekisteröitiin elektro-okulografia (EOG) -menetelmällä 8 minuuttia kestävän sakkaditestin aikana. Tehtävässä suoriutumista arvioitiin sen aikana suoritettujen sakkadien lukumäärällä. Sakkadien lukumäärä laski hereilläoloajan funktiona kaikilla tutkittavilla. Sakkaditehtävässä suoriutuminen vaihteli henkilöiden välillä. Testin toistettavuutta tutkittiin viidellä henkilöllä ja se todettiin toistettavaksi. Vuorokaudenaika vaikutti tehtävässä suoriutumiseen ja tästä syystä vuorokausivaihteluun liittyvä sirkadiaaninen rytmi poistettiin vireystilaa mallintavan mallin avulla (three-process model of alertness, TPMA). Sirkadiaanisen rytmin poistamisen jälkeen sakkadien lukumäärän lasku hereilläoloajan funktiona oli lineaarinen kuudella tutkimushenkilöllä yhdestätoista. Väitöskirjassa esitettyjen tulosten perusteella EOG-menetelmällä mitattujen silmänliikeiden avulla voidaan estimoida hereilläoloaikaa kenttäolosuhteissa. Tällä hetkellä mittaus vaatii henkilökohtaisen kalibrointimittauksen ennen varsinaista testimittausta. Lisää tutkimustyötä tarvitaan henkilöiden yksilöllisten erojen tutkimiseen, sekä mittausasetelman optimointiin kenttäolosuhteisiin laajemmin sopivaksi

Electro-Encephalography and Electro-Oculography in Aeronautics: A Review Over the Last Decade (2010–2020)

Electro-encephalography (EEG) and electro-oculography (EOG) are methods of electrophysiological monitoring that have potentially fruitful applications in neuroscience, clinical exploration, the aeronautical industry, and other sectors. These methods are often the most straightforward way of evaluating brain oscillations and eye movements, as they use standard laboratory or mobile techniques. This review describes the potential of EEG and EOG systems and the application of these methods in aeronautics. For example, EEG and EOG signals can be used to design brain-computer interfaces (BCI) and to interpret brain activity, such as monitoring the mental state of a pilot in determining their workload. The main objectives of this review are to, (i) offer an in-depth review of literature on the basics of EEG and EOG and their application in aeronautics; (ii) to explore the methodology and trends of research in combined EEG-EOG studies over the last decade; and (iii) to provide methodological guidelines for beginners and experts when applying these methods in environments outside the laboratory, with a particular focus on human factors and aeronautics. The study used databases from scientific, clinical, and neural engineering fields. The review first introduces the characteristics and the application of both EEG and EOG in aeronautics, undertaking a large review of relevant literature, from early to more recent studies. We then built a novel taxonomy model that includes 150 combined EEG-EOG papers published in peer-reviewed scientific journals and conferences from January 2010 to March 2020. Several data elements were reviewed for each study (e.g., pre-processing, extracted features and performance metrics), which were then examined to uncover trends in aeronautics and summarize interesting methods from this important body of literature. Finally, the review considers the advantages and limitations of these methods as well as future challenges

Intelligent Biosignal Analysis Methods

This book describes recent efforts in improving intelligent systems for automatic biosignal analysis. It focuses on machine learning and deep learning methods used for classification of different organism states and disorders based on biomedical signals such as EEG, ECG, HRV, and others

Detecting fatigue in car drivers and aircraft pilots by using eye-motion metrics

Fatigue is widely recognised for risking the safety of aviation and ground transportation. To enhance transport safety, fatigue detection systems based on psychophysiological measures have been under development for many years. However, a reliable and robust fatigue detection system is still missing. This thesis starts with a literature review of fatigue concepts in the transportation field and the current psychophysiological measures to fatigue, and narrows down the focus to improving fatigue detection systems using eye-motion measures. A research gap was identified between current fatigue systems only focusing on part of sleepy symptoms and a comprehensive fatigue detection system including mental fatigue needed. To address this gap, four studies were conducted to reshape the understanding of fatigue in transportation and explore effective eye-motion metrics for indicating fatigue considering different causal factors. Studies 1 and 2 investigated the influence of two types of task-related fatigue on eye movement. Twenty participants completed a vigilance task before and after a 1-h simulator-based drive with a secondary task. Forty participants, divided equally into two groups, finished the same task before and after a 1-h and 1.5-h monotonous driving task. The results demonstrated that two types of task-related fatigue caused by cognitive overload and prolonged underload induced different physiological responses to eye-motion metrics. The results also proved that the increased mental fatigue decreased driver’s vigilance. Studies 3 and 4 simulated two hazardous fatigue scenarios for pilots. Study 3 explored the relationship between eye-motion metrics and pilot fatigue in an underload flight condition with sleep deprivation (low workload and sleep pressure). Study 4 explored the effective eye-motion metrics to estimate pilot’s cognitive fatigue imposed by time on task and high workload. The results suggested different eye-motion metrics to indicate sleepiness and mental fatigue. In addition, based on the sleepiness and mental fatigue indicators in Studies 3 and 4, several classifiers were built and evaluated to accurately detect sleepiness and mental fatigue. These findings show that considering casual factors such as sleep pressure, time on task and workload when using eye-motion metrics to detect fatigue can improve the accuracy and face validity of the current fatigue detection systems