A Driving Performance Forecasting System Based on Brain Dynamic State Analysis Using 4-D Convolutional Neural Networks.

Vehicle accidents are the primary cause of fatalities worldwide. Most often, experiencing fatigue on the road leads to operator errors and behavioral lapses. Thus, there is a need to predict the cognitive state of drivers, particularly their fatigue level. Electroencephalography (EEG) has been demonstrated to be effective for monitoring changes in the human brain state and behavior. Thirty-seven subjects participated in this driving experiment and performed a perform lane-keeping task in a visual-reality environment. Three domains, namely, frequency, temporal, and 2-D spatial information, of the EEG channel location were comprehensively considered. A 4-D convolutional neural-network (4-D CNN) algorithm was then proposed to associate all information from the EEG signals and the changes in the human state and behavioral performance. A 4-D CNN achieves superior forecasting performance over 2-D CNN, 3-D CNN, and shallow networks. The results showed a 3.82% improvement in the root mean-square error, a 3.45% improvement in the error rate, and a 11.98% improvement in the correlation coefficient with 4-D CNN compared with 3-D CNN. The 4-D CNN algorithm extracts the significant θ and alpha activations in the frontal and posterior cingulate cortices under distinct fatigue levels. This work contributes to enhancing our understanding of deep learning methods in the analysis of EEG signals. We even envision that deep learning might serve as a bridge between translation neuroscience and further real-world applications

EEG index for control operators’ mental fatigue monitoring using interactions between brain regions

Mental fatigue is a gradual and cumulative phenomenon induced by the time spent on a tedious but mentally demanding task, which is associated with a decrease in vigilance. It may be dangerous for operators controlling air traffic or monitoring plants. An index that estimates this state on-line from EEG signals recorded in 6 brain regions is proposed. It makes use of the Frobenius distance between the EEG spatial covariance matrices of each of the 6 regions calculated on 20s epochs to a mean covariance matrix learned during an initial reference state. The index is automatically tuned from the learning set for each subject. Its performance is analyzed on data from a group of 15 subjects who performed for 90 min an experiment that modulates mental workload. It is shown that the index based on the alpha band is well correlated with an ocular index that measures external signs of mental fatigue and can accurately assess mental fatigue over long periods of time

Steering a Tractor by Means of an EMG-Based Human-Machine Interface

An electromiographic (EMG)-based human-machine interface (HMI) is a communication pathway between a human and a machine that operates by means of the acquisition and processing of EMG signals. This article explores the use of EMG-based HMIs in the steering of farm tractors. An EPOC, a low-cost human-computer interface (HCI) from the Emotiv Company, was employed. This device, by means of 14 saline sensors, measures and processes EMG and electroencephalographic (EEG) signals from the scalp of the driver. In our tests, the HMI took into account only the detection of four trained muscular events on the driver’s scalp: eyes looking to the right and jaw opened, eyes looking to the right and jaw closed, eyes looking to the left and jaw opened, and eyes looking to the left and jaw closed. The EMG-based HMI guidance was compared with manual guidance and with autonomous GPS guidance. A driver tested these three guidance systems along three different trajectories: a straight line, a step, and a circumference. The accuracy of the EMG-based HMI guidance was lower than the accuracy obtained by manual guidance, which was lower in turn than the accuracy obtained by the autonomous GPS guidance; the computed standard deviations of error to the desired trajectory in the straight line were 16 cm, 9 cm, and 4 cm, respectively. Since the standard deviation between the manual guidance and the EMG-based HMI guidance differed only 7 cm, and this difference is not relevant in agricultural steering, it can be concluded that it is possible to steer a tractor by an EMG-based HMI with almost the same accuracy as with manual steering

Analysis of Road-User Interaction by Extraction of Driver Behavior Features Using Deep Learning

In this study, an improved deep learning model is proposed to explore the complex interactions between the road environment and driver's behaviour throughout the generation of a graphical representation. The proposed model consists of an unsupervised Denoising Stacked Autoencoder (SDAE) able to provide output layers in RGB colors. The dataset comes from an experimental driving test where kinematic measures were tracked with an in-vehicle GPS device. The graphical outcomes reveal the method ability to efficiently detect patterns of simple driving behaviors, as well as the road environment complexity and some events encountered along the path

Regression Based Continuous Driving Fatigue Estimation: Towards Practical Implementation

Mental fatigue in drivers is one of the leading causes that give rise to traffic accidents. Electroencephalography (EEG) based driving fatigue studies showed promising performance in fatigue monitoring. However, complex methodologies are not suitable for practical implementation. In our simulation based setup that retained the constraints of real driving, we took a step closer to fatigue estimation in a practical scenario. We adopted a pre-processing pipeline with low computational complexity, which can be easily and practically implemented in real-time. Moreover, regression-based continuous fatigue estimation was achieved using power spectral features in conjunction with time as the fatigue label. We sought to compare three regression models and three time windows to demonstrate their effects on the performance of fatigue estimation. Dynamic time warping was proposed as a new measure for evaluating the performance of fatigue estimation. The results derived from the validation of the proposed framework on 19 subjects showed that our proposed framework was promising towards practical implementation. Fatigue estimation by the support vector regression with radial basis function kernel and 5-second window length achieved the best performance. We also provided a comprehensive analysis on the spatial distribution of channels and frequency bands mostly contributing to fatigue estimation, which can inform the feature and channel reduction for real-time fatigue monitoring in practical driving. After reducing the number of electrodes by 75%, the proposed framework retained comparable performance in fatigue estimation. This study demonstrates the feasibility and adaptability of our proposed framework in practical implementation of mental fatigue estimation

A multidisciplinary research approach for experimental applications in road-driver interaction analysis

This doctoral dissertation represents a cluster of the research activities conducted at the DICAM Department of the University of Bologna during a three years Ph.D. course. In relation to the broader research topic of “road safety”, the presented research focuses on the investigation of the interaction between the road and the drivers according to human factor principles and supported by the following strategies: 1) The multidisciplinary structure of the research team covering the following academic disciplines: Civil Engineering, Psychology, Neuroscience and Computer Science Engineering. 2) The development of several experimental real driving tests aimed to provide investigators with knowledge and insights on the relation between the driver and the surrounding road environment by focusing on the behaviour of drivers. 3) The use of innovative technologies for the experimental studies, capable to collect data of the vehicle and on the user: a GPS data recorder, for recording the kinematic parameters of the vehicle; an eye tracking device, for monitoring the drivers’ visual behaviour; a neural helmet, for the detection of drivers’ cerebral activity (electroencephalography, EEG). 4) The use of mathematical-computational methodologies (deep learning) for data analyses from experimental studies. The outcomes of this work consist of new knowledge on the casualties between drivers’ behaviour and road environment to be considered for infrastructure design. In particular, the ground-breaking results are represented by: - the reliability and effectiveness of the methodology based on human EEG signals to objectively measure driver’s mental workload with respect to different road factors; - the successful approach for extracting latent features from multidimensional driving behaviour data using a deep learning technique, obtaining driving colour maps which represent an immediate visualization with potential impacts on road safety

Intelligent Biosignal Analysis Methods

This book describes recent efforts in improving intelligent systems for automatic biosignal analysis. It focuses on machine learning and deep learning methods used for classification of different organism states and disorders based on biomedical signals such as EEG, ECG, HRV, and others

A taxonomy of fatigue concepts and their relation to hearing loss

Fatigue is common in individuals with a variety of chronic health conditions and can have significant negative effects on quality of life. Although limited in scope, recent work suggests persons with hearing loss may be at increased risk for fatigue, in part due to effortful listening that is exacerbated by their hearing impairment. However, the mechanisms responsible for hearing loss-related fatigue, and the efficacy of audiologic interventions for reducing fatigue, remain unclear. To improve our understanding of hearing loss-related fatigue, as a field it is important to develop a common conceptual understanding of this construct. In this article, the broader fatigue literature is reviewed to identify and describe core constructs, consequences, and methods for assessing fatigue and related constructs. Finally, the current knowledge linking hearing loss and fatigue is described and may be summarized as follows: Hearing impairment may increase the risk of subjective fatigue and vigor deficits; adults with hearing loss require more time to recover from fatigue after work and have more work absences; sustained, effortful, listening can be fatiguing; optimal methods for eliciting and measuring fatigue in persons with hearing loss remain unclear and may vary with listening condition; and amplification may minimize decrements in cognitive processing speed during sustained effortful listening. Future research is needed to develop reliable measurement methods to quantify hearing loss-related fatigue, explore factors responsible for modulating fatigue in people with hearing loss, and identify and evaluate potential interventions for reducing hearing loss-related fatigue

Signal processing methods for mental fatigue measurement and monitoring using EEG

Ph.DDOCTOR OF PHILOSOPH