A LightGBM-Based EEG Analysis Method for Driver Mental States Classification

Fatigue driving can easily lead to road traffic accidents and bring great harm to individuals and families. Recently, electroencephalography- (EEG-) based physiological and brain activities for fatigue detection have been increasingly investigated. However, how to find an effective method or model to timely and efficiently detect the mental states of drivers still remains a challenge. In this paper, we combine common spatial pattern (CSP) and propose a light-weighted classifier, LightFD, which is based on gradient boosting framework for EEG mental states identification. ,e comparable results with traditional classifiers, such as support vector machine (SVM), convolutional neural network (CNN), gated recurrent unit (GRU), and large margin nearest neighbor (LMNN), show that the proposed model could achieve better classification performance, as well as the decision efficiency. Furthermore, we also test and validate that LightFD has better transfer learning performance in EEG classification of driver mental states. In summary, our proposed LightFD classifier has better performance in real-time EEG mental state prediction, and it is expected to have broad application prospects in practical brain-computer interaction (BCI)

Fleet Prognosis with Physics-informed Recurrent Neural Networks

Services and warranties of large fleets of engineering assets is a very profitable business. The success of companies in that area is often related to predictive maintenance driven by advanced analytics. Therefore, accurate modeling, as a way to understand how the complex interactions between operating conditions and component capability define useful life, is key for services profitability. Unfortunately, building prognosis models for large fleets is a daunting task as factors such as duty cycle variation, harsh environments, inadequate maintenance, and problems with mass production can lead to large discrepancies between designed and observed useful lives. This paper introduces a novel physics-informed neural network approach to prognosis by extending recurrent neural networks to cumulative damage models. We propose a new recurrent neural network cell designed to merge physics-informed and data-driven layers. With that, engineers and scientists have the chance to use physics-informed layers to model parts that are well understood (e.g., fatigue crack growth) and use data-driven layers to model parts that are poorly characterized (e.g., internal loads). A simple numerical experiment is used to present the main features of the proposed physics-informed recurrent neural network for damage accumulation. The test problem consist of predicting fatigue crack length for a synthetic fleet of airplanes subject to different mission mixes. The model is trained using full observation inputs (far-field loads) and very limited observation of outputs (crack length at inspection for only a portion of the fleet). The results demonstrate that our proposed hybrid physics-informed recurrent neural network is able to accurately model fatigue crack growth even when the observed distribution of crack length does not match with the (unobservable) fleet distribution.Comment: Data and codes (including our implementation for both the multi-layer perceptron, the stress intensity and Paris law layers, the cumulative damage cell, as well as python driver scripts) used in this manuscript are publicly available on GitHub at https://github.com/PML-UCF/pinn. The data and code are released under the MIT Licens

Fatigue detection system to aid in remote work

During the Covid-19 pandemic there was a noticeable surge in the amount of remote workers. In the aftermath of the pandemic working from home still remains a reality for many workers with noticeable impacts on the mental health of people. With the increased stress caused by current situation and the harder time establishing boundaries there was an increase in the overall stress and fatigue in workers, leading to burnouts. Fatigue detection systems are used in several areas, mainly in the automotive industry as a mean to decrease the number of accidents. This research started by approaching the Artificial Intelligence (AI) area and its domains, followed by a study of the current techniques used in order to predict fatigue. With the main ones utilising eye state, facial landmarks, electrocardiogram or heart rate. After a research into existing Fatigue detection systems was done in order to identify the strengths of solutions currently in the market, whether in the automotive industry or other applications. This thesis proposes the creation of a system able to detect fatigue in a user as well as warn him when fatigue levels increase. This system incorporates a webcam analysing the users face and performing eye state detection in order to calculate the percentage of the time the eyes are closed (PERCLOS). Heart rate data was also analysed and a model was developed in order to incorporate this data, the percentage of time the eyes are closed, the program the user has open and time of day in order to predict the level of fatigue. By combining these two different techniques this system can be more effective and more accurate in giving predictions of the level of fatigue. The review of literature showed that the conjunction of these two techniques in predicting fatigue is novelty. The developed system also contains integration with smartwatch technology in order to both harness heart rate data as well as communicate with the user via pop up notifications to inform him when fatigue levels get too high. The conclusion of this work is that eye state detection using Artificial Intelligence can achieve a high accuracy and be a reliable tool in identifying fatigue in an user. The combination of Heart Rate and PERCLOS allows the system to have a higher accuracy as well as not being completely reliant on one sensor. The creation of a fatigue prediction model was hindered by the lack of existent data in order to train a model, a problem that could be fixed with the adoption of the system in a broader scope.Durante a pandemia de Covid-19, houve um aumento notável na quantidade de trabalhadores remotos. No rescaldo da pandemia, trabalhar a partir de casa continua a ser uma realidade para muitos trabalhadores, com impactos visíveis na saúde mental das pessoas. Com o aumento do stresse causado pela situação atual e a dificuldade de estabelecer limites, houve um aumento do stresse geral e da fadiga dos trabalhadores, levando ao esgotamento. Os sistemas de detecção de fadiga são utilizados em diversas áreas, principalmente na indústria automobilística como forma de diminuir o número de acidentes. Este estudo começou por abordar a área de Inteligência Artificial (IA) e os seus domínios, seguida de um estudo das técnicas atuais utilizadas para prever a fadiga. Com os principais utilizando o estado dos olhos, pontos de referência faciais, eletrocardiograma ou frequência cardíaca. Depois foi feita uma pesquisa sobre os sistemas de detecção de fadiga existentes de forma a identificar os pontos fortes das soluções actualmente no mercado, quer seja na indústria automóvel ou outras aplicações. Esta dissertação propõe a criação de um sistema capaz de detectar fadiga num utilizador, bem como alertar quando os níveis de fadiga aumentam. Este sistema incorpora uma webcam que analisa a face do utilizador e realiza a detecção do estado dos olhos para calcular a percentagem de tempo em que os olhos estão fechados (PERCLOS). Os dados de frequência cardíaca também foram analisados e um modelo foi desenvolvido para incorporar estes dados, a percentagem de tempo que os olhos ficam fechados, o programa que o utilizador tem aberto e a hora do dia para prever o nível de fadiga. Ao combinar essas duas técnicas diferentes, este sistema pode ser mais eficaz e mais preciso em fornecer previsões do nível de fadiga. A revisão da literatura mostrou que a conjunção dessas duas técnicas na previsão da fadiga é novidade. O sistema desenvolvido também contém integração com a tecnologia smartwatch para aproveitar os dados da frequência cardíaca e comunicar com o utilizador por meio de notificações pop-up para informá-lo quando os níveis de fadiga se encontrarem altos. A conclusão deste trabalho é que a detecção do estado ocular usando Inteligência Artificial pode alcançar uma alta precisão e ser uma ferramenta confiável na identificação de fadiga num utilizador. A combinação da frequência cardíaca e PERCLOS permite que o sistema tenha maior precisão, além de não depender completamente de um unico sensor. A criação de um modelo de previsão de fadiga foi dificultada pela falta de dados existentes para treinar um modelo, problema que poderia ser colmatado com a adoção do sistema numa população maior

A systematic review of physiological signals based driver drowsiness detection systems.

Driving a vehicle is a complex, multidimensional, and potentially risky activity demanding full mobilization and utilization of physiological and cognitive abilities. Drowsiness, often caused by stress, fatigue, and illness declines cognitive capabilities that affect drivers' capability and cause many accidents. Drowsiness-related road accidents are associated with trauma, physical injuries, and fatalities, and often accompany economic loss. Drowsy-related crashes are most common in young people and night shift workers. Real-time and accurate driver drowsiness detection is necessary to bring down the drowsy driving accident rate. Many researchers endeavored for systems to detect drowsiness using different features related to vehicles, and drivers' behavior, as well as, physiological measures. Keeping in view the rising trend in the use of physiological measures, this study presents a comprehensive and systematic review of the recent techniques to detect driver drowsiness using physiological signals. Different sensors augmented with machine learning are utilized which subsequently yield better results. These techniques are analyzed with respect to several aspects such as data collection sensor, environment consideration like controlled or dynamic, experimental set up like real traffic or driving simulators, etc. Similarly, by investigating the type of sensors involved in experiments, this study discusses the advantages and disadvantages of existing studies and points out the research gaps. Perceptions and conceptions are made to provide future research directions for drowsiness detection techniques based on physiological signals. [Abstract copyright: © The Author(s), under exclusive licence to Springer Nature B.V. 2022. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Computing driver tiredness and fatigue in automobile via eye tracking and body movements

The aim of this paper is to classify the driver tiredness and fatigue in automobile via eye tracking and body movements using deep learning based Convolutional Neural Network (CNN) algorithm. Vehicle driver face localization serves as one of the most widely used real-world applications in fields like toll control, traffic accident scene analysis, and suspected vehicle tracking. The research proposed a CNN classifier for simultaneously localizing the region of human face and eye positioning. The classifier, rather than bounding rectangles, gives bounding quadrilaterals, which gives a more precise indication for vehicle driver face localization. The adjusted regions are preprocessed to remove noise and passed to the CNN classifier for real time processing. The preprocessing of the face features extracts connected components, filters them by size, and groups them into face expressions. The employed CNN is the well-known technology for human face recognition. One we aim to extract the facial landmarks from the frames, we will then leverage classification models and deep learning based convolutional neural networks that predict the state of the driver as 'Alert' or 'Drowsy' for each of the frames extracted. The CNN model could predict the output state labels (Alert/Drowsy) for each frame, but we wanted to take care of sequential image frames as that is extremely important while predicting the state of an individual. The process completes, if all regions have a sufficiently high score or a fixed number of retries are exhausted. The output consists of the detected human face type, the list of regions including the extracted mouth and eyes with recognition reliability through CNN with an accuracy of 98.57% with 100 epochs of training and testing

Physiological-based Driver Monitoring Systems: A Scoping Review

A physiological-based driver monitoring system (DMS) has attracted research interest and has great potential for providing more accurate and reliable monitoring of the driver’s state during a driving experience. Many driving monitoring systems are driver behavior-based or vehicle-based. When these non-physiological based DMS are coupled with physiological-based data analysis from electroencephalography (EEG), electrooculography (EOG), electrocardiography (ECG), and electromyography (EMG), the physical and emotional state of the driver may also be assessed. Drivers’ wellness can also be monitored, and hence, traffic collisions can be avoided. This paper highlights work that has been published in the past five years related to physiological-based DMS. Specifically, we focused on the physiological indicators applied in DMS design and development. Work utilizing key physiological indicators related to driver identification, driver alertness, driver drowsiness, driver fatigue, and drunk driver is identified and described based on the PRISMA Extension for Scoping Reviews (PRISMA-Sc) Framework. The relationship between selected papers is visualized using keyword co-occurrence. Findings were presented using a narrative review approach based on classifications of DMS. Finally, the challenges of physiological-based DMS are highlighted in the conclusion. Doi: 10.28991/CEJ-2022-08-12-020 Full Text: PD

Sensors and Systems for Monitoring Mental Fatigue: A systematic review

Mental fatigue is a leading cause of motor vehicle accidents, medical errors, loss of workplace productivity, and student disengagements in e-learning environment. Development of sensors and systems that can reliably track mental fatigue can prevent accidents, reduce errors, and help increase workplace productivity. This review provides a critical summary of theoretical models of mental fatigue, a description of key enabling sensor technologies, and a systematic review of recent studies using biosensor-based systems for tracking mental fatigue in humans. We conducted a systematic search and review of recent literature which focused on detection and tracking of mental fatigue in humans. The search yielded 57 studies (N=1082), majority of which used electroencephalography (EEG) based sensors for tracking mental fatigue. We found that EEG-based sensors can provide a moderate to good sensitivity for fatigue detection. Notably, we found no incremental benefit of using high-density EEG sensors for application in mental fatigue detection. Given the findings, we provide a critical discussion on the integration of wearable EEG and ambient sensors in the context of achieving real-world monitoring. Future work required to advance and adapt the technologies toward widespread deployment of wearable sensors and systems for fatigue monitoring in semi-autonomous and autonomous industries is examined.Comment: 19 Pages, 3 Figure