A LightGBM-Based EEG Analysis Method for Driver Mental States Classification

Fatigue driving can easily lead to road traffic accidents and bring great harm to individuals and families. Recently, electroencephalography- (EEG-) based physiological and brain activities for fatigue detection have been increasingly investigated. However, how to find an effective method or model to timely and efficiently detect the mental states of drivers still remains a challenge. In this paper, we combine common spatial pattern (CSP) and propose a light-weighted classifier, LightFD, which is based on gradient boosting framework for EEG mental states identification. ,e comparable results with traditional classifiers, such as support vector machine (SVM), convolutional neural network (CNN), gated recurrent unit (GRU), and large margin nearest neighbor (LMNN), show that the proposed model could achieve better classification performance, as well as the decision efficiency. Furthermore, we also test and validate that LightFD has better transfer learning performance in EEG classification of driver mental states. In summary, our proposed LightFD classifier has better performance in real-time EEG mental state prediction, and it is expected to have broad application prospects in practical brain-computer interaction (BCI)

Systematic Review of Experimental Paradigms and Deep Neural Networks for Electroencephalography-Based Cognitive Workload Detection

This article summarizes a systematic review of the electroencephalography (EEG)-based cognitive workload (CWL) estimation. The focus of the article is twofold: identify the disparate experimental paradigms used for reliably eliciting discreet and quantifiable levels of cognitive load and the specific nature and representational structure of the commonly used input formulations in deep neural networks (DNNs) used for signal classification. The analysis revealed a number of studies using EEG signals in its native representation of a two-dimensional matrix for offline classification of CWL. However, only a few studies adopted an online or pseudo-online classification strategy for real-time CWL estimation. Further, only a couple of interpretable DNNs and a single generative model were employed for cognitive load detection till date during this review. More often than not, researchers were using DNNs as black-box type models. In conclusion, DNNs prove to be valuable tools for classifying EEG signals, primarily due to the substantial modeling power provided by the depth of their network architecture. It is further suggested that interpretable and explainable DNN models must be employed for cognitive workload estimation since existing methods are limited in the face of the non-stationary nature of the signal.Comment: 10 Pages, 4 figure

Validation of fNIRS System as a Technique to Monitor Cognitive Workload

CognitiveWorkload (CW) is a key factor in the human learning context. Knowing the optimal amount of CW is essential to maximise cognitive performance, emerging as an important variable in e-learning systems and Brain-Computer Interfaces (BCI) applications. Functional Near-Infrared Spectroscopy (fNIRS) has emerged as a promising avenue of brain discovery because of its easy setup and robust results. It is, in fact, along with Electroencephalography (EEG), an encouraging technique in the context of BCI. Brain- Computer Interfaces, by tracking the user’s cognitive state, are suitable for educational systems. Thus, this work sought to validate the fNIRS technique for monitoring different CW stages. For this purpose, we acquired the fNIRS and EEG signals when performing cognitive tasks, which included a progressive increase of difficulty and simulation of the learning process. We also used the breathing sensor and the participants’ facial expressions to assess their cognitive status. We found that both visual inspections of fNIRS signals and power spectral analysis of EEG bands are not sufficient for discriminating cognitive states, nor quantify CW. However, by applying machine learning (ML) algorithms, we were able to distinguish these states with mean accuracies of 79.8%, reaching a value of 100% in one specific case. Our findings provide evidence that fNIRS technique has the potential to monitor different levels of CW. Furthermore, our results suggest that this technique allied with the EEG and combined via ML algorithms is a promising tool to be used in the e-learning and BCI fields for its skill to discriminate and characterize cognitive states.O esforço cognitivo (CW) é um factor relevante no contexto da aprendizagem humana. Conhecer a quantidade óptima de CW é essencial para maximizar o desempenho cognitivo, surgindo como uma variável importante em sistemas de e-learning e aplicações de Interfaces Cérebro-Computador (BCI). A Espectroscopia Funcional de Infravermelho Próximo (fNIRS) emergiu como uma via de descoberta do cérebro devido à sua fácil configuração e resultados robustos. É, de facto, juntamente com a Electroencefalografia (EEG), uma técnica encorajadora no contexto de BCI. As interfaces cérebro-computador, ao monitorizar o estado cognitivo do utilizador, são adequadas para sistemas educativos. Assim, este trabalho procurou validar o sistema de fNIRS como uma técnica de monitorização de CW. Para este efeito, adquirimos os sinais fNIRS e EEG aquando da execução de tarefas cognitivas, que incluiram um aumento progressivo de dificuldade e simulação do processo de aprendizagem. Utilizámos, ainda, o sensor de respiração e as expressões faciais dos participantes para avaliar o seu estado cognitivo. Verificámos que tanto a inspeção visual dos sinais de fNIRS como a análise espectral dos sinais de EEG não são suficientes para discriminar estados cognitivos, nem para quantificar o CW. No entanto, aplicando algoritmos de machine learning (ML), fomos capazes de distinguir estes estados com exatidões médias de 79.8%, chegando a atingir o valor de 100% num caso específico. Os nossos resultados fornecem provas da prospecção da técnica fNIRS para supervisionar diferentes níveis de CW. Além disso, os nossos resultados sugerem que esta técnica aliada à de EEG e combinada via algoritmos ML é uma ferramenta promissora a ser utilizada nos campos do e-learning e de BCI, pela sua capacidade de discriminar e caracterizar estados cognitivos

Inspection and maintenance KPIs to support decision making integrated into Digital Twin tool

In the H2020 European project ASHVIN “Assistants for Healthy, Safe, and Productive Virtual Construction Design, Operation & Maintenance using a Digital Twin”, a set of Key Performance Indicators (KPIs) and Performance Indicators (PIs) to plan and control productive, resource efficient, and safe maintenance are being developed for transport infrastructure. This paper is presenting PIs and KPIs for the assessment and monitoring of the following aspects: Productivity, Resource Efficiency, Cost, Health & Safety during the operational life cycle stage, which is mainly focusing on the inspection and maintenance planning. Quantifiable and measurable PIs and KPIs are proposed and applied on two demonstration projects, highway bridge in Spain and airport runway in Croatia, as part of transportation infrastructure. Proposed PIs and KPIs are integrated into digital twins of the analyzed assets and into decision making tools for risk based maintenance planning. This paper presents the overview of the proposed digital PIs and KPIs applied on two demonstration projects and the integration into decision support tools for efficient and sustainable maintenance planning

Inspection and maintenance KPIs to support decision making integrated into Digital Twin tool

In the H2020 European project ASHVIN “Assistants for Healthy, Safe, and Productive Virtual Construction Design, Operation & Maintenance using a Digital Twin”, a set of Key Performance Indicators (KPIs) and Performance Indicators (PIs) to plan and control productive, resource efficient, and safe maintenance are being developed for transport infrastructure. This paper is presenting PIs and KPIs for the assessment and monitoring of the following aspects: Productivity, Resource Efficiency, Cost, Health & Safety during the operational life cycle stage, which is mainly focusing on the inspection and maintenance planning. Quantifiable and measurable PIs and KPIs are proposed and applied on two demonstration projects, highway bridge in Spain and airport runway in Croatia, as part of transportation infrastructure. Proposed PIs and KPIs are integrated into digital twins of the analyzed assets and into decision making tools for risk based maintenance planning. This paper presents the overview of the proposed digital PIs and KPIs applied on two demonstration projects and the integration into decision support tools for efficient and sustainable maintenance planning.Postprint (published version

Induced brain activity as indicator of cognitive processes: experimental-methodical analyses and algorithms for online-applications

Die Signalverarbeitung von elektroenzephalographischen (EEG) Signalen ist ein entscheidendes Werkzeug, um die kognitiven Prozessen verstehen zu können. Beispielweise wird induzierte Hirnaktivität in mehreren Untersuchungen mit kognitiver Leistung assoziiert. Deshalb ist die Gewinnung von elektrophysiologischen Parametern grundlegend für die Charakterisierung von kognitiven Prozessen sowie von kognitiven Dysfunktionen in neurologischen Erkrankungen. Besonders bei Epilepsie treten häufig Störungen wie Gedächtnis-, oder Aufmerksamkeitsprobleme auf, zusätzlich zu Anfällen. Neurofeedback (bzw. EEG-Biofeedback) ist eine Therapiemethode, die zusätzlich zu medikamentösen- und chirurgischen Therapien bei der Behandlung vieler neurologischer Krankheiten, einschließlich Epilepsie, erfolgreich praktiziert wird. Neurofeedback wird jedoch meist dafür angewendet, eine Anfallsreduzierung zu erzielen. Dagegen wird eine Verbesserung kognitiver Fähigkeiten auf der Basis elektrophysiologischer Änderungen selten vorgesehen. Darüber hinaus sind die aktuellen Neurofeedbackstrategien für diesen Zweck ungeeignet. Der Grund dafür sind unter anderem nicht adäquate Verfahren für die Gewinnung und Quantifizierung induzierter Hirnaktivität. Unter Berücksichtigung der oben genannten Punkten wurden die kognitiven Leistungen von einer Patientengruppe (Epilepsie) und einer Probandengruppe anhand der ereignisbezogenen De-/Synchronisation (ERD/ERS) Methode untersucht. Signifikante Unterschiede wurden im Theta bzw. Alpha Band festgestellt. Diese Ergebnisse unterstützen die Verwertung von auf ERD/ERS basierten kognitiven Parametern bei Epilepsie. Anhand einer methodischen Untersuchung von dynamischen Eigenschaften wurde ein onlinefähiger ERD/ERS Algorithmus für zukünftige Neurofeedback Applikationen ausgewählt. Basierend auf dem ausgewählten Parameter wurde eine Methodik für die online Gewinnung und Quantifizierung von kognitionsbezogener induzierter Hirnaktivität entwickelt. Die dazugehörigen Prozeduren sind in Module organisiert, um die Prozessapplikabilität zu erhöhen. Mehrere Bestandteile der Methodik, einschließlich der Rolle von Elektrodenmontagen sowie die Eliminierung bzw. Reduktion der evozierten Aktivität, wurden anhand kognitiver Aufgaben evaluiert und optimiert. Die Entwicklung einer geeigneten Neurofeedback Strategie sowie die Bestätigung der psychophysiologischen Hypothese anhand einer Pilotstudie sollen Gegenstand der zukünftigen Arbeitschritte sein.Processing of electroencephalographic (EEG) signals is a key step towards understanding cognitive brain processes. Particularly, there is growing evidence that the analysis of induced brain oscillations is a powerful tool to analyze cognitive performance. Thus, the extraction of electrophysiological features characterizing not only cognitive processes but also cognitive dysfunctions by neurological diseases is fundamental. Especially in the case of epilepsy, cognitive dysfunctions such as memory or attentional problems are often present additionally to seizures. Neurofeedback (or EEG-biofeedback) is a psychological technique that, as a supplement to medication and surgical therapies, has been demonstrated to provide further improvement in many neurological diseases, including epilepsy. However, most efforts of neurofeedback have traditionally been dedicated to the reduction of seizure frequency, and little attention has been paid for improving cognitive deficits by means of specific electrophysiological changes. Furthermore, current neurofeedback approaches are not suitable for these purposes because the parameters used do not take into consideration the relationship between memory performance and event-induced brain activity. Considering all these aspects, the cognitive performance of a group of epilepsy patients and a group of healthy controls was analyzed based on the event-related de /synchronization (ERD/ERS) method. Significant differences between both populations in the theta and upper alpha bands were observed. These findings support the possible exploitation of cognitive quantitative parameters in epilepsy based on ERD/ERS. An algorithm for the online ERD/ERS calculation was selected for future neurofeedback applications, as the result of a comparative dynamic study. Subsequently, a methodology for the online extraction and quantification of cognitive-induced brain activity was developed based on the selected algorithm. The procedure is functionally organized in blocks of algorithms in order to increase applicability. Several aspects, including the role of electrode montages and the reduction or minimization of the evoked activity, were examined based on cognitive studies as part of the optimization process. Future steps should include the design of a special training paradigm as well as a pilot study for confirming the theoretical approach proposed in this work

UK Research Information Shared Service (UKRISS) Final Report, July 2014

The reporting of research information is a complex and expensive activity for research organisations (ROs). There is little alignment between funders of the reporting requests made to institutions and requests made to individual researchers about their research outputs and outcomes. This inevitably results in duplication and increased costs across the sector, whilst limiting the potential sharing and reuse of the information. The UK Research Information Shared Service (UKRISS) project conducted a feasibility and scoping study for the reporting of research information at a national level based on CERIF (Common European Research Information Format), with the objective of increasing efficiency, productivity and quality across the sector. The aim was to define and prototype solutions which are compelling, easy to use, have a low entry barrier, and support innovative information sharing and benchmarking. CERIF has emerged as the preferred format for expressing research information across Europe. To date, CERIF has been piloted for specific applications, but not as a format for reporting requirements across all UK ROs. The final report presents the work carried out by the UKRISS project, including requirements gathering, modelling and prototyping, as well as recommendation for sustainability. UKRISS was divided into two phases. Phase 1, mapping the reporting landscape, ran from March 2012 to December 2012. Phase 2, exploring delivery of potential solutions, began in February 2013 and ended in December 2013

Thought-controlled games with brain-computer interfaces

Nowadays, EEG based BCI systems are starting to gain ground in games for health research. With reduced costs and promising an innovative and exciting new interaction paradigm, attracted developers and researchers to use them on video games for serious applications. However, with researchers focusing mostly on the signal processing part, the interaction aspect of the BCIs has been neglected. A gap between classification performance and online control quality for BCI based systems has been created by this research disparity, resulting in suboptimal interactions that lead to user fatigue and loss of motivation over time. Motor-Imagery (MI) based BCIs interaction paradigms can provide an alternative way to overcome motor-related disabilities, and is being deployed in the health environment to promote the functional and structural plasticity of the brain. A BCI system in a neurorehabilitation environment, should not only have a high classification performance, but should also provoke a high level of engagement and sense of control to the user, for it to be advantageous. It should also maximize the level of control on user’s actions, while not requiring them to be subject to long training periods on each specific BCI system. This thesis has two main contributions, the Adaptive Performance Engine, a system we developed that can provide up to 20% improvement to user specific performance, and NeuRow, an immersive Virtual Reality environment for motor neurorehabilitation that consists of a closed neurofeedback interaction loop based on MI and multimodal feedback while using a state-of-the-art Head Mounted Display.Hoje em dia, os sistemas BCI baseados em EEG estão a começar a ganhar terreno em jogos relacionados com a saúde. Com custos reduzidos e prometendo um novo e inovador paradigma de interação, atraiu programadores e investigadores para usá-los em vídeo jogos para aplicações sérias. No entanto, com os investigadores focados principalmente na parte do processamento de sinal, o aspeto de interação dos BCI foi negligenciado. Um fosso entre o desempenho da classificação e a qualidade do controle on-line para sistemas baseados em BCI foi criado por esta disparidade de pesquisa, resultando em interações subótimas que levam à fadiga do usuário e à perda de motivação ao longo do tempo. Os paradigmas de interação BCI baseados em imagética motora (IM) podem fornecer uma maneira alternativa de superar incapacidades motoras, e estão sendo implementados no sector da saúde para promover plasticidade cerebral funcional e estrutural. Um sistema BCI usado num ambiente de neuro-reabilitação, para que seja vantajoso, não só deve ter um alto desempenho de classificação, mas também deve promover um elevado nível de envolvimento e sensação de controlo ao utilizador. Também deve maximizar o nível de controlo nas ações do utilizador, sem exigir que sejam submetidos a longos períodos de treino em cada sistema BCI específico. Esta tese tem duas contribuições principais, o Adaptive Performance Engine, um sistema que desenvolvemos e que pode fornecer até 20% de melhoria para o desempenho específico do usuário, e NeuRow, um ambiente imersivo de Realidade Virtual para neuro-reabilitação motora, que consiste num circuito fechado de interação de neuro-feedback baseado em IM e feedback multimodal e usando um Head Mounted Display de última geração