Efficient Workload Classification based on Ignored Auditory Probes: A Proof of Concept

Mental workload is a mental state that is currently one of the main research focuses in neuroergonomics. It can notably be estimated using measurements in electroencephalography (EEG), a method that allows for direct mental state assessment. Auditory probes can be used to elicit event-related potentials (ERPs) that are modulated by workload. Although, some papers do report ERP modulations due to workload using attended or ignored probes, to our knowledge there is no literature regarding effective workload classification based on ignored auditory probes. In this paper, in order to efficiently estimate workload, we advocate for the use of such ignored auditory probes in a single-stimulus paradigm and a signal processing chain that includes a spatial filtering step. The effectiveness of this approach is demonstrated on data acquired from participants that performed the Multi-Attribute Task Battery – II. They carried out this task during two 10-min blocks. Each block corresponded to a workload condition that was pseudorandomly assigned. The easy condition consisted of two monitoring tasks performed in parallel, and the difficult one consisted of those two tasks with an additional plane driving task. Infrequent auditory probes were presented during the tasks and the participants were asked to ignore them. The EEG data were denoised and the probes’ ERPs were extracted and spatially filtered using a canonical correlation analysis. Next, binary classification was performed using a Fisher LDA and a fivefold cross-validation procedure. Our method allowed for a very high estimation performance with a classification accuracy above 80% for every participant, and minimal intrusiveness thanks to the use of a single-stimulus paradigm. Therefore, this study paves the way to the efficient use of ERPs for mental state monitoring in close to real-life settings and contributes toward the development of adaptive user interfaces

Advancing Pattern Recognition Techniques for Brain-Computer Interfaces: Optimizing Discriminability, Compactness, and Robustness

In dieser Dissertation formulieren wir drei zentrale Zielkriterien zur systematischen Weiterentwicklung der Mustererkennung moderner Brain-Computer Interfaces (BCIs). Darauf aufbauend wird ein Rahmenwerk zur Mustererkennung von BCIs entwickelt, das die drei Zielkriterien durch einen neuen Optimierungsalgorithmus vereint. Darüber hinaus zeigen wir die erfolgreiche Umsetzung unseres Ansatzes für zwei innovative BCI Paradigmen, für die es bisher keine etablierte Mustererkennungsmethodik gibt

Assessing the depth of cognitive processing as the basis for potential user-state adaptation

Objective: Decoding neurocognitive processes on a single-trial basis with Brain-Computer Interface (BCI) techniques can reveal the user's internal interpretation of the current situation. Such information can potentially be exploited to make devices and interfaces more user aware. In this line of research, we took a further step by studying neural correlates of different levels of cognitive processes and developing a method that allows to quantify how deeply presented information is processed in the brain. Methods/Approach: Seventeen participants took part in an EEG study in which we evaluated different levels of cognitive processing (no processing, shallow, and deep processing) within three distinct domains (memory, language, and visual imagination). Our investigations showed gradual differences in the amplitudes of event-related potentials (ERPs) and in the extend and duration of event-related desynchronization (ERD) which both correlate with task difficulty. We performed multi-modal classification to map the measured correlates of neurocognitive processing to the corresponding level of processing. Results: Successful classification of the neural components was achieved, which reflects the level of cognitive processing performed by the participants. The results show performances above chance level for each participant and a mean performance of 70–90% for all conditions and classification pairs. Significance: The successful estimation of the level of cognition on a single-trial basis supports the feasibility of user-state adaptation based on ongoing neural activity. There is a variety of potential use cases such as: a user-friendly adaptive design of an interface or the development of assistance systems in safety critical workplaces.DFG, 325093850, Open Access Publizieren 2017 - 2018 / Technische Universität Berli

Brain-wave measures of workload in advanced cockpits: The transition of technology from laboratory to cockpit simulator, phase 2

The present Phase 2 small business innovation research study was designed to address issues related to scalp-recorded event-related potential (ERP) indices of mental workload and to transition this technology from the laboratory to cockpit simulator environments for use as a systems engineering tool. The project involved five main tasks: (1) Two laboratory studies confirmed the generality of the ERP indices of workload obtained in the Phase 1 study and revealed two additional ERP components related to workload. (2) A task analysis' of flight scenarios and pilot tasks in the Advanced Concepts Flight Simulator (ACFS) defined cockpit events (i.e., displays, messages, alarms) that would be expected to elicit ERPs related to workload. (3) Software was developed to support ERP data analysis. An existing ARD-proprietary package of ERP data analysis routines was upgraded, new graphics routines were developed to enhance interactive data analysis, and routines were developed to compare alternative single-trial analysis techniques using simulated ERP data. (4) Working in conjunction with NASA Langley research scientists and simulator engineers, preparations were made for an ACFS validation study of ERP measures of workload. (5) A design specification was developed for a general purpose, computerized, workload assessment system that can function in simulators such as the ACFS

The Berlin Brain-Computer Interface: Progress Beyond Communication and Control

The combined effect of fundamental results about neurocognitive processes and advancements in decoding mental states from ongoing brain signals has brought forth a whole range of potential neurotechnological applications. In this article, we review our developments in this area and put them into perspective. These examples cover a wide range of maturity levels with respect to their applicability. While we assume we are still a long way away from integrating Brain-Computer Interface (BCI) technology in general interaction with computers, or from implementing neurotechnological measures in safety-critical workplaces, results have already now been obtained involving a BCI as research tool. In this article, we discuss the reasons why, in some of the prospective application domains, considerable effort is still required to make the systems ready to deal with the full complexity of the real world.EC/FP7/611570/EU/Symbiotic Mind Computer Interaction for Information Seeking/MindSeeEC/FP7/625991/EU/Hyperscanning 2.0 Analyses of Multimodal Neuroimaging Data: Concept, Methods and Applications/HYPERSCANNING 2.0DFG, 103586207, GRK 1589: Verarbeitung sensorischer Informationen in neuronalen Systeme

自然視条件下脳波計測の精度向上を可能にする眼球運動情報を用いた解析方法に関する研究

As the technique of electroencephalogram (EEG) developed for such many years, its application spreads and permeates into different areas, such like, clinical diagnosis, brain-computer interface, mental state estimation, and so on. Recently, using EEG as a tool for estimate people’s mental state and its extensional applications have jump into the limelight. These practical applications are urgently needed because the lack of subjectively estimating methods for the so called metal states, such as the concentration during study, the cognitive workload in driving, the calmness under mental training and so on. On the other hand, the application of EEG signals under daily life conditions especially when eye movements are totally without any constrains under a ‘fully free-view’ condition are obedient to the traditional ocular artifact suppression methods and how it meets the neuroscience standard has not been clearly expounded. This cause the ambiguities of explaining the obtain data and lead to susceptive results from data analysis. In our research, based on the basic idea of employing and extending EEG as the main tool for the estimation to mental state for daily life use, we confirmed the direction sensitivity of ocular artifacts induced by various types of eye movements and showed the most sensitive areas to the influence from it by multi zone-of-view experiment with standard neuroscience-targeted EEG devices. Enlightened from the results, we extended heuristic result on the use of more practical portable EEG devices. Besides, for a more realistic solution of the EEG based mental state estimation which is supposed to be applied for daily life environment, we studied the signal processing techniques of artifact suppression on low density electrode EEG and showed the importance of taking direction/eye position information into account when ocular artifact removal/suppression. In summary, this thesis has helped pave the practical way of using EEG signals toward the general use in daily life which has irregular eye movement patterns. We also pointed out the view-direction sensitivity of ocular artifact which helps the future studies to overcome the difficulties imposed on EEG applications by the free-view EEG tasks.九州工業大学博士学位論文 学位記番号:生工博甲第262号 学位授与年月日:平成28年3月26日1 Introduction|2 EEG measurements and ocular artifacts|3 Regression based solutions to ocular artifact suppression or removal in EEG|4 Measuring EEG with eye-tracking system|5 Direction and viewing area-sensitive influence of EOG artifacts revealed in the EEG topographic pattern analysis|6 Performance improvement of artifact removal with ocular information|7 Summary九州工業大学平成27年

Noninvasive Physiological Measures And Workload Transitions:an Investigation Of Thresholds Using Multiple Synchronized Sensors

The purpose of this study is to determine under what conditions multiple minimally intrusive physiological sensors can be used together and validly applied for use in areas which rely on adaptive systems including adaptive automation and augmented cognition. Specifically, this dissertation investigated the physiological transitions of operator state caused by changes in the level of taskload. Three questions were evaluated including (1) Do differences exist between physiological indicators when examined between levels of difficulty? (2) Are differences of physiological indicators (which may exist) between difficulty levels affected by spatial ability? (3) Which physiological indicators (if any) account for variation in performance on a spatial task with varying difficulty levels? The Modular Cognitive State Gauge model was presented and used to determine which basic physiological sensors (EEG, ECG, EDR and eye-tracking) could validly assess changes in the utilization of two-dimensional spatial resources required to perform a spatial ability dependent task. Thirty-six volunteers (20 female, 16 male) wore minimally invasive physiological sensing devices while executing a challenging computer based puzzle task. Specifically, participants were tested with two measures of spatial ability, received training, a practice session, an experimental trial and completed a subjective workload survey. The results of this experiment confirmed that participants with low spatial ability reported higher subjective workload and performed poorer when compared to those with high spatial ability. Additionally, there were significant changes for a majority of the physiological indicators between two difficulty levels and most importantly three measures (EEG, ECG and eye-tracking) were shown to account for variability in performance on the spatial task

Identification of audio evoked response potentials in ambulatory EEG data

Electroencephalography (EEG) is commonly used for observing brain function over a period of time. It employs a set of invasive electrodes on the scalp to measure the electrical activity of the brain. EEG is mainly used by researchers and clinicians to study the brain’s responses to a specific stimulus - the event-related potentials (ERPs). Different types of undesirable signals, which are known as artefacts, contaminate the EEG signal. EEG and ERP signals are very small (in the order of microvolts); they are often obscured by artefacts with much larger amplitudes in the order of millivolts. This greatly increases the difficulty of interpreting EEG and ERP signals.Typically, ERPs are observed by averaging EEG measurements made with many repetitions of the stimulus. The average may require many tens of repetitions before the ERP signal can be observed with any confidence. This greatly limits the study and useof ERPs. This project explores more sophisticated methods of ERP estimation from measured EEGs. An Optimal Weighted Mean (OWM) method is developed that forms a weighted average to maximise the signal to noise ratio in the mean. This is developedfurther into a Bayesian Optimal Combining (BOC) method where the information in repetitions of ERP measures is combined to provide a sequence of ERP estimations with monotonically decreasing uncertainty. A Principal Component Analysis (PCA) isperformed to identify the basis of signals that explains the greatest amount of ERP variation. Projecting measured EEG signals onto this basis greatly reduces the noise in measured ERPs. The PCA filtering can be followed by OWM or BOC. Finally, crosschannel information can be used. The ERP signal is measured on many electrodes simultaneously and an improved estimate can be formed by combining electrode measurements. A MAP estimate, phrased in terms of Kalman Filtering, is developed using all electrode measurements.The methods developed in this project have been evaluated using both synthetic and measured EEG data. A synthetic, multi-channel ERP simulator has been developed specifically for this project.Numerical experiments on synthetic ERP data showed that Bayesian Optimal Combining of trial data filtered using a combination of PCA projection and Kalman Filtering, yielded the best estimates of the underlying ERP signal. This method has been applied to subsets of real Ambulatory Electroencephalography (AEEG) data, recorded while participants performed a range of activities in different environments. From this analysis, the number of trials that need to be collected to observe the P300 amplitude and delay has been calculated for a range of scenarios