Cognitive Workload of Tugboat Captains in Realistic Scenarios: Adaptive Spatial Filtering for Transfer Between Conditions

Changing and often class-dependent non-stationarities of signals are a big challenge in the transfer of common findings in cognitive workload estimation using Electroencephalography (EEG) from laboratory experiments to realistic scenarios or other experiments. Additionally, it often remains an open question whether actual cognitive workload reflected by brain signals was the main contribution to the estimation or discriminative and class-dependent muscle and eye activity, which can be secondary effects of changing workload levels. Within this study, we investigated a novel approach to spatial filtering based on beamforming adapted to changing settings. We compare it to no spatial filtering and Common Spatial Patterns (CSP). We used a realistic maneuvering task, as well as an auditory n-back secondary task on a tugboat simulator as two different conditions to induce workload changes on professional tugboat captains. Apart from the typical within condition classification, we investigated the ability of the different classification methods to transfer between the n-back condition and the maneuvering task. The results show a clear advantage of the proposed approach over the others in the challenging transfer setting. While no filtering leads to lowest within-condition normalized classification loss on average in two scenarios (22 and 10%), our approach using adaptive beamforming (30 and 18%) performs comparably to CSP (33 and 15%). Importantly, in the transfer from one to another setting, no filtering and CSP lead to performance around chance level (45 to 53%), while our approach in contrast is the only one capable of classifying in all other scenarios (34 and 35%) with a significant difference from chance level. The changing signal composition over the scenarios leads to a need to adapt the spatial filtering in order to be transferable. With our approach, the transfer is successful due to filtering being optimized for the extraction of neural components and additional investigation of their scalp patterns revealed mainly neural origin. Interesting findings are that rather the patterns slightly change between conditions. We conclude that the approaches with low normalized loss depend on eye and muscle activity which is successful for classification within conditions, but fail in the classifier transfer since eye and muscle contributions are highly condition-specific.EC/FP7/611570/EU/Symbiotic Mind Computer Interaction for Information Seeking/MindSeeBMBF, 01GQ0850, Verbundprojekt: Bernstein Fokus Neurotechnologie - Nichtinvasive Neurotechnologie für Mensch-Maschine Interaktion - Teilprojekte A1, A3, A4, B4, W3, Zentru

Multi-Timescale spectra as Features for continuous Workload estimation in Realistic Settings

Der Gesamttagungsband kann hier abgerufen werden: http://dx.doi.org/10.3217/978-3-85125-533-

Decoding auditory attention to instruments in polyphonic music using single-trial EEG classification

Objective. Polyphonic music (music consisting of several instruments playing in parallel) is an intuitive way of embedding multiple information streams. The different instruments in a musical piece form concurrent information streams that seamlessly integrate into a coherent and hedonistically appealing entity. Here, we explore polyphonic music as a novel stimulation approach for use in a brain–computer interface. Approach. In a multi-streamed oddball experiment, we had participants shift selective attention to one out of three different instruments in music audio clips. Each instrument formed an oddball stream with its own specific standard stimuli (a repetitive musical pattern) and oddballs (deviating musical pattern). Main results. Contrasting attended versus unattended instruments, ERP analysis shows subject- and instrument-specific responses including P300 and early auditory components. The attended instrument can be classified offline with a mean accuracy of 91% across 11 participants. Significance. This is a proof of concept that attention paid to a particular instrument in polyphonic music can be inferred from ongoing EEG, a finding that is potentially relevant for both brain–computer interface and music research

Maritime cognitive workload assessment

The human factor plays the key role for safety in many industrial and civil every-day operations in our technologized world. Human failure is more likely to cause accidents than technical failure, e.g. in the challenging job of tugboat captains. Here, cognitive workload is crucial, as its excess is a main cause of dangerous situations and accidents while being highly participant and situation dependent. However, knowing the captain’s level of workload can help to improve man-machine interaction. The main contributions of this paper is a successful workload indication and a transfer of cognitive workload knowledge from laboratory to realistic settings

Die Neurophysiologie des Elektroenzephalogramms und die Physik des Kopfes : Theorie und Anwendung für das Spontan-EEG

Neuroscientific research using Electroencephalography is one of the most important tools for understanding human brain function and dysfunction. Not many other methods can non-invasively and directly access neural activity with millisecond precision. While basic event related modulations of brain activity can easily be replicated, the spatial resolution remains poor. Thus, complex or higher brain function and detailed aspects are much more challenging to analyze. The properties of volume conduction in the human head drastically decrease the spatial resolution of EEG. The mixing of neuronal sources on the scalp is linear but spatially smeared. EEG is highly autocorrelated while additionally non-stationary and non-linear in its emergence. Special methods are needed to differentiate sources based on temporal, spectral and spatial considerations. Many approaches based on common assumptions fail in practice. We need new ideas and a paradigm shift towards new perspectives in order to advance technology. This thesis introduces a new theory for the interpretation and classification of neural signals and develops algorithms based on it. The theory includes new perspectives on volume conduction as well as the propagation of oscillations and resonances in the cortex. This thesis suggests novel approaches to models of volume conduction, spatial filtering and optimal classifiers. The new perspective on volume conduction is based on but not limited to impedance measurements and includes approximate head model, sensor position and homogeneous conductivity estimations. In spatial filtering, novel side constraints on the common spatial patterns algorithm are investigated. Optimal Bayesian classifiers are derived for the direct classification of variance data and are related to established approaches based on the logarithm of the variances or Riemannian Geometry. The new perspectives on oscillations and resonance can be linked to the genesis of spectral harmonics due to the non-linear relation of synaptic input and firing frequency in single neurons. The results show the need for new approaches in head modeling, the interpretation of oscillations, spatial filtering and classification. Furthermore, they deliver implications for the investigation of functional connectivity and neuronal dynamics: the brain is a large musical instrument with finely tuned resonances in various spectral and spatial modes where the current harmony is based on the present and past perceptual input. The single resonances non-linearly depend on each other which leads to the necessary emergence of harmonics based on the principles of compression and soft-clipping.Neurowissenschaftliche Forschung durch das Elektroenzephalogramm (EEG) ist eines der wichtigsten Werkzeuge zum Verständnis menschlicher Hirnfunktion und -dysfunktion. Nur wenige andere Methoden können nicht-invasiv und mit der Präzision von Millisekunden direkt auf die neuronale Aktivität zugreifen. Während einfache ereignisbasierte Veränderungen von Hirnaktivität leicht replizierbar sind, ist die räumliche Auflösung des EEG nicht optimal. Die Analyse detaillierter Zusammenhänge sowie komplexer und höherer Hirnfunktion bleibt eine Herausforderung. Die Volumenleitung führt zu einer schlechten räumlichen Auflösung des EEG und hat die starke Vermischung neuronaler Quellen zur Folge, während diese jedoch glücklicherweise linear ist. EEG weist durch dies und die spezielle Form seiner Nicht-Stationarität eine hochgradige Autokorrelation auf, ist jedoch zusätzlich nicht-linear in seiner Entstehung. Spezielle Forschungsansätze sind notwendig um die unterschiedlichen Quellen unter zeitlichen, räumlichen und spektralen Gesichtspunkten zu trennen. Da viele herkömmliche Ansätze versagen, braucht es neue Ideen und Perspektiven um die EEG-Technologie weiterzuentwickeln. Diese Arbeit stellt neue Theorien zur Interpretation und Klassifikation neuronaler Signale und darauf basierende Algorithmen vor. Die Theorien umfassen neue Aspekte der Volumenleitung sowie zur Entstehung und Weiterleitung von Oszillation und Resonanzen im Kortex. Biophysikalische Kopfmodelle, räumliche Filter und optimale Klassifikatoren werden daraus abgeleitet. Die Ansätze zur Volumenleitung basieren auf Impedanzmessungen, bieten jedoch darüber hinaus Näherungslösungen für individuelle Kopfmodelle sowie die Schätzung von Sensorpositionen und homogener Gewebeleitfähigkeiten. Neuartige Randbedingungen in der räumlichen Filterung werden untersucht. Bayessche optimale Klassifikatoren für Varianzdaten werden hergeleitet und in Bezug zu etablierten Ansätzen gesetzt, die auf dem Logarithmus der Varianzen oder Riemannscher Geometrie basieren. Die Entstehung im EEG messbarer spektraler Oberwellen wird auf Grund herkömmlicher Annahmen über den Zusammenhang von Feuerrate und synaptischem Input einzelner Neuronen aufgezeigt. Diese Arbeit zeigt die Notwendigkeit neuer Perspektiven in der Modellierung der Volumenleitung, der Interpretation von Oszillationen, der räumlichen Filterung und der Klassifikation. Die Ergebnisse eröffnen Implikationen für die Untersuchung funktionaler Konnektivität und neuronaler Dynamik: Das Gehirn ist ein riesiges musikalisches Instrument mit fein abgestimmten Resonanzen in vielfältigen räumlichen und spektralen Moden, dessen momentane Harmonie auf gegenwärtigem und vergangenem perzeptuellem Input basiert. Die Resonanzen hängen nicht-linear voneinander ab, was zur notwendigen Entstehung von Oberwellen durch die Prinzipien der Kompression führt