Hemodynamic Analysis for Cognitive Load Assessment and Classification in Motor Learning Tasks Using Type-2 Fuzzy Sets

The paper addresses a novel approach to assess and classify the cognitive load of subjects from their hemodynamic response while engaged in motor learning tasks, such as vehicle-driving. A set of complex motor-activity-learning stimuli for braking, steering-control and acceleration is prepared to experimentally measure and classify the cognitive load of the car-drivers in three distinct classes: High, Medium and Low. New models of General and Interval Type-2 Fuzzy classifiers are proposed to reduce the scope of uncertainty in cognitive load classification due to the fluctuation of the hemodynamic features within and across sessions. The proposed classifiers offer high classification accuracy over 96%, leaving behind the traditional type-1/type-2 fuzzy and other standard classifiers. Experiments undertaken also offer a deep biological insight concerning the shift of brain-activations from the orbito-frontal to the ventro-lateral prefrontal cortex during high-to-low transition in cognitive load. Further, the activation of the dorsolateral prefrontal cortex is also reduced during low cognitive load of subjects. The proposed research outcome may directly be utilized to identify driving learners with low cognitive load for difficult motor learning tasks, such as taking a U-turn in a narrow space and motion control on the top of a bridge to avoid possible collision with the car ahead

Biomedical Signal Analysis of the Brain and Systemic Physiology

Near-infrared spectroscopy (NIRS) is a non-invasive and easy-to-use diagnostic technique that enables real-time tissue oxygenation measurements applied in various contexts and for different purposes. Continuous monitoring with NIRS of brain oxygenation, for example, in neonatal intensive care units (NICUs), is essential to prevent lifelong disabilities in newborns. Moreover, NIRS can be applied to observe brain activity associated with hemodynamic changes in blood flow due to neurovascular coupling. In the latter case, NIRS contributes to studying cognitive processes allowing to conduct experiments in natural and socially interactive contexts of everyday life. However, it is essential to measure systemic physiology and NIRS signals concurrently. The combination of brain and body signals enables to build sophisticated systems that, for example, reduce the false alarms that occur in NICUs. Furthermore, since fNIRS signals are influenced by systemic physiology, it is essential to understand how the latter impacts brain signals in functional studies. There is an interesting brain body coupling that has rarely been investigated yet. To take full advantage of these brain and body data, the aim of this thesis was to develop novel approaches to analyze these biosignals to extract the information and identify new patterns, to solve different research or clinical questions. For this the development of new methodological approaches and sophisticated data analysis is necessary, because often the identification of these patterns is challenging or not possible with traditional methods. In such cases, automatic machine learning (ML) techniques are beneficial. The first contribution of this work was to assess the known systemic physiology augmented (f)NIRS approach for clinical use and in everyday life. Based on physiological and NIRS signals of preterm infants, an ML-based classification system has been realized, able to reduce the false alarms in NICUs by providing a high sensitivity rate. In addition, the SPA-fNIRS approach was further applied in adults during a breathing task. The second contribution of this work was the advancement of the classical fNIRS hyperscanning method by adding systemic physiology measures. For this, new biosignal analyses in the time-frequency domain have been developed and tested in a simple nonverbal synchrony task between pairs of subjects. Furthermore, based on SPA-fNIRS hyperscanning data, another ML-based system was created, which is able distinguish familiar and unfamiliar pairs with high accuracy. This approach enables to determine the strength of social bonds in a wide range of social interaction contexts. In conclusion, we were the first group to perform a SPA-fNIRS hyperscanning study capturing changes in cerebral oxygenation and hemodynamics as well as systemic physiology in two subjects simultaneously. We applied new biosignals analysis methods enabling new insights into the study of social interactions. This work opens the door to many future inter-subjects fNIRS studies with the benefit of assessing the brain-to-brain, the brain-to-body, and body-to-body coupling between pairs of subjects

Speech Processes for Brain-Computer Interfaces

Speech interfaces have become widely used and are integrated in many applications and devices. However, speech interfaces require the user to produce intelligible speech, which might be hindered by loud environments, concern to bother bystanders or the general in- ability to produce speech due to disabilities. Decoding a usera s imagined speech instead of actual speech would solve this problem. Such a Brain-Computer Interface (BCI) based on imagined speech would enable fast and natural communication without the need to actually speak out loud. These interfaces could provide a voice to otherwise mute people. This dissertation investigates BCIs based on speech processes using functional Near In- frared Spectroscopy (fNIRS) and Electrocorticography (ECoG), two brain activity imaging modalities on opposing ends of an invasiveness scale. Brain activity data have low signal- to-noise ratio and complex spatio-temporal and spectral coherence. To analyze these data, techniques from the areas of machine learning, neuroscience and Automatic Speech Recog- nition are combined in this dissertation to facilitate robust classification of detailed speech processes while simultaneously illustrating the underlying neural processes. fNIRS is an imaging modality based on cerebral blood flow. It only requires affordable hardware and can be set up within minutes in a day-to-day environment. Therefore, it is ideally suited for convenient user interfaces. However, the hemodynamic processes measured by fNIRS are slow in nature and the technology therefore offers poor temporal resolution. We investigate speech in fNIRS and demonstrate classification of speech processes for BCIs based on fNIRS. ECoG provides ideal signal properties by invasively measuring electrical potentials artifact- free directly on the brain surface. High spatial resolution and temporal resolution down to millisecond sampling provide localized information with accurate enough timing to capture the fast process underlying speech production. This dissertation presents the Brain-to- Text system, which harnesses automatic speech recognition technology to decode a textual representation of continuous speech from ECoG. This could allow to compose messages or to issue commands through a BCI. While the decoding of a textual representation is unparalleled for device control and typing, direct communication is even more natural if the full expressive power of speech - including emphasis and prosody - could be provided. For this purpose, a second system is presented, which directly synthesizes neural signals into audible speech, which could enable conversation with friends and family through a BCI. Up to now, both systems, the Brain-to-Text and synthesis system are operating on audibly produced speech. To bridge the gap to the final frontier of neural prostheses based on imagined speech processes, we investigate the differences between audibly produced and imagined speech and present first results towards BCI from imagined speech processes. This dissertation demonstrates the usage of speech processes as a paradigm for BCI for the first time. Speech processes offer a fast and natural interaction paradigm which will help patients and healthy users alike to communicate with computers and with friends and family efficiently through BCIs

Functional Neuroanatomy of Dynamic Visuo-Spatial Imagery

The aim of this thesis was the examination of the neural bases of dynamic visuo-spatial imagery. In addition to the assessment of brain activity during dy-namic visuo-spatial imagery using single-trial functional magnetic resonance im-aging (fMRI) and slow cortical potentials (SCPs), several methodological issues have been investigated. The theoretical part of this thesis consists of a selective overview of fMRI and SCPs, and of the advantages of their combination for functional neuroimaging (chapter 2). The methodological and empirical chapters include: Ø the presentation of a new, highly accurate and practicable method for the co-registration of MRI- and EEG-data (chapter 3), Ø the description of the increase in the accuracy of SCP mapping resulting from the use of individual electrode coordinates and realistic head models (chapter 4), Ø the description of regional differences in the consistency of brain activity across several executions of the same task type, as assessed by a new analysis con-cept based on single-trial fMRI data (chapter 5), Ø the demonstration of the involvement of premotor regions in dynamic visuo-spatial imagery, as assessed via a combination of single-trial fMRI and SCPs (chapter 6), Ø the description of a combined fMRI-SCP investigation in which earlier findings concerning individual differences in neural efficiency during dynamic imagery could not be replicated (chapter 7)