The influence of graphical user interface on motion onset brain-computer interface performance and the effect of data augmentation on motor imagery brain-computer interface

Motor Imagery Brain Computer Interface (MI BCI) is one of the most frequently used BCI modalities, due to the versatility of its applications. However, it still has unresolved issues like time-consuming calibration, low information transfer rate, and inconsistent performance across individuals. Combining MI BCI with Motion Onset Visual Evoked Potential (mVEP) BCI in a hybrid structure may solve some of these problems. Combining MI BCI with more robust mVEP BCI, would increase the degrees of freedom thereby increasing the information transfer rate, and would also indirectly improve intrasubject consistency in performance by replacing some MI-based tasks with mVEP. Unfortunately, due to Covid -19 pandemic experimental research on hybrid BCI was not possible, therefore this thesis focuses on two BCI separately. Chapter 1 provides an overview of different BCIs modalities and the underlying neurophysiological principles, followed by the objectives of the thesis. The research contributions are also highlighted. Finally, the thesis outlines are presented at the end of this chapter. Chapter 2 presents a comprehensive state of the art to the thesis, drawing on a wide range of literature in relevant fields. Specifically, it delves into MI BCI, mVEP BCI, Deep Learning, Transfer Learning (TL), Data Augmentation (DA) and Generative Adversarial Networks (GANs). Chapter 3 investigates the effect of graphical elements, in online and offline experiments. In the offline experiment, graphical elements such as the color, size, position, and layout were explored. Replacing a default red moving bar with a green and blue bar, changing the background color from white to gray, and using smaller visual angles did not lead to statistically significant improvement in accuracy. However, the effect size of η2 (0.085) indicated a moderate effect for these changes of graphical factors. Similarly, no statistically significant difference was found for the two different layouts in online experiments. Overall, the mVEP BCI has achieved a classification accuracy of approximately 80%, and it is relatively impervious to changes in graphical interface parameters. This suggests that mVEP is a promising candidate for a hybrid BCI system combined with MI, that requires dynamic, versatile graphical design features. In Chapter 4, various DA methods are explored, including Segmentation and Recombination in Time Domain, Segmentation and Recombination in Time-Frequency Domain, and Spatial Analogy. These methods are evaluated based on three feature extraction approaches: Common Spatial Patterns, Time Domain Parameters (TDP), and Band Power. The evaluation was conducted using a validated BCI set, namely the BCI Competition IV dataset 2a, as well as a dataset obtained from our research group. The methods are effective when a small dataset of single subject are available. All three DA methods significantly affect the performance of the TDP feature extraction method. Chapter 5 explored the use of GANs for DA in combination with TL and cropped training strategies using ShallowFBCSP classifier. It also used the same validated dataset (BCI competition IV dataset 2a) as in Chapter 4. In contrast to DA method explored in Chapter 4, this DA is suitable for larger datasets and for generalizing training based on other people’s data. Applying GAN-based DA to the dataset resulted on average in a 2% improvement in average accuracy (from 68.2% to 70.7%). This study provides a novel method to enable MI GAN training with only 40 trials per participant with the rest 8 people’s data for TL, addressing the data insufficiency issue for GANs. The evaluation of generated artificial trials revealed the importance of inter-class differences in MI patterns, which can be easily identified by GANs. Overall the thesis addressed the main practical issues of both mVEP and MI BCI paving the way for their successful combination in future experiments

Topological Changes in the Functional Brain Networks Induced by Isometric Force Exertions Using a Graph Theoretical Approach: An EEG-based Neuroergonomics Study

Neuroergonomics, the application of neuroscience to human factors and ergonomics, is an emerging science focusing on the human brain concerning performance at work and in everyday settings. The advent of portable neurophysiological methods, including electroencephalography (EEG), has enabled measurements of real-time brain activity during physical tasks without restricting body movements. However, the EEG signatures of different physical exertion activity levels that involve the musculoskeletal system in everyday settings remain poorly understood. Furthermore, the assessment of functional connectivity among different brain regions during different force exertion levels remains unclear. One approach to investigating the brain connectome is to model the underlying mechanism of the brain as a complex network. This study applied employed a graph-theoretical approach to characterize the topological properties of the functional brain network induced by predefined force exertion levels, namely extremely light (EL), light (L), somewhat hard (SWH), hard (H), and extremely hard (EH) in two frequency bands, i.e., alpha and beta. Twelve female participants performed an isometric force exertion task and rated their perception of physical comfort at different physical exertion levels. A CGX-Mobile-64 EEG was used for recording spontaneous brain electrical activity. After preprocessing the EEG data, a source localization method was applied to study the functional brain connectivity at the source level. Subsequently, the alpha and beta networks were constructed by calculating the coherence between all pairs of 84 brain regions of interests that were selected using Brodmann Areas. Graph -theoretical measures were then employed to quantify the topological properties of the functional brain networks at different levels of force exertions at each frequency band. During an \u27extremely hard\u27 exertion level, a small-world network was observed for the alpha coherence network, whereas an ordered network was observed for the beta coherence network. The results suggest that high-level force exertions are associated with brain networks characterized by a more significant clustering coefficient, more global and local efficiency, and shorter characteristic path length under alpha coherence. The above suggests that brain regions are communicating and cooperating to a more considerable degree when the muscle force exertions increase to meet physically challenging tasks. The exploration of the present study extends the current understanding of the neurophysiological basis of physical efforts with different force levels of human physical exertion to reduce work-related musculoskeletal disorders