On The Effects Of Data Normalisation For Domain Adaptation On EEG Data

In the Machine Learning (ML) literature, a well-known problem is the Dataset Shift problem where, differently from the ML standard hypothesis, the data in the training and test sets can follow different probability distributions, leading ML systems toward poor generalisation performances. This problem is intensely felt in the Brain-Computer Interface (BCI) context, where bio-signals as Electroencephalographic (EEG) are often used. In fact, EEG signals are highly non-stationary both over time and between different subjects. To overcome this problem, several proposed solutions are based on recent transfer learning approaches such as Domain Adaption (DA). In several cases, however, the actual causes of the improvements remain ambiguous. This paper focuses on the impact of data normalisation, or standardisation strategies applied together with DA methods. In particular, using \textit{SEED}, \textit{DEAP}, and \textit{BCI Competition IV 2a} EEG datasets, we experimentally evaluated the impact of different normalization strategies applied with and without several well-known DA methods, comparing the obtained performances. It results that the choice of the normalisation strategy plays a key role on the classifier performances in DA scenarios, and interestingly, in several cases, the use of only an appropriate normalisation schema outperforms the DA technique.Comment: Published in its final version on Engineering Applications of Artificial Intelligence (EAAI) https://doi.org/10.1016/j.engappai.2023.10620

Developing robust movement decoders for local field potentials

textBrain Computer Interfaces (BCI) are devices that translate acquired neural signals to command and control signals. Applications of BCI include neural rehabilitation and neural prosthesis (thought controlled wheelchair, thought controlled speller etc.) to aid patients with disabilities and to augment human computer interaction. A successful practical BCI requires a faithful acquisition modality to record high quality neural signals; a signal processing system to construct appropriate features from these signals; and an algorithm to translate these features to appropriate outputs. Intracortical recordings like local field potentials provide reliable high SNR signals over long periods and suit BCI applications well. However, the non-stationarity of neural signals poses a challenge in robust decoding of subject behavior. Most BCI research focuses either on developing daily re-calibrated decoders that require exhaustive training sessions; or on providing cross-validation results. Such results ignore the variation of signal characteristics over different sessions and provide an optimistic estimate of BCI performance. Specifically, traditional BCI algorithms fail to perform at the same level on chronological data recordings. Neural signals are susceptible to variations in signal characteristics due to changes in subject behavior and learning, and variability in electrode characteristics due to tissue interactions. While training day-specific BCI overcomes signal variability, BCI re-training causes user frustration and exhaustion. This dissertation presents contributions to solve these challenges in BCI research. Specifically, we developed decoders trained on a single recording session and applied them on subsequently recorded sessions. This strategy evaluates BCI in a practical scenario with a potential to alleviate BCI user frustration without compromising performance. The initial part of the dissertation investigates extracting features that remain robust to changes in neural signal over several days of recordings. It presents a qualitative feature extraction technique based on ranking the instantaneous power of multichannel data. These qualitative features remain robust to outliers and changes in the baseline of neural recordings, while extracting discriminative information. These features form the foundation in developing robust decoders. Next, this dissertation presents a novel algorithm based on the hypothesis that multiple neural spatial patterns describe the variation in behavior. The presented algorithm outperforms the traditional methods in decoding over chronological recordings. Adapting such a decoder over multiple recording sessions (over 6 weeks) provided > 90% accuracy in decoding eight movement directions. In comparison, performance of traditional algorithms like Common Spatial Patterns deteriorates to 16% over the same time. Over time, adaptation reinforces some spatial patterns while diminishing others. Characterizing these spatial patterns reduces model complexity without user input, while retaining the same accuracy levels. Lastly, this dissertation provides an algorithm that overcomes the variation in recording quality. Chronic electrode implantation causes changes in signal-to-noise ratio (SNR) of neural signals. Thus, some signals and their corresponding features available during training become unavailable during testing and vice-versa. The proposed algorithm uses prior knowledge on spatial pattern evolution to estimate unknown neural features. This algorithm overcomes SNR variations and provides up to 93% decoding of eight movement directions over 6 weeks. Since model training requires only one session, this strategy reduces user frustration. In a practical closed-loop BCI, the user learns to produce stable spatial patterns, which improves performance of the proposed algorithms.Electrical and Computer Engineerin

BCIAUT-P300: A Multi-Session and Multi-Subject Benchmark Dataset on Autism for P300-Based Brain-Computer-Interfaces

There is a lack of multi-session P300 datasets for Brain-Computer Interfaces (BCI). Publicly available datasets are usually limited by small number of participants with few BCI sessions. In this sense, the lack of large, comprehensive datasets with various individuals and multiple sessions has limited advances in the development of more effective data processing and analysis methods for BCI systems. This is particularly evident to explore the feasibility of deep learning methods that require large datasets. Here we present the BCIAUT-P300 dataset, containing 15 autism spectrum disorder individuals undergoing 7 sessions of P300-based BCI joint-attention training, for a total of 105 sessions. The dataset was used for the 2019 IFMBE Scientific Challenge organized during MEDICON 2019 where, in two phases, teams from all over the world tried to achieve the best possible object-detection accuracy based on the P300 signals. This paper presents the characteristics of the dataset and the approaches followed by the 9 finalist teams during the competition. The winner obtained an average accuracy of 92.3% with a convolutional neural network based on EEGNet. The dataset is now publicly released and stands as a benchmark for future P300-based BCI algorithms based on multiple session data

Review of the BCI Competition IV

The BCI competition IV stands in the tradition of prior BCI competitions that aim to provide high quality neuroscientific data for open access to the scientific community. As experienced already in prior competitions not only scientists from the narrow field of BCI compete, but scholars with a broad variety of backgrounds and nationalities. They include high specialists as well as students. The goals of all BCI competitions have always been to challenge with respect to novel paradigms and complex data. We report on the following challenges: (1) asynchronous data, (2) synthetic, (3) multi-class continuous data, (4) session-to-session transfer, (5) directionally modulated MEG, (6) finger movements recorded by ECoG. As after past competitions, our hope is that winning entries may enhance the analysis methods of future BCIs.BMBF, 01IB001A, LOKI - Lernen zur Organisation komplexer Systeme der Informationsverarbeitung - Lernen im Kontext der SzenenanalyseBMBF, 01GQ0850, Bernstein Fokus Neurotechnologie - Nichtinvasive Neurotechnologie für Mensch-Maschine InteraktionEC/FP7/224631/EU/Tools for Brain-Computer Interaction/TOBIEC/FP7/216886/EU/Pattern Analysis, Statistical Modelling and Computational Learning 2/PASCAL2BMBF, 01GQ0420, Verbundprojekt: Bernstein-Zentrum für Neural Dynamics, Freiburg - CNDFBMBF, 01GQ0761, Bewegungsassoziierte Aktivierung - Dekodierung bewegungsassoziierter GehirnsignaleBMBF, 01GQ0762, Bewegungsassoziierte Aktivierung - Gehirn- und Maschinenlerne

Electroencephalography brain computer interface using an asynchronous protocol

A dissertation submitted to the Faculty of Science, University of the Witwatersrand, in ful llment of the requirements for the degree of Master of Science. October 31, 2016.Brain Computer Interface (BCI) technology is a promising new channel for communication between humans and computers, and consequently other humans. This technology has the potential to form the basis for a paradigm shift in communication for people with disabilities or neuro-degenerative ailments. The objective of this work is to create an asynchronous BCI that is based on a commercial-grade electroencephalography (EEG) sensor. The BCI is intended to allow a user of possibly low income means to issue control signals to a computer by using modulated cortical activation patterns as a control signal. The user achieves this modulation by performing a mental task such as imagining waving the left arm until the computer performs the action intended by the user. In our work, we make use of the Emotiv EPOC headset to perform the EEG measurements. We validate our models by assessing their performance when the experimental data is collected using clinical-grade EEG technology. We make use of a publicly available data-set in the validation phase. We apply signal processing concepts to extract the power spectrum of each electrode from the EEG time-series data. In particular, we make use of the fast Fourier transform (FFT). Specific bands in the power spectra are used to construct a vector that represents an abstract state the brain is in at that particular moment. The selected bands are motivated by insights from neuroscience. The state vector is used in conjunction with a model that performs classification. The exact purpose of the model is to associate the input data with an abstract classification result which can then used to select the appropriate set of instructions to be executed by the computer. In our work, we make use of probabilistic graphical models to perform this association. The performance of two probabilistic graphical models is evaluated in this work. As a preliminary step, we perform classification on pre-segmented data and we assess the performance of the hidden conditional random fields (HCRF) model. The pre-segmented data has a trial structure such that each data le contains the power spectra measurements associated with only one mental task. The objective of the assessment is to determine how well the HCRF models the spatio-spectral and temporal relationships in the EEG data when mental tasks are performed in the aforementioned manner. In other words, the HCRF is to model the internal dynamics of the data corresponding to the mental task. The performance of the HCRF is assessed over three and four classes. We find that the HCRF can model the internal structure of the data corresponding to different mental tasks. As the final step, we perform classification on continuous data that is not segmented and assess the performance of the latent dynamic conditional random fields (LDCRF). The LDCRF is used to perform sequence segmentation and labeling at each time-step so as to allow the program to determine which action should be taken at that moment. The sequence segmentation and labeling is the primary capability that we require in order to facilitate an asynchronous BCI protocol. The continuous data has a trial structure such that each data le contains the power spectra measurements associated with three different mental tasks. The mental tasks are randomly selected at 15 second intervals. The objective of the assessment is to determine how well the LDCRF models the spatio-spectral and temporal relationships in the EEG data, both within each mental task and in the transitions between mental tasks. The performance of the LDCRF is assessed over three classes for both the publicly available data and the data we obtained using the Emotiv EPOC headset. We find that the LDCRF produces a true positive classification rate of 82.31% averaged over three subjects, on the validation data which is in the publicly available data. On the data collected using the Emotiv EPOC, we find that the LDCRF produces a true positive classification rate of 42.55% averaged over two subjects. In the two assessments involving the LDCRF, the random classification strategy would produce a true positive classification rate of 33.34%. It is thus clear that our classification strategy provides above random performance on the two groups of data-sets. We conclude that our results indicate that creating low-cost EEG based BCI technology holds potential for future development. However, as discussed in the final chapter, further work on both the software and low-cost hardware aspects is required in order to improve the performance of the technology as it relates to the low-cost context.LG201

Brain Music : Sistema generativo para la creación de música simbólica a partir de respuestas neuronales afectivas

gráficas, tablasEsta tesis de maestría presenta una metodología de aprendizaje profundo multimodal innovadora que fusiona un modelo de clasificación de emociones con un generador musical, con el propósito de crear música a partir de señales de electroencefalografía, profundizando así en la interconexión entre emociones y música. Los resultados alcanzan tres objetivos específicos: Primero, ya que el rendimiento de los sistemas interfaz cerebro-computadora varía considerablemente entre diferentes sujetos, se introduce un enfoque basado en la transferencia de conocimiento entre sujetos para mejorar el rendimiento de individuos con dificultades en sistemas de interfaz cerebro-computadora basados en el paradigma de imaginación motora. Este enfoque combina datos de EEG etiquetados con datos estructurados, como cuestionarios psicológicos, mediante un método de "Kernel Matching CKA". Utilizamos una red neuronal profunda (Deep&Wide) para la clasificación de la imaginación motora. Los resultados destacan su potencial para mejorar las habilidades motoras en interfaces cerebro-computadora. Segundo, proponemos una técnica innovadora llamada "Labeled Correlation Alignment"(LCA) para sonificar respuestas neurales a estímulos representados en datos no estructurados, como música afectiva. Esto genera características musicales basadas en la actividad cerebral inducida por las emociones. LCA aborda la variabilidad entre sujetos y dentro de sujetos mediante el análisis de correlación, lo que permite la creación de envolventes acústicos y la distinción entre diferente información sonora. Esto convierte a LCA en una herramienta prometedora para interpretar la actividad neuronal y su reacción a estímulos auditivos. Finalmente, en otro capítulo, desarrollamos una metodología de aprendizaje profundo de extremo a extremo para generar contenido musical MIDI (datos simbólicos) a partir de señales de actividad cerebral inducidas por música con etiquetas afectivas. Esta metodología abarca el preprocesamiento de datos, el entrenamiento de modelos de extracción de características y un proceso de emparejamiento de características mediante Deep Centered Kernel Alignment, lo que permite la generación de música a partir de señales EEG. En conjunto, estos logros representan avances significativos en la comprensión de la relación entre emociones y música, así como en la aplicación de la inteligencia artificial en la generación musical a partir de señales cerebrales. Ofrecen nuevas perspectivas y herramientas para la creación musical y la investigación en neurociencia emocional. Para llevar a cabo nuestros experimentos, utilizamos bases de datos públicas como GigaScience, Affective Music Listening y Deap Dataset (Texto tomado de la fuente)This master’s thesis presents an innovative multimodal deep learning methodology that combines an emotion classification model with a music generator, aimed at creating music from electroencephalography (EEG) signals, thus delving into the interplay between emotions and music. The results achieve three specific objectives: First, since the performance of brain-computer interface systems varies significantly among different subjects, an approach based on knowledge transfer among subjects is introduced to enhance the performance of individuals facing challenges in motor imagery-based brain-computer interface systems. This approach combines labeled EEG data with structured information, such as psychological questionnaires, through a "Kernel Matching CKA"method. We employ a deep neural network (Deep&Wide) for motor imagery classification. The results underscore its potential to enhance motor skills in brain-computer interfaces. Second, we propose an innovative technique called "Labeled Correlation Alignment"(LCA) to sonify neural responses to stimuli represented in unstructured data, such as affective music. This generates musical features based on emotion-induced brain activity. LCA addresses variability among subjects and within subjects through correlation analysis, enabling the creation of acoustic envelopes and the distinction of different sound information. This makes LCA a promising tool for interpreting neural activity and its response to auditory stimuli. Finally, in another chapter, we develop an end-to-end deep learning methodology for generating MIDI music content (symbolic data) from EEG signals induced by affectively labeled music. This methodology encompasses data preprocessing, feature extraction model training, and a feature matching process using Deep Centered Kernel Alignment, enabling music generation from EEG signals. Together, these achievements represent significant advances in understanding the relationship between emotions and music, as well as in the application of artificial intelligence in musical generation from brain signals. They offer new perspectives and tools for musical creation and research in emotional neuroscience. To conduct our experiments, we utilized public databases such as GigaScience, Affective Music Listening and Deap DatasetMaestríaMagíster en Ingeniería - Automatización IndustrialInvestigación en Aprendizaje Profundo y señales BiológicasEléctrica, Electrónica, Automatización Y Telecomunicaciones.Sede Manizale

Theoretical and experimental study of P300 ERP in the context of Brain-computer interfaces. Part I: Study and analysis of functional connectivity methods.

Trabajo Fin de Máster en Ingeniería InformáticaThe analysis of connectivity in brain networks has been widely researched and it has been shown that certain cognitive processes require the integration of distributed brain areas. Functional connectivity attempts to statistically quantify the interdependencies between these brain areas. For this study, an analysis of functional connectivity in an ERP context, more specifically on the P300 component using the Granger Causality metric was proposed. To this end, an analysis method is proposed which consists in quantifying the causality in the P300 signal and the non-P300 signal using the MVCG toolbox to determine if there are differences between the two results obtained. In this respect, a dataset from a Brain-Computer Interface (BCI) based on P300 is analyzed. Causality is determined in overlapping windows calculated from the signals under three aspects: i) Using standard electrodes, ii) Using electrodes selected by Bayesian Linear Discriminant Analysis and exhaustive search by forward selection (BLDA-FS), and iii) Using electrodes selected by the coefficient of determination (r2). Based on this analysis, it is shown that the Granger Causality metric is valid to show the existence of a significant connectivity difference between the P300 signal and the non-P300 signal. This measure shows higher connectivity values for the P300 signal and lower connectivity values for the non-P300 signal. Among the three approaches considered, the standard electrodes and the electrodes selected with BLDA-FS were found to be more discriminative in showing differences between P300 and nonP300 connectivity. Furthermore, through this study, it was possible to differentiate the level of functional connectivity between subjects with cognitive disabilities and nondisabled subjects, observing that the measured functional connectivity was higher in subjects without an underlying cognitive pathology. Studying functional connectivity with Granger Causality may help to incorporate this information as new features that allow better detection of the P300 signal and consequently improve the performance of P300-based BCIs