TOWARDS PREDICTION AND IMPROVEMENT OF EEG-BASED MI-BCI PERFORMANCE.

Ph.DDOCTOR OF PHILOSOPH

EEG Signal Processing in Motor Imagery Brain Computer Interfaces with Improved Covariance Estimators

Desde hace unos años hasta la actualidad, el desarrollo en el campo de los interfaces cerebro ordenador ha ido aumentando. Este aumento viene motivado por una serie de factores distintos. A medida que aumenta el conocimiento acerca del cerebro humano y como funciona (del que aún se conoce relativamente poco), van surgiendo nuevos avances en los sistemas BCI que, a su vez, sirven de motivación para que se investigue más acerca de este órgano. Además, los sistemas BCI abren una puerta para que cualquier persona pueda interactuar con su entorno independientemente de la discapacidad física que pueda tener, simplemente haciendo uso de sus pensamientos. Recientemente, la industria tecnológica ha comenzado a mostrar su interés por estos sistemas, motivados tanto por los avances con respecto a lo que conocemos del cerebro y como funciona, como por el uso constante que hacemos de la tecnología en la actuali- dad, ya sea a través de nuestros smartphones, tablets u ordenadores, entre otros muchos dispositivos. Esto motiva que compañías como Facebook inviertan en el desarrollo de sistemas BCI para que tanto personas sin discapacidad como aquellas que, si las tienen, puedan comunicarse con los móviles usando solo el cerebro. El trabajo desarrollado en esta tesis se centra en los sistemas BCI basados en movimien- tos imaginarios. Esto significa que el usuario piensa en movimientos motores que son interpretados por un ordenador como comandos. Las señales cerebrales necesarias para traducir posteriormente a comandos se obtienen mediante un equipo de EEG que se coloca sobre el cuero cabelludo y que mide la actividad electromagnética producida por el cere- bro. Trabajar con estas señales resulta complejo ya que son no estacionarias y, además, suelen estar muy contaminadas por ruido o artefactos. Hemos abordado esta temática desde el punto de vista del procesado estadístico de la señal y mediante algoritmos de aprendizaje máquina. Para ello se ha descompuesto el sistema BCI en tres bloques: preprocesado de la señal, extracción de características y clasificación. Tras revisar el estado del arte de estos bloques, se ha resumido y adjun- tado un conjunto de publicaciones que hemos realizado durante los últimos años, y en las cuales podemos encontrar las diferentes aportaciones que, desde nuestro punto de vista, mejoran cada uno de los bloques anteriormente mencionados. De manera muy resumida, para el bloque de preprocesado proponemos un método mediante el cual conseguimos nor- malizar las fuentes de las señales de EEG. Al igualar las fuentes efectivas conseguimos mejorar la estima de las matrices de covarianza. Con respecto al bloque de extracción de características, hemos conseguido extender el algoritmo CSP a casos no supervisados. Por último, en el bloque de clasificación también hemos conseguido realizar una sepa- ración de clases de manera no supervisada y, por otro lado, hemos observado una mejora cuando se regulariza el algoritmo LDA mediante un método específico para Gaussianas.The research and development in the field of Brain Computer Interfaces (BCI) has been growing during the last years, motivated by several factors. As the knowledge about how the human brain is and works (of which we still know very little) grows, new advances in BCI systems are emerging that, in turn, serve as motivation to do more re- search about this organ. In addition, BCI systems open a door for anyone to interact with their environment regardless of the physical disabilities they may have, by simply using their thoughts. Recently, the technology industry has begun to show its interest in these systems, mo- tivated both by the advances about what we know of the brain and how it works, and by the constant use we make of technology nowadays, whether it is by using our smart- phones, tablets or computers, among many other devices. This motivates companies like Facebook to invest in the development of BCI systems so that people (with or without disabilities) can communicate with their devices using only their brain. The work developed in this thesis focuses on BCI systems based on motor imagery movements. This means that the user thinks of certain motor movements that are in- terpreted by a computer as commands. The brain signals that we need to translate to commands are obtained by an EEG device that is placed on the scalp and measures the electromagnetic activity produced by the brain. Working with these signals is complex since they are non-stationary and, in addition, they are usually heavily contaminated by noise or artifacts. We have approached this subject from the point of view of statistical signal processing and through machine learning algorithms. For this, the BCI system has been split into three blocks: preprocessing, feature extraction and classification. After reviewing the state of the art of these blocks, a set of publications that we have made in recent years has been summarized and attached. In these publications we can find the different contribu- tions that, from our point of view, improve each one of the blocks previously mentioned. As a brief summary, for the preprocessing block we propose a method that lets us nor- malize the sources of the EEG signals. By equalizing the effective sources, we are able to improve the estimation of the covariance matrices. For the feature extraction block, we have managed to extend the CSP algorithm for unsupervised cases. Finally, in the classification block we have also managed to perform a separation of classes in an blind way and we have also observed an improvement when the LDA algorithm is regularized by a specific method for Gaussian distributions

Study of Adaptation Methods Towards Advanced Brain-computer Interfaces

Ph.DDOCTOR OF PHILOSOPH

Novel Transfer Learning Approaches forImproving Brain Computer Interfaces

Despite several recent advances, most of the electroencephalogram(EEG)-based brain-computer interface (BCI) applications are still limited to the laboratory due to their long calibration time. Due toconsiderable inter-subject/inter-session and intra-session variations, atime-consuming and fatiguing calibration phase is typically conductedat the beginning of each new session to acquire sufficient labelled train-ing data to train the subject-specific BCI model.This thesis focuses on developing reliable machine learning algorithmsand approaches that reduce BCI calibration time while keeping accu-racy in an acceptable range. Calibration time could be reduced viatransfer learning approaches where data from other sessions or sub-jects are mined and used to compensate for the lack of labelled datafrom the current user or session. In BCI, transfer learning can beapplied on either raw EEG, feature or classification domains.In this thesis, firstly, a novel weighted transfer learning approach isproposed in the classification domain to improve the MI-based BCIperformance when only few subject-specific trials are available fortraining.Transfer learning techniques should be applied in a different domainbefore the classification domain to improve the classification accuracyfor subjects whom their subject-specific features for different classesare not separable. Thus, secondly, this thesis proposes a novel regu-larized common spatial patterns framework based on dynamic timewarping and transfer learning (DTW-R-CSP) in raw EEG and featuredomains.In previous transfer learning approaches, it is hypothesised that thereare enough labelled trials available from the previous subjects or ses-sions. However, in the case when there are no labelled trials available from other subjects or sessions, domain adaptation transfer learningcould potentially mitigate problems of having small training size byreducing variations between the testing and training trials. Thus, todeal with non-stationarity between training and testing trials, a novelensemble adaptation framework with temporal alignment is proposed

Modelling and Classification of Motor Imagery EEG for BCI

Ph.DDOCTOR OF PHILOSOPH

Advancing Pattern Recognition Techniques for Brain-Computer Interfaces: Optimizing Discriminability, Compactness, and Robustness

In dieser Dissertation formulieren wir drei zentrale Zielkriterien zur systematischen Weiterentwicklung der Mustererkennung moderner Brain-Computer Interfaces (BCIs). Darauf aufbauend wird ein Rahmenwerk zur Mustererkennung von BCIs entwickelt, das die drei Zielkriterien durch einen neuen Optimierungsalgorithmus vereint. Darüber hinaus zeigen wir die erfolgreiche Umsetzung unseres Ansatzes für zwei innovative BCI Paradigmen, für die es bisher keine etablierte Mustererkennungsmethodik gibt

Statistical Data Modeling and Machine Learning with Applications

The modeling and processing of empirical data is one of the main subjects and goals of statistics. Nowadays, with the development of computer science, the extraction of useful and often hidden information and patterns from data sets of different volumes and complex data sets in warehouses has been added to these goals. New and powerful statistical techniques with machine learning (ML) and data mining paradigms have been developed. To one degree or another, all of these techniques and algorithms originate from a rigorous mathematical basis, including probability theory and mathematical statistics, operational research, mathematical analysis, numerical methods, etc. Popular ML methods, such as artificial neural networks (ANN), support vector machines (SVM), decision trees, random forest (RF), among others, have generated models that can be considered as straightforward applications of optimization theory and statistical estimation. The wide arsenal of classical statistical approaches combined with powerful ML techniques allows many challenging and practical problems to be solved. This Special Issue belongs to the section “Mathematics and Computer Science”. Its aim is to establish a brief collection of carefully selected papers presenting new and original methods, data analyses, case studies, comparative studies, and other research on the topic of statistical data modeling and ML as well as their applications. Particular attention is given, but is not limited, to theories and applications in diverse areas such as computer science, medicine, engineering, banking, education, sociology, economics, among others. The resulting palette of methods, algorithms, and applications for statistical modeling and ML presented in this Special Issue is expected to contribute to the further development of research in this area. We also believe that the new knowledge acquired here as well as the applied results are attractive and useful for young scientists, doctoral students, and researchers from various scientific specialties

Breaking Down the Barriers To Operator Workload Estimation: Advancing Algorithmic Handling of Temporal Non-Stationarity and Cross-Participant Differences for EEG Analysis Using Deep Learning

This research focuses on two barriers to using EEG data for workload assessment: day-to-day variability, and cross- participant applicability. Several signal processing techniques and deep learning approaches are evaluated in multi-task environments. These methods account for temporal, spatial, and frequential data dependencies. Variance of frequency- domain power distributions for cross-day workload classification is statistically significant. Skewness and kurtosis are not significant in an environment absent workload transitions, but are salient with transitions present. LSTMs improve day- to-day feature stationarity, decreasing error by 59% compared to previous best results. A multi-path convolutional recurrent model using bi-directional, residual recurrent layers significantly increases predictive accuracy and decreases cross-participant variance. Deep learning regression approaches are applied to a multi-task environment with workload transitions. Accounting for temporal dependence significantly reduces error and increases correlation compared to baselines. Visualization techniques for LSTM feature saliency are developed to understand EEG analysis model biases

Network Modeling of Motor Pathways from Neural Recordings

During cued motor tasks, for both speech and limb movement, information propagates from primary sensory areas, to association areas, to primary and supplementary motor and language areas. Through the recent advent of high density recordings at multiple scales, it has become possible to simultaneously observe activity occurring from these disparate regions at varying resolution. Models of brain activity generally used in brain-computer interface (BCI) control do not take into account the global differences in recording site function, or the interactions between them. Through the use of connectivity measures, however, it has been made possible to determine the contribution of individual recording sites to the global activity, as they vary with task progression. This dissertation extends those connectivity models to provide summary information about the importance of individual sites. This is achieved through the application of network measures on the adjacency structure determined by connectivity measures. Similarly, by analyzing the coordinated activity of all of the electrode sites simultaneously during task performance, it is possible to elucidate discrete functional units through clustering analysis of the electrode recordings. In this dissertation, I first describe a BCI system using simple motor movement imagination at single recording sites. I then incorporate connectivity through the use of TV-DBN modeling on higher resolution electrode recordings, specifically electrocorticography (ECoG). I show that PageRank centrality reveals information about task progression and regional specificity which was obscured by direct application of the connectivity measures, due to the combinatorial increase in feature dimensionality. I then show that clustering of ECoG recordings using a method to determine the inherent cluster count algorithmically provides insight into how network involvement in task execution evolves, though in a manner dependent on grid coverage. Finally, I extend clustering analysis to show how individual neurons in motor cortex form distinct functional communities. These communities are shown to be task-specific, suggesting that neurons can form functional units with distinct neural populations across multiple recording sites in a context dependent impermanent manner. This work demonstrates that network measures of connectivity models of neurophysiological recordings are a rich source of information relevant to the field of neuroscience, as well as offering the promise of improved degree-of-freedom and naturalness possible through direct BCI control. These models are shown to be useful at multiple recording scales, from cortical-area level ECoG, to highly localized single unit microelectrode recordings

A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts

Machine learning methods strive to acquire a robust model during training that can generalize well to test samples, even under distribution shifts. However, these methods often suffer from a performance drop due to unknown test distributions. Test-time adaptation (TTA), an emerging paradigm, has the potential to adapt a pre-trained model to unlabeled data during testing, before making predictions. Recent progress in this paradigm highlights the significant benefits of utilizing unlabeled data for training self-adapted models prior to inference. In this survey, we divide TTA into several distinct categories, namely, test-time (source-free) domain adaptation, test-time batch adaptation, online test-time adaptation, and test-time prior adaptation. For each category, we provide a comprehensive taxonomy of advanced algorithms, followed by a discussion of different learning scenarios. Furthermore, we analyze relevant applications of TTA and discuss open challenges and promising areas for future research. A comprehensive list of TTA methods can be found at \url{https://github.com/tim-learn/awesome-test-time-adaptation}.Comment: Discussions, comments, and questions are all welcomed in \url{https://github.com/tim-learn/awesome-test-time-adaptation