MEG:hen perustuvan aivo-tietokone -käyttöliittymän kehitys

Brain–computer interfaces (BCI) have recently gained interest both in basic neuroscience and clinical interventions. The majority of noninvasive BCIs measure brain activity with electroencephalography (EEG). However, the real-time signal analysis and decoding of brain activity suffer from low signal-to-noise ratio and poor spatial resolution of EEG. These limitations could be overcome by using magnetoencephalography (MEG) as an alternative measurement modality. The aim of this thesis is to develop an MEG-based BCI for decoding hand motor imagery, which could eventually serve as a therapeutic method for patients recovering from e.g. cerebral stroke. Here, machine learning methods for decoding motor imagery -related brain activity are validated with healthy subjects’ MEG measurements. The first part of the thesis (Study I) involves a comparison of feature extraction methods for classifying left- vs right-hand motor imagery (MI), and MI vs rest. It was found that spatial filtering and further extraction of bandpower features yield better classification accuracy than time–frequency features extracted from parietal gradiometers. Furthermore, prior spatial filtering improved the discrimination capability of time–frequency features. The training data for a BCI is typically collected in the beginning of each measurement session. However, as this can be time-consuming and exhausting for the subject, the training data from other subjects’ measurements could be used as well. In the second part of the thesis (Study II), methods for across-subject classification of MI were compared. The results showed that a classifier based on multi-task learning with a l2,1-norm regularized logistic regression was the best method for across-subject decoding for both MEG and EEG. In Study II, we also compared the decoding results of simultaneously measured EEG and MEG data, and investigated whether the MEG responses to passive hand movements could be used to train a classifier to detect MI. MEG yielded altogether slightly, but not significantly, better results than EEG. Training the classifiers with subject’s own or other subjects’ passive movements did not result in high accuracy, which indicates that passive movements should not be used for calibrating an MI-BCI. The methods presented in this thesis are suitable for a real-time MEG-based BCI. The decoding results can be used as a benchmark when developing other classifiers specifically for motor imagery -related MEG data.Aivo-tietokone -käyttöliittymät (brain–computer interface; BCI) ovat viime aikoina herättäneet kiinnostusta niin neurotieteen perustutkimuksessa kuin kliinisissä interventioissakin. Suurin osa ei-invasiivisista BCI:stä mittaa aivotoimintaa elektroenkefalografialla (EEG). EEG:n matala signaali-kohinasuhde ja huono avaruudellinen resoluutio kuitenkin hankaloittavat reaaliaikais-ta signaalianalyysia ja aivotoiminnan luokittelua. Nämä rajoitteet voidaan kiertää käyttämällä magnetoenkefalografiaa (MEG) vaihtoehtoisena mittausmenetelmänä. Tämän työn tavoitteena on kehittää käden liikkeen kuvittelua luokitteleva, MEG:hen perustuva BCI, jota voidaan myöhemmin käyttää terapeuttisena menetelmänä esimerkiksi aivoinfarktista toipuvien potilaiden kuntoutuk-sessa. Tutkimuksessa validoidaan terveillä koehenkilöillä tehtyjen MEG-mittausten perusteella koneoppimismenetelmiä, joilla luokitellaan liikkeen kuvittelun aiheuttamaa aivotoimintaa. Ensimmäisessä osatyössä (Tutkimus I) vertailtiin piirteenirrotusmenetelmiä, joita käytetään erottamaan toisistaan vasemman ja oikean käden kuvittelu sekä liikkeen kuvittelu ja lepotila. Ha-vaittiin, että avaruudellisesti suodatettujen signaalien taajuuskaistan teho luokittelupiirteenä tuotti parempia luokittelutarkkuuksia kuin parietaalisista gradiometreistä mitatut aika-taajuuspiirteet. Lisäksi edeltävä avaruudellinen suodatus paransi aika-taajuuspiirteiden erottelukykyä luokittelu-tehtävissä.BCI:n opetusdata kerätään yleensä kunkin mittauskerran alussa. Koska tämä voi kuitenkin olla aikaavievää ja uuvuttavaa koehenkilölle, opetusdatana voidaan käyttää myös muilta koehenkilöiltä kerättyjä mittaussignaaleja. Toisessa osatyössä (Tutkimus II) vertailtiin koehenkilöiden väliseen luo-kitteluun soveltuvia menetelmiä. Tulosten perusteella monitehtäväoppimista ja l2,1-regularisoitua logistista regressiota käyttävä luokittelija oli paras menetelmä koehenkilöiden väliseen luokitteluun sekä MEG:llä että EEG:llä. Toisessa osatyössä vertailtiin myös samanaikaisesti mitattujen MEG:n ja EEG:n tuottamia luokit-telutuloksia, sekä tutkittiin voidaanko passiivisten kädenliikkeiden aikaansaamia MEG-vasteita käyttää liikkeen kuvittelua tunnistavien luokittelijoiden opetukseen. MEG tuotti hieman, muttei merkittävästi, parempia tuloksia kuin EEG. Luokittelijoiden opetus koehenkilöiden omilla tai mui-den koehenkilöiden passiiviliikkeillä ei tuottanut hyviä luokittelutarkkuuksia, mikä osoittaa että passiiviliikkeitä ei tulisi käyttää liikkeen kuvittelua tunnistavan BCI:n kalibrointiin. Työssä esitettyjä menetelmiä voidaan käyttää reaaliaikaisessa MEG-BCI:ssä. Luokittelutuloksia voidaan käyttää vertailukohtana kehitettäessä muita liikkeen kuvitteluun liittyvän MEG-datan luokittelijoita

Spatio-spectral patterns based on stein kernel for EEG signal classification

El trastorno por déficit de atención con hiperactividad (TDAH) es un trastorno neurológico de inicio en la niñez que puede persistir en la adolescencia y la vida adulta, reduciendo la concentración, la memoria y la productividad. El principal inconveniente de las anomalías de la salud mental de este tipo es la técnica de diagnóstico tradicional, ya que se basa exclusivamente en una descripción sintomatológica sin considerar ningún dato biológico, lo que genera altas tasas de sobrediagnóstico. Para abordar el problema anterior, los investigadores clínicos están intentando extraer biomarcadores de TDAH a partir de señales electroencefalográficas (EEG) registradas. Entre los biomarcadores más comunes se encuentran la relación Theta / Beta y P300, de los cuales estudios recientes han demostrado una falta de importancia en las diferencias entre el TDAH y los sujetos de control. Además, otro gran desafío en el procesamiento del electroencefalograma viene dado por la sensibilidad de las señales, ya que pueden verse fácilmente afectadas por ruidos de fondo, artefactos musculares, movimientos de la cabeza y parpadeos que perjudican enormemente su calidad, lo que limita su introducción en aplicaciones del mundo real. Este trabajo propone una metodología de representación de señales de EEG para identificar discrepancias de respuestas inhibitorias en el sujeto, decodificar la estructura de datos y respaldar el diagnóstico de trastornos mentales. Para esto, primero desarrollamos un enfoque de extracción de características basado en los patrones espaciales comunes (CSP) de las señales de EEG para respaldar el diagnóstico de TDAH como se muestra en el capítulo 3. Luego, desarrollamos una metodología para la representación de señales de EEG que utiliza la similitud entre series de tiempo a través de sus matrices de covarianza en la variedad riemanniana de matrices semidefinitas positivas (PSD), utilizando la divergencia logdet de Jensen Bregman, el kernel de Stein y la alineación de kernel centrada (CKA) como una función de costo para realizar una optimización de filtros espaciales. Finalmente, en el capítulo 5 presentamos una metodología para el apoyo diagnóstico del TDAH. La propuesta implica el uso de los patrones espaciales óptimos desarrollados en el capítulo 4, una descomposición en los ritmos cerebrales y la decodificación discriminativa del capítulo 3. Las características subjetivas resultantes alimentaron un análisis discriminante lineal como herramienta de diagnóstico. La tasa de precisión alcanzada del 93% demuestra que el índice discriminativo basado en los patrones espaciales de stein supera a los biomarcadores convencionales en el diagnóstico de TDAH.Attention-Deficit/Hyperactivity Disorder (ADHD) is a childhood-onset neurological disorder that can persist in adolescence and adult life, reducing concentration, memory, and productivity. The main drawback with mental health abnormalities of this type is the traditional diagnostic technique. Since this is based exclusively on a symptomatological description without considering any biological data, leading to high overdiagnosis rates. To address the above problem, clinical researchers are attempting to extract ADHD biomarkers from recorded electroencephalographic (EEG) signals. Among the most common biomarkers are Theta/Beta Ratio and P300, of which recent studies have shown a lack of significance on the differences between ADHD and control subjects. Besides, another great challenge in EEG processing is given by the sensitivity of the signals, since they can be easily affected by background noise, muscle artifacts, head movements and flickering that greatly impair their quality, which limits its introduction into real world applications. This work proposes an EEG signal representation methodology for identifying subject-wise discrepancies of inhibitory responses, decoding the data structure, and supporting diagnosis of mental disorders. For this, first we develop a feature extraction approach based on the common spatial patterns (CSP) from EEG signals to support the ADHD diagnosis as show in chapter 3. Then, we develop a methodology for the representation of EEG signals that uses the similarity between time series through their covariance matrices in the Riemannian manifold of positive semidefinite matrices (PSD), using the logdet-divergence of Jensen Bregman, the Stein kernel, and Centered Kernel Alignment (CKA) as a cost function to perform a spatial filters optimization. Finally, in chapter 5 we present a methodology for the diagnostic support of ADHD. The proposal involves the use of the optimal spatial patterns developed in chapter 4, a decomposition in brain rhythms, and the discriminative decoding of chapter 3. The resulting subject-wise features fed a linear discriminant analysis as the supported-diagnosis tool. Achieved 93% accuracy rate proves that the discriminative index based on the stein spatial patterns outperforms conventional biomarkers in the ADHD diagnosis.MaestríaMagíster en Ingeniería EléctricaContents 1 List of Symbols and Abbreviations 6 1.1 Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.2 Abbrevations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2 Introduction 8 2.1 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.2 Justification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.3 State of the art . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.4 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.4.1 General objective . . . . . . . . . . . . . . . . . . . . . . . . 12 2.4.2 Specific objectives . . . . . . . . . . . . . . . . . . . . . . . 12 3 CSP-based discriminative capacity index from EEG 13 3.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.1.1 Common Spatial Patterns . . . . . . . . . . . . . . . . . . . . 13 3.1.2 Discriminative decoding of CSP . . . . . . . . . . . . . . . . 14 3.2 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.2.1 Synthetic EEG records . . . . . . . . . . . . . . . . . . . . . 15 3.2.2 Real EEG records . . . . . . . . . . . . . . . . . . . . . . . . 16 3.2.3 Proposed scheme for feature extraction . . . . . . . . . . . . 19 3.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.3.1 Discriminative decoding on simulated data . . . . . . . . . . 19 3.3.2 Feature extraction by discriminative decoding . . . . . . . . . 21 3.3.3 Diagnostic support of ADHD . . . . . . . . . . . . . . . . . 21 4 Multiple Kernel Stein Spatial Patterns 24 4.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 4.1.1 EEG Decomposition . . . . . . . . . . . . . . . . . . . . . . 24 4.1.2 Time-Series Similarity through the Stein Kernel for PSD Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 4.1.3 Spatial Filter Optimization Using Centered Kernel Alignment 27 4.1.4 Assembling of Multiple Kernel Representations . . . . . . . . 27 4.2 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 4.2.1 Dataset IIa from BCI Competition IV (BCICIV2a) . . . . . . 28 4.2.2 Proposed BCI Methodology . . . . . . . . . . . . . . . . . . 29 4.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.3.1 Performance Results . . . . . . . . . . . . . . . . . . . . . . 30 4.3.2 Model Interpretability . . . . . . . . . . . . . . . . . . . . . 33 5 SSP-based discriminative capacity index from EEG supporting ADHD di agnosis 37 5.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 5.1.1 Brain rhythms EEG decomposition . . . . . . . . . . . . . . 38 5.1.2 Stein Spatial Patterns (SSP) . . . . . . . . . . . . . . . . . . 39 5.1.3 Discriminative decoding of SSP . . . . . . . . . . . . . . . . 39 5.1.4 Generative-supervised feature relevance . . . . . . . . . . . . 40 5.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 6 Conclusions 45 6.1 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Brain signal analysis in space-time-frequency domain: an application to brain computer interfacing

In this dissertation, advanced methods for electroencephalogram (EEG) signal analysis in the space-time-frequency (STF) domain with applications to eye-blink (EB) artifact removal and brain computer interfacing (BCI) are developed. The two methods for EB artifact removal from EEGs are presented which respectively include the estimated spatial signatures of the EB artifacts into the signal extraction and the robust beamforming frameworks. In the developed signal extraction algorithm, the EB artifacts are extracted as uncorrelated signals from EEGs. The algorithm utilizes the spatial signatures of the EB artifacts as priori knowledge in the signal extraction stage. The spatial distributions are identified using the STF model of EEGs. In the robust beamforming approach, first a novel space-time-frequency/time-segment (STF-TS) model for EEGs is introduced. The estimated spatial signatures of the EBs are then taken into account in order to restore the artifact contaminated EEG measurements. Both algorithms are evaluated by using the simulated and real EEGs and shown to produce comparable results to that of conventional approaches. Finally, an effective paradigm for BCI is introduced. In this approach prior physiological knowledge of spectrally band limited steady-state movement related potentials is exploited. The results consolidate the method

Towards a home-use BCI: fast asynchronous control and robust non-control state detection

Eine Hirn-Computer Schnittstelle (engl. Brain-Computer Interface, BCI) erlaubt einem Nutzer einen Computer nur mittels Gehirn-Aktivität zu steuern. Der Hauptanwendungszweck ist die Wiederherstellung verschiedener Funktionen von motorisch eingeschränkten Menschen, zum Beispiel, die Wiederherstellung der Kommunikationsfähigkeit. Bisherige BCIs die auf visuell evozierten Potentialen (VEPs) basieren, erlauben bereits hohe Kommunikationsgeschwindigkeiten. VEPs sind Reaktionen, die im Gehirn durch visuelle Stimulation hervorgerufen werden. Allerdings werden bisherige BCIs hauptsächlich in der Forschung verwendet und sind nicht für reale Anwendungszwecke geeignet. Grund dafür ist, dass sie auf dem synchronen Steuerungsprinzip beruhen, dies bedeutet, dass Aktionen nur in vorgegebenen Zeitslots ausgeführt werden können. Dies bedeutet wiederum, dass der Nutzer keine Aktionen nach seinem Belieben ausführen kann, was für reale Anwendungszwecke ein Problem darstellt. Um dieses Problem zu lösen, müssen BCIs die Intention des Nutzers, das System zu steuern oder nicht, erkennen. Solche BCIs werden asynchron oder selbstbestimmt genannt. Bisherige asynchrone BCIs zeigen allerdings keine ausreichende Genauigkeit bei der Erkennung der Intention und haben zudem eine deutlich reduzierte Kommunikationsgeschwindigkeit im Vergleich zu synchronen BCIs. In dieser Doktorarbeit wird das erste asynchrone BCI vorgestellt, welches sowohl eine annäherungsweise perfekte Erkennung der Intention des Nutzers als auch eine ähnliche Kommunikationsgeschwindigkeit wie synchrone BCIs erzielt. Dies wurde durch die Entwicklung eines allgemeinen Modells für die Vorhersage von sensorischen Reizen erzielt. Dadurch können beliebige visuelle Stimulationsmuster basierend auf den gemessenen VEPs vorhergesagt werden. Das Modell wurde sowohl mit einem "traditionellen" maschinellen Lernverfahren als auch mit einer deep-learning Methode implementiert und evaluiert. Das resultierende asynchrone BCI übertrifft bisherige Methoden in mehreren Disziplinen um ein Vielfaches und ist ein wesentlicher Schritt, um BCI-Anwendungen aus dem Labor in die Praxis zu bringen. Durch weitere Optimierungen, die in dieser Arbeit diskutiert werden, könnte es sich zum allerersten geeigneten BCI für Endanwender entwickeln, da es effektiv (hohe Genauigkeit), effizient (schnelle Klassifizierungen), und einfach zu bedienen ist. Ein weiteres Alleinstellungsmerkmal ist, dass das entwickelte BCI für beliebige Szenarien verwendet werden kann, da es annähernd unendlich viele gleichzeitige Aktionsfelder erlaubt.Brain-Computer Interfaces (BCIs) enable users to control a computer by using pure brain activity. Their main purpose is to restore several functionalities of motor disabled people, for example, to restore the communication ability. Recent BCIs based on visual evoked potentials (VEPs), which are brain responses to visual stimuli, have shown to achieve high-speed communication. However, BCIs have not really found their way out of the lab yet. This is mainly because all recent high-speed BCIs are based on synchronous control, which means commands can only be executed in time slots controlled by the BCI. Therefore, the user is not able to select a command at his own convenience, which poses a problem in real-world applications. Furthermore, all those BCIs are based on stimulation paradigms which restrict the number of possible commands. To be suitable for real-world applications, a BCI should be asynchronous, or also called self-paced, and must be able to identify the user’s intent to control the system or not. Although there some asynchronous BCI approaches, none of them achieved suitable real-world performances. In this thesis, the first asynchronous high-speed BCI is proposed, which allows using a virtually unlimited number of commands. Furthermore, it achieved a nearly perfect distinction between intentional control (IC) and non-control (NC), which means commands are only executed if the user intends to. This was achieved by a completely different approach, compared to recent methods. Instead of using a classifier trained on specific stimulation patterns, the presented approach is based on a general model that predicts arbitrary stimulation patterns. The approach was evaluated with a "traditional" as well as a deep machine learning method. The resultant asynchronous BCI outperforms recent methods by a multi-fold in multiple disciplines and is an essential step for moving BCI applications out of the lab and into real life. With further optimization, discussed in this thesis, it could evolve to the very first end-user suitable BCI, as it is effective (high accuracy), efficient (fast classifications), ease of use, and allows to perform as many different tasks as desired

Investigation and Quantification of FES Exercise – Isometric Electromechanics and Perceptions of Its Usage as an Exercise Modality for Various Populations

Functional Electrical Stimulation (FES) is the triggering of muscle contraction by use of an electrical current. It can be used to give paralyzed individuals several health benefits, through allowing artificial movement and exercise. Although many FES devices exist, many aspects require innovation to increase usability and home translation. In addition, the effect of changing electrical parameters on limb biomechanics is not entirely understood; in particular with regards to stimulation duty cycle. This thesis has two distinct components. In the first (public health component), interview studies were conducted to understand several issues related to FES technology enhancement, implementation and home translation. In the second (computational biomechanics component), novel signal processing algorithms were designed that can be used to measure mechanical responses of muscles subjected to electrical stimulation. These experiments were performed by changing duty cycle and measuring its effect on quadriceps-generated knee torque. The studies of this thesis have presented several ideas, toolkits and results which have the potential to guide future FES biomechanics studies and the translatability of systems into regular usage for patients. The public health studies have provided conceptual frameworks upon which FES may be used in the home by patients. In addition, they have elucidated a range of issues that need to be addressed should FES technology reach its true potential as a therapy. The computational biomechanics studies have put forward novel data analysis techniques which may be used for understanding how muscle responds to electrical stimulation, as measured via torque. Furthermore, the effect of changing the electrical stimulation duty cycle on torque was successfully described, adding to an understanding of how electrical stimulation parameter modulation can influence joint biomechanics