Rhythmic activities of the brain: quantifying the high complexity of beta and gamma oscillations during visuomotor tasks

Electroencephalography (EEG) signals depict the electrical activity that take place at the surface of the brain, and provide an important tool for understanding a variety of cognitive processes. The EEG are the product of synchronized activity of the brain and variations in EEG oscillations patterns reflect the underlying changes in neuronal synchrony. Our aim is to characterize the complexity of the EEG rhythmic oscillations bands when the subjects performs a visuomotor or imagined cognitive tasks (imagined movement), providing a causal mapping of the dynamical rhythmic activities of the brain as a measure of attentional investment. We estimate the intrinsic correlational structure of the signals within the causality entropy-complexity plane H x C, where the enhanced complexity in the gamma 1, gamma 2 and beta 1 bands allow us to distinguish motor-visual memory tasks from control conditions. We identify the dynamics of the gamma 1, gamma 2 and beta 1 rhythmic oscillations within the zone of a chaotic dissipative behavior, while in contrast the beta 2 band shows a much higher level of entropy and a significant low level of complexity that corresponds to a non-invertible cubic map. Our findings enhance the importance of the gamma band during attention in perceptual feature binding during the visuomotor/imagery tasks.Instituto de Física de Líquidos y Sistemas Biológico

Rhythmic activities of the brain: quantifying the high complexity of beta and gamma oscillations during visuomotor tasks

Electroencephalography (EEG) signals depict the electrical activity that take place at the surface of the brain, and provide an important tool for understanding a variety of cognitive processes. The EEG are the product of synchronized activity of the brain and variations in EEG oscillations patterns reflect the underlying changes in neuronal synchrony. Our aim is to characterize the complexity of the EEG rhythmic oscillations bands when the subjects performs a visuomotor or imagined cognitive tasks (imagined movement), providing a causal mapping of the dynamical rhythmic activities of the brain as a measure of attentional investment. We estimate the intrinsic correlational structure of the signals within the causality entropy-complexity plane H x C, where the enhanced complexity in the gamma 1, gamma 2 and beta 1 bands allow us to distinguish motor-visual memory tasks from control conditions. We identify the dynamics of the gamma 1, gamma 2 and beta 1 rhythmic oscillations within the zone of a chaotic dissipative behavior, while in contrast the beta 2 band shows a much higher level of entropy and a significant low level of complexity that corresponds to a non-invertible cubic map. Our findings enhance the importance of the gamma band during attention in perceptual feature binding during the visuomotor/imagery tasks.Fil: Baravalle, Román. Universidad Nacional de La Plata. Facultad de Ciencias Exactas. Departamento de Física; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - La Plata. Instituto de Física de Líquidos y Sistemas Biológicos. Universidad Nacional de La Plata. Facultad de Ciencias Exactas. Instituto de Física de Líquidos y Sistemas Biológicos; ArgentinaFil: Rosso, Osvaldo Aníbal. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Bahía Blanca. Centro de Recursos Naturales Renovables de la Zona Semiárida. Universidad Nacional del Sur. Centro de Recursos Naturales Renovables de la Zona Semiárida; ArgentinaFil: Montani, Fernando Fabián. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - La Plata. Instituto de Física de Líquidos y Sistemas Biológicos. Universidad Nacional de La Plata. Facultad de Ciencias Exactas. Instituto de Física de Líquidos y Sistemas Biológicos; Argentin

Enhancement and optimization of a multi-command-based brain-computer interface

Brain-computer interfaces (BCI) assist disabled person to control many appliances without any physically interaction (e.g., pressing a button). SSVEP is brain activities elicited by evoked signals that are observed by visual stimuli paradigm. In this dissertation were addressed the problems which are oblige more usability of BCI-system by optimizing and enhancing the performance using particular design. Main contribution of this work is improving brain reaction response depending on focal approaches

Coherence analysis: Methods, solutions and problems

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.A coherence function is a measure of the correlation of two signals and may be used as a measure for functional relationship between brain areas. In studying functional relationships, referenced EEG (REEG) coherence analysis yields important new aspects of brain activities, which complement the data obtained by power spectral analysis. However, REEG-based coherence tends to show a false high value due to volume conduction from un correlated sources (VCUS). Existing signal processing methods address this issue using a Fourier coherence function of scalp Laplacian. Although this method has been proved useful to reveal correlation between EEG signals with minimum VCUS effects, it only provides frequency-domain analysis. Since EEG signals are highly non-stationary, it is more appropriate to use time-frequency methods for coherence analysis of scalp Laplacian. Thus this research applies the wavelet transform on coherence analysis of scalp Laplacian. To verify our technique, already recorded EEG data of event related potentials were obtained from a study of two large groups of alcoholic and abstinent alcoholic subjects, performing visual picture-recognition tasks. The proposed coherence method successfully detected time-frequency correlation between EEG signals with minimum VCUS effects. It showed significant spatial specificity and revealed detailed coherence patterns. Some new important results regarding time-frequency characteristics of VCUS effects on wavelet and short-time Fourier transform (STFT) coherence analysis of REEG signals were deduced. The proposed coherence method was also compared to a conventional wavelet coherence method of REEG signals in the study of coherence difference between coherences of alcoholic and abstinent alcoholic EEG signals. Results of this study provided substantial evidence that VCUS effects are not additive and therefore can not be ignored in comparison of different brain states between groups of subjects

Predicting Humans’ Identity and Mental Load from EEG: Performed by AI

EEG-based brain machine/computer interfaces (BMIs/BCIs) have a wide range of clinical and non-clinical applications. Mental workload (MW) classification, emotion recognition, motor imagery, seizure detection, and sleep stage scoring are among the active BCI research areas. One of the relatively new BCI area is EEG-based human subject recognition (i.e., EEG biometric). There still exist several challenges that need to be addressed to design a successful EEG-based biometric model applicable for real-world environments. First, there is a need for a protocol that can elicit the individual dependent EEG responses in a short period of time. A classification algorithm with high generalization power is also required to deal with the EEG signals classification task. The latter is a common challenge for all EEG-based BCI paradigms; given the non-stationary nature of the EEG signals and the small size of the EEG datasets. In addition, to building a stable EEG biometric model, the effects of human mental states (e.g., emotion, mental load) on the model performance needs to be carefully examined. In this thesis, a new protocol for the area of the EEG biometric has been proposed. The proposed protocol called “(the) N-back task” is based on the human working memory and the experimental results obtained in this thesis prove that the EEG signals elicited by the N-back task contain subject specific features, even for very short time intervals. It has also been shown that three load levels of the typical N-back task are all capable of evoking subject specific EEG features. As a result, the N-back task can be used as a protocol having more than one mode (i.e, cancelable protocol) that comes with added security benefits. The EEG signals evoked by the N-back task have been used to train a compact convolutional neural network called the EEGNet. A configuration of the EEGNet having 16 temporal and 2 spatial filters has reached an identification accuracy of approximately 97% using data instances as short as 1.1s for a pool of 26 subjects. To further improve the accuracy, a novel ensemble classifier has been designed in this thesis. The principle underlying the proposed ensemble is the “division and exclusion” of the EEG channels guided by scalp locations. The ensemble classifier has (statistically significantly) improved the subject recognition rate from 97% to 99%. Performance of the proposed ensemble model has also been assessed in the EEG-based MW classification paradigm. The ensemble classifier outperformed the single EEGNet as well as a state-of-the-art classifier called WLnet in the challenging scenario of the subject-independent (cross-subject) MW classification. The results suggest that the ensemble structure proposed in this thesis can generalize to different BCI paradigms. Finally, effects of the mental workload on the performance of the EEG-based subject authentication models have been thoroughly explored in this thesis. The obtained results affirm that MW of the genuine and impostor subjects at the train and test phases have significant effects on both false negative rate (FNR) and false positive rate (FPR) of an authentication system. Different subjects have also shown different clusters of authentication behaviors when affected by the MW changes. This finding establishes the importance of the human’s mental load in the design of real-world EEG authentication systems and introduces a new investigation line for the EEG biometric community

Leveraging EEG-based speech imagery brain-computer interfaces

Speech Imagery Brain-Computer Interfaces (BCIs) provide an intuitive and flexible way of interaction via brain activity recorded during imagined speech. Imagined speech can be decoded in form of syllables or words and captured even with non-invasive measurement methods as for example the Electroencephalography (EEG). Over the last decade, research in this field has made tremendous progress and prototypical implementations of EEG-based Speech Imagery BCIs are numerous. However, most work is still conducted in controlled laboratory environments with offline classification and does not find its way to real online scenarios. Within this thesis we identify three main reasons for these circumstances, namely, the mentally and physically exhausting training procedures, insufficient classification accuracies and cumbersome EEG setups with usually high-resolution headsets. We furthermore elaborate on possible solutions to overcome the aforementioned problems and present and evaluate new methods in each of the domains. In detail we introduce two new training concepts for imagined speech BCIs, one based on EEG activity during silently reading and the other recorded during overtly speaking certain words. Insufficient classification accuracies are addressed by introducing the concept of a Semantic Speech Imagery BCI, which classifies the semantic category of an imagined word prior to the word itself to increase the performance of the system. Finally, we investigate on different techniques for electrode reduction in Speech Imagery BCIs and aim at finding a suitable subset of electrodes for EEG-based imagined speech detection, therefore facilitating the cumbersome setups. All of our presented results together with general remarks on experiences and best practice for study setups concerning imagined speech are summarized and supposed to act as guidelines for further research in the field, thereby leveraging Speech Imagery BCIs towards real-world application.Speech Imagery Brain-Computer Interfaces (BCIs) bieten eine intuitive und flexible Möglichkeit der Interaktion mittels Gehirnaktivität, aufgezeichnet während der bloßen Vorstellung von Sprache. Vorgestellte Sprache kann in Form von Silben oder Wörtern auch mit nicht-invasiven Messmethoden wie der Elektroenzephalographie (EEG) gemessen und entschlüsselt werden. In den letzten zehn Jahren hat die Forschung auf diesem Gebiet enorme Fortschritte gemacht, und es gibt zahlreiche prototypische Implementierungen von EEG-basierten Speech Imagery BCIs. Die meisten Arbeiten werden jedoch immer noch in kontrollierten Laborumgebungen mit Offline-Klassifizierung durchgeführt und finden nicht denWeg in reale Online-Szenarien. In dieser Arbeit identifizieren wir drei Hauptgründe für diesen Umstand, nämlich die geistig und körperlich anstrengenden Trainingsverfahren, unzureichende Klassifizierungsgenauigkeiten und umständliche EEG-Setups mit meist hochauflösenden Headsets. Darüber hinaus erarbeiten wir mögliche Lösungen zur Überwindung der oben genannten Probleme und präsentieren und evaluieren neue Methoden für jeden dieser Bereiche. Im Einzelnen stellen wir zwei neue Trainingskonzepte für Speech Imagery BCIs vor, von denen eines auf der Messung von EEG-Aktivität während des stillen Lesens und das andere auf der Aktivität während des Aussprechens bestimmter Wörter basiert. Unzureichende Klassifizierungsgenauigkeiten werden durch die Einführung des Konzepts eines Semantic Speech Imagery BCI angegangen, das die semantische Kategorie eines vorgestellten Wortes vor dem Wort selbst klassifiziert, um die Performance des Systems zu erhöhen. Schließlich untersuchen wir verschiedene Techniken zur Elektrodenreduktion bei Speech Imagery BCIs und zielen darauf ab, eine geeignete Teilmenge von Elektroden für die EEG-basierte Erkennung von vorgestellter Sprache zu finden, um so die umständlichen Setups zu erleichtern. Alle unsere Ergebnisse werden zusammen mit allgemeinen Bemerkungen zu Erfahrungen und Best Practices für Studien-Setups bezüglich vorgestellter Sprache zusammengefasst und sollen als Richtlinien für die weitere Forschung auf diesem Gebiet dienen, um so Speech Imagery BCIs für die Anwendung in der realenWelt zu optimieren

Brain signal recognition using deep learning

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel UniversityBrain Computer Interface (BCI) has the potential to offer a new generation of applications independent of muscular activity and controlled by the human brain. Brain imaging technologies are used to transfer the cognitive tasks into control commands for a BCI system. The electroencephalography (EEG) technology serves as the best available non-invasive solution for extracting signals from the brain. On the other hand, speech is the primary means of communication, but for patients suffering from locked-in syndrome, there is no easy way to communicate. Therefore, an ideal communication system for locked-in patients is a thought-to-speech BCI system. This research aims to investigate methods for the recognition of imagined speech from EEG signals using deep learning techniques. In order to design an optimal imagined speech recognition BCI, variety of issues have been solved. These include 1) proposing new feature extraction and classification framework for recognition of imagined speech from EEG signals, 2) grammatical class recognition of imagined words from EEG signals, 3) discriminating different cognitive tasks associated with speech in the brain such as overt speech, covert speech, and visual imagery. In this work machine learning, deep learning methods were used to analyze EEG signals. For recognition of imagined speech from EEG signals, a new EEG database was collected while the participants mentally spoke (imagined speech) the presented words. Along with imagined speech, EEG data was recorded for visual imagery (imagining a scene or an image) and overt speech (verbal speech). Spectro-temporal and spatio-temporal domain features were investigated for the classification of imagined words from EEG signals. Further, a deep learning framework using the convolutional network and attention mechanism was implemented for learning features in the spatial, temporal, and spectral domains. The method achieved a recognition rate of 76.6% for three binary word pairs. These experiments show that deep learning algorithms are ideal for imagined speech recognition from EEG signals due to their ability to interpret features from non-linear and non-stationary signals. Grammatical classes of imagined words from EEG signals were also recognized using a multi-channel convolution network framework. This method was extended to a multi-level recognition system for multi-class classification of imagined words which achieved an accuracy of 52.9% for 10 words, which is much better in comparison to previous work. In order to investigate the difference between imagined speech with verbal speech and visual imagery from EEG signals, we used multivariate pattern analysis (MVPA). MVPA provided the time segments when the neural oscillation for the different cognitive tasks was linearly separable. Further, frequencies that result in most discrimination between the different cognitive tasks were also explored. A framework was proposed to discriminate two cognitive tasks based on the spatio-temporal patterns in EEG signals. The proposed method used the K-means clustering algorithm to find the best electrode combination and convolutional-attention network for feature extraction and classification. The proposed method achieved a high recognition rate of 82.9% and 77.7%. The results in this research suggest that a communication based BCI system can be designed using deep learning methods. Further, this work add knowledge to the existing work in the field of communication based BCI system

Applied Cognitive Sciences

Cognitive science is an interdisciplinary field in the study of the mind and intelligence. The term cognition refers to a variety of mental processes, including perception, problem solving, learning, decision making, language use, and emotional experience. The basis of the cognitive sciences is the contribution of philosophy and computing to the study of cognition. Computing is very important in the study of cognition because computer-aided research helps to develop mental processes, and computers are used to test scientific hypotheses about mental organization and functioning. This book provides a platform for reviewing these disciplines and presenting cognitive research as a separate discipline

Electrocortical underpinnings of error monitoring in health and pathology

It becomes clear from the literature described above (Chapter 1), that the error monitoring mechanisms play a fundamental role in signalling the need for cognitive control. Many studies already provided a consistent evidence on the existence of peculiar ways in which the brain signals this need through electrophysiological changes. However, the following set of empirical studies aims to gain further insight into these complex processes by measuring brain activity changes in situations that alter the way one experience errors. The second Chapter (Chapter 2) consists of a brief commentary that was made in response to an article on the brain activity to action errors. In this commentary we propose new possibilities to explore our topic of interest, by taking advantage of EEG and modern virtual reality facilities. The thesis includes three EEG-VR studies: one on the error-mechanism in healthy participants (Chapter 3) and two studies on error monitoring system in pathological populations (Chapter 4, 5), as main parts of the core of the thesis. As a collateral project, in the Appendix, there is an EEG study on action observation in elite players (Chapter 7). In the first study (Chapter 3), we investigated a very simple but fundamental question. As we saw in the introduction, error-related signatures are evoked when an error occurs. But it is not clear how much of this is due to the occurrence of a violation of the intended goal or simply to the observation of a rare – thus less predictable – event. To this aim, we used a paradigm developed in the former years in our laboratory (Pavone et al., 2016; Spinelli et al., 2017), characterized by a setup in immersive Virtual Reality (VR) and simultaneous EEG recording. Building on the previous findings, we designed an EEG-VR study in which we manipulated the probability of observing errors in actions. In another study (Chapter 4) we investigated how erroneous actions are experienced by people with brain damage and diagnosis of Apraxia. Apraxic patients are people with hemispheric lesions and defective awareness on a variety of aspects that cover perceptuo-motor, cognitive or emotional domains. This study was developed after the results obtained by Canzano and colleagues (2014) in a behavioral study in which apraxic patients were asked to imitate the actions executed by the experimenter and judge their correctness; results revealed that bucco-facial apraxic patients manifest a specific deficit in detecting their own gestural errors when they are explicitly asked to judge them. With the present study we wanted to investigate apraxic brain’ response to action errors, while they embody an avatar in first person perspective (EEG-VR setup). The third study (Chapter 5) investigates the integrity of the error-monitoring system in Parkinson’s Disease and the impact of the dopaminergic treatment in the brain response to errors. To this aim we used the proposed VR action-observation paradigm, in which Parkinson patients observed successful and unsuccessful reach-to-grasp actions in first person perspective while EEG activity was recorded; the same patients were tested while being under dopaminergic treatment and during a dopaminergic withdrawal state. In another chapter we provide a critical overview of the findings of this work (General Discussion, Chapter 6). In the last chapter, the Appendix (Chapter 7), there is a collateral project of another research line of the Laboratory, in which I have being involved. In this study we are investigating the cortical underpinning of elite players during observation of goal-directed actions, in their domain of expertise. We recorded the EEG activity of elite wheelchair basketball players while observing free-throws performed by paraplegic athletes. We expected their brain correlates to be different from novice players and to be able to easily discriminate whether a basketball shot would be successful or unsuccessful (project still ongoing)