Evaluation of a congruent auditory feedback for Motor Imagery BCI

International audienceDesigning a feedback that helps participants to achieve higher performances is an important concern in brain-computer interface (BCI) research. In a pilot study, we demonstrate how a congruent auditory feedback could improve classification in a electroencephalography (EEG) motor imagery BCI. This is a promising result for creating alternate feedback modality

Brain-Computer Interface and Motor Imagery Training: The Role of Visual Feedback and Embodiment

Controlling a brain-computer interface (BCI) is a difficult task that requires extensive training. Particularly in the case of motor imagery BCIs, users may need several training sessions before they learn how to generate desired brain activity and reach an acceptable performance. A typical training protocol for such BCIs includes execution of a motor imagery task by the user, followed by presentation of an extending bar or a moving object on a computer screen. In this chapter, we discuss the importance of a visual feedback that resembles human actions, the effect of human factors such as confidence and motivation, and the role of embodiment in the learning process of a motor imagery task. Our results from a series of experiments in which users BCI-operated a humanlike android robot confirm that realistic visual feedback can induce a sense of embodiment, which promotes a significant learning of the motor imagery task in a short amount of time. We review the impact of humanlike visual feedback in optimized modulation of brain activity by the BCI users

Standardization of Protocol Design for User Training in EEG-based Brain-Computer Interface

International audienceBrain-computer interfaces (BCIs) are systems that enable a personto interact with a machine using only neural activity. Such interaction canbe non-intuitive for the user hence training methods are developed to increaseone’s understanding, confidence and motivation, which would in parallel increasesystem performance. To clearly address the current issues in the BCI usertraining protocol design, here it is divided intointroductoryperiod and BCIinteractionperiod. First, theintroductoryperiod (before BCI interaction) mustbe considered as equally important as the BCI interaction for user training. Tosupport this claim, a review of papers show that BCI performance can dependon the methodologies presented in such introductory period. To standardize itsdesign, the literature from human-computer interaction (HCI) is adjusted to theBCI context. Second, during the user-BCI interaction, the interface can takea large spectrum of forms (2D, 3D, size, color etc.) and modalities (visual,auditory or haptic etc.) without following any design standard or guidelines.Namely, studies that explore perceptual affordance on neural activity show thatmotor neurons can be triggered from a simple observation of certain objects, anddepending on objects’ properties (size, location etc.) neural reactions can varygreatly. Surprisingly, the effects of perceptual affordance were not investigatedin the BCI context. Both inconsistent introductions to BCI as well as variableinterface designs make it difficult to reproduce experiments, predict their outcomesand compare results between them. To address these issues, a protocol designstandardization for user training is proposed

Brain Music : Sistema generativo para la creación de música simbólica a partir de respuestas neuronales afectivas

gráficas, tablasEsta tesis de maestría presenta una metodología de aprendizaje profundo multimodal innovadora que fusiona un modelo de clasificación de emociones con un generador musical, con el propósito de crear música a partir de señales de electroencefalografía, profundizando así en la interconexión entre emociones y música. Los resultados alcanzan tres objetivos específicos: Primero, ya que el rendimiento de los sistemas interfaz cerebro-computadora varía considerablemente entre diferentes sujetos, se introduce un enfoque basado en la transferencia de conocimiento entre sujetos para mejorar el rendimiento de individuos con dificultades en sistemas de interfaz cerebro-computadora basados en el paradigma de imaginación motora. Este enfoque combina datos de EEG etiquetados con datos estructurados, como cuestionarios psicológicos, mediante un método de "Kernel Matching CKA". Utilizamos una red neuronal profunda (Deep&Wide) para la clasificación de la imaginación motora. Los resultados destacan su potencial para mejorar las habilidades motoras en interfaces cerebro-computadora. Segundo, proponemos una técnica innovadora llamada "Labeled Correlation Alignment"(LCA) para sonificar respuestas neurales a estímulos representados en datos no estructurados, como música afectiva. Esto genera características musicales basadas en la actividad cerebral inducida por las emociones. LCA aborda la variabilidad entre sujetos y dentro de sujetos mediante el análisis de correlación, lo que permite la creación de envolventes acústicos y la distinción entre diferente información sonora. Esto convierte a LCA en una herramienta prometedora para interpretar la actividad neuronal y su reacción a estímulos auditivos. Finalmente, en otro capítulo, desarrollamos una metodología de aprendizaje profundo de extremo a extremo para generar contenido musical MIDI (datos simbólicos) a partir de señales de actividad cerebral inducidas por música con etiquetas afectivas. Esta metodología abarca el preprocesamiento de datos, el entrenamiento de modelos de extracción de características y un proceso de emparejamiento de características mediante Deep Centered Kernel Alignment, lo que permite la generación de música a partir de señales EEG. En conjunto, estos logros representan avances significativos en la comprensión de la relación entre emociones y música, así como en la aplicación de la inteligencia artificial en la generación musical a partir de señales cerebrales. Ofrecen nuevas perspectivas y herramientas para la creación musical y la investigación en neurociencia emocional. Para llevar a cabo nuestros experimentos, utilizamos bases de datos públicas como GigaScience, Affective Music Listening y Deap Dataset (Texto tomado de la fuente)This master’s thesis presents an innovative multimodal deep learning methodology that combines an emotion classification model with a music generator, aimed at creating music from electroencephalography (EEG) signals, thus delving into the interplay between emotions and music. The results achieve three specific objectives: First, since the performance of brain-computer interface systems varies significantly among different subjects, an approach based on knowledge transfer among subjects is introduced to enhance the performance of individuals facing challenges in motor imagery-based brain-computer interface systems. This approach combines labeled EEG data with structured information, such as psychological questionnaires, through a "Kernel Matching CKA"method. We employ a deep neural network (Deep&Wide) for motor imagery classification. The results underscore its potential to enhance motor skills in brain-computer interfaces. Second, we propose an innovative technique called "Labeled Correlation Alignment"(LCA) to sonify neural responses to stimuli represented in unstructured data, such as affective music. This generates musical features based on emotion-induced brain activity. LCA addresses variability among subjects and within subjects through correlation analysis, enabling the creation of acoustic envelopes and the distinction of different sound information. This makes LCA a promising tool for interpreting neural activity and its response to auditory stimuli. Finally, in another chapter, we develop an end-to-end deep learning methodology for generating MIDI music content (symbolic data) from EEG signals induced by affectively labeled music. This methodology encompasses data preprocessing, feature extraction model training, and a feature matching process using Deep Centered Kernel Alignment, enabling music generation from EEG signals. Together, these achievements represent significant advances in understanding the relationship between emotions and music, as well as in the application of artificial intelligence in musical generation from brain signals. They offer new perspectives and tools for musical creation and research in emotional neuroscience. To conduct our experiments, we utilized public databases such as GigaScience, Affective Music Listening and Deap DatasetMaestríaMagíster en Ingeniería - Automatización IndustrialInvestigación en Aprendizaje Profundo y señales BiológicasEléctrica, Electrónica, Automatización Y Telecomunicaciones.Sede Manizale

P300-Based BCI Performance Prediction through Examination of Paradigm Manipulations and Principal Components Analysis.

Severe neuromuscular disorders can produce locked-in syndrome (LIS), a loss of nearly all voluntary muscle control. A brain-computer interface (BCI) using the P300 event-related potential provides communication that does not depend on neuromuscular activity and can be useful for those with LIS. Currently, there is no way of determining the effectiveness of P300-based BCIs without testing a person\u27s performance multiple times. Additionally, P300 responses in BCI tasks may not resemble the typical P300 response. I sought to clarify the relationship between the P300 response and BCI task parameters and examine the possibility of a predictive relationship between traditional oddball tasks and BCI performance. Both waveform and component analysis have revealed several task-dependent aspects of brain activity that show significant correlation with the user\u27s performance. These components may provide a fast and reliable metric to indicate whether the BCI system will work for a given individual

EEG-based Brain-Computer Interfaces for people with Disorders of Consciousness: Features and applications. A systematic review

Background: Disorders of Consciousness (DoC) are clinical conditions following a severe acquired brain injury (ABI) characterized by absent or reduced awareness, known as coma, Vegetative State (VS)/Unresponsive Wakefulness Syndrome (VS/UWS), and Minimally Conscious State (MCS). Misdiagnosis rate between VS/UWS and MCS is attested around 40% due to the clinical and behavioral fluctuations of the patients during bedside consciousness assessments. Given the large body of evidence that some patients with DoC possess "covert" awareness, revealed by neuroimaging and neurophysiological techniques, they are candidates for intervention with brain-computer interfaces (BCIs). Objectives: The aims of the present work are (i) to describe the characteristics of BCI systems based on electroencephalography (EEG) performed on DoC patients, in terms of control signals adopted to control the system, characteristics of the paradigm implemented, classification algorithms and applications (ii) to evaluate the performance of DoC patients with BCI. Methods: The search was conducted on Pubmed, Web of Science, Scopus and Google Scholar. The PRISMA guidelines were followed in order to collect papers published in english, testing a BCI and including at least one DoC patient. Results: Among the 527 papers identified with the first run of the search, 27 papers were included in the systematic review. Characteristics of the sample of participants, behavioral assessment, control signals employed to control the BCI, the classification algorithms, the characteristics of the paradigm, the applications and performance of BCI were the data extracted from the study. Control signals employed to operate the BCI were: P300 (N = 19), P300 and Steady-State Visual Evoked Potentials (SSVEP; hybrid system, N = 4), sensorimotor rhythms (SMRs; N = 5) and brain rhythms elicited by an emotional task (N = 1), while assessment, communication, prognosis, and rehabilitation were the possible applications of BCI in DoC patients. Conclusion: Despite the BCI is a promising tool in the management of DoC patients, supporting diagnosis and prognosis evaluation, results are still preliminary, and no definitive conclusions may be drawn; even though neurophysiological methods, such as BCI, are more sensitive to covert cognition, it is suggested to adopt a multimodal approach and a repeated assessment strategy