Cortical encoding and decoding models of speech production

To speak is to dynamically orchestrate the movements of the articulators (jaw, tongue, lips, and larynx), which in turn generate speech sounds. It is an amazing mental and motor feat that is controlled by the brain and is fundamental for communication. Technology that could translate brain signals into speech would be transformative for people who are unable to communicate as a result of neurological impairments. This work first investigates how articulator movements that underlie natural speech production are represented in the brain. Building upon this, this work also presents a neural decoder that can synthesize audible speech from brain signals. Data to support these results were from direct cortical recordings of the human sensorimotor cortex while participants spoke natural sentences. Neural activity at individual electrodes encoded a diversity of articulatory kinematic trajectories (AKTs), each revealing coordinated articulator movements towards specific vocal tract shapes. The neural decoder was designed to leverage the kinematic trajectories encoded in the sensorimotor cortex which enhanced performance even with limited data. In closed vocabulary tests, listeners could readily identify and transcribe speech synthesized from cortical activity. These findings advance the clinical viability of using speech neuroprosthetic technology to restore spoken communication

Graph neural networks for electroencephalogram analysis

El objetivo de este trabajo es proporcionar un modelo capaz de identificar la enfermedad de Alzheimer y el Deterioro Cognitivo Leve (DCL) en registros de electroencefalogramas (EEG). A pesar de que los EEGs son una de las pruebas más pruebas utilizadas para los trastornos neurológicos, hoy en día el diagnóstico de estas enfermedades se basa en el comportamiento del paciente. comportamiento del paciente. Esto se debe a que la precisión de los expertos en el reconocimiento visual de los EEGs se estima en torno al 50%. Para resolver las dificultades de la tarea mencionada, esta tesis propone un modelo de Red Neural Gráfica (GNN) para clasificar a los sujetos utilizando únicamente las señales registradas. Para desarrollar el modelo final, primero propusimos varios procedimientos para construir gráficos a partir de las señales de EEGs, explorando diferentes formas de representar la conectividad entre canales, así como métodos para la extracción de de las características relevantes. Por el momento, no hay modelos GNN propuestos para la detección de Alzheimer o DCL. Por lo tanto, utilizamos arquitecturas empleadas en tareas similares y las modificamos para nuestro dominio específico. Por último, se evalúa un conjunto de combinaciones coherentes de grafos y modelos GNN bajo el mismo conjunto de métricas. Además, para las combinaciones con mejor rendimiento, se realiza un estudio del impacto de varios hiperparámetros se lleva a cabo. Con el fin de manejar todos los experimentos posibles, hemos desarrollado un marco de software para construir fácilmente construir los diferentes tipos de gráficos, crear los modelos y evaluar su rendimiento. La mejor combinación de construcción de grafos y diseño de modelos, basada en capas convolucionales de atención a los grafos, conduce a un 92,31% de precisión en la clasificación binaria de sujetos sanos y enfermos de Alzheimer y a un 87,59% de precisión cuando se evalúan también las grabaciones de pacientes con Deterioro Cognitivo Leve, que son comparables a los resultados del estado del arte. resultados del estado del arte. Aunque este trabajo se realiza en un campo novedoso y existen muchas posibilidades aún posibilidades aún por explorar, concluimos que las GNNs muestran capacidades sobrehumanas para la detección de Alzheimer y DCL utilizando EEGs.The aim of this work is to provide a model able to identify Alzheimer's disease and Mild Cognitive Impairment (MCI) in electroencephalogram's (EEGs) recordings. Despite EEGs being one of the most common tests used for neurological disorders, nowadays the diagnose of these diseases is based on the patient's behaviour. This is because expert's accuracy on EEGs visual recognition is estimated to be around 50%. To solve the difficulties of the aforementioned task, this thesis proposes a Graph Neural Network (GNN) model to classify the subjects using only the recorded signals. To develop the final model, first we proposed several procedures to build graphs from the EEGs signals, exploring different ways of representing the inter-channel connectivity as well as methods for relevant features extraction. For the time being, there are not GNN models proposed for Alzheimer or MCI detection. Hence, we used architectures employed by similar tasks and modified them for our specific domain. Finally, a set of coherent combinations of graph and GNN model is evaluated under the same set of metrics. Moreover, for the best performing combinations, a study of the impact of several hyperparameters is carried out. In order to handle all the possible experiments, we developed a software framework to easily build the different types of graphs, create the models and evaluate their performance. The best combination of graph building and model design, based on graph attention convolutional layers, leads to a 92.31% of accuracy in the binary classification of healthy subjects and Alzheimer's patients and to a 87.59% of accuracy when also evaluating MCI patients recordings, these are comparable to state of the art results. Although this work is done within a novel field and there exist many possibilities yet to be explored, we conclude that GNNs show super-human capabilities for Alzheimer and MCI detection using EEGs

Reading Your Own Mind: Dynamic Visualization of Real-Time Neural Signals

Brain Computer Interfaces: BCI) systems which allow humans to control external devices directly from brain activity, are becoming increasingly popular due to dramatic advances in the ability to both capture and interpret brain signals. Further advancing BCI systems is a compelling goal both because of the neurophysiology insights gained from deriving a control signal from brain activity and because of the potential for direct brain control of external devices in applications such as brain injury recovery, human prosthetics, and robotics. The dynamic and adaptive nature of the brain makes it difficult to create classifiers or control systems that will remain effective over time. However it is precisely these qualities that offer the potential to use feedback to build on simple features and create complex control features that are robust over time. This dissertation presents work that addresses these opportunities for the specific case of Electrocorticography: ECoG) recordings from clinical epilepsy patients. First, queued patient tasks were used to explore the predictive nature of both local and global features of the ECoG signal. Second, an algorithm was developed and tested for estimating the most informative features from naive observations of ECoG signal. Third, a software system was built and tested that facilitates real-time visualizations of ECoG signal patients and allows ECoG epilepsy patients to engage in an interactive BCI control feature screening process