Deep Learning on VR-Induced Attention

Some evidence suggests that virtual reality (VR) approaches may lead to a greater attentional focus than experiencing the same scenarios presented on computer monitors. The aim of this study is to differentiate attention levels captured during a perceptual discrimination task presented on two different viewing platforms, standard personal computer (PC) monitor and head-mounted-display (HMD)-VR, using a well-described electroencephalography (EEG)-based measure (parietal P3b latency) and deep learning-based measure (that is EEG features extracted by a compact convolutional neural network-EEGNet and visualized by a gradient-based relevance attribution method-DeepLIFT). Twenty healthy young adults participated in this perceptual discrimination task in which according to a spatial cue they were required to discriminate either a "Target" or "Distractor" stimuli on the screen of viewing platforms. Experimental results show that the EEGNet-based classification accuracies are highly correlated with the p values of statistical analysis of P3b. Also, the visualized EEG features are neurophysiologically interpretable. This study provides the first visualized deep learning-based EEG features captured during an HMD-VR-based attentional tas

A Lightweight Multi-Scale Convolutional Neural Network for P300 Decoding: Analysis of Training Strategies and Uncovering of Network Decision

Convolutional neural networks (CNNs), which automatically learn features from raw data to approximate functions, are being increasingly applied to the end-to-end analysis of electroencephalographic (EEG) signals, especially for decoding brain states in brain-computer interfaces (BCIs). Nevertheless, CNNs introduce a large number of trainable parameters, may require long training times, and lack in interpretability of learned features. The aim of this study is to propose a CNN design for P300 decoding with emphasis on its lightweight design while guaranteeing high performance, on the effects of different training strategies, and on the use of post-hoc techniques to explain network decisions. The proposed design, named MS-EEGNet, learned temporal features in two different timescales (i.e., multi-scale, MS) in an efficient and optimized (in terms of trainable parameters) way, and was validated on three P300 datasets. The CNN was trained using different strategies (within-participant and within-session, within-participant and cross-session, leave-one-subject-out, transfer learning) and was compared with several state-of-the-art (SOA) algorithms. Furthermore, variants of the baseline MS-EEGNet were analyzed to evaluate the impact of different hyper-parameters on performance. Lastly, saliency maps were used to derive representations of the relevant spatio-temporal features that drove CNN decisions. MS-EEGNet was the lightest CNN compared with the tested SOA CNNs, despite its multiple timescales, and significantly outperformed the SOA algorithms. Post-hoc hyper-parameter analysis confirmed the benefits of the innovative aspects of MS-EEGNet. Furthermore, MS-EEGNet did benefit from transfer learning, especially using a low number of training examples, suggesting that the proposed approach could be used in BCIs to accurately decode the P300 event while reducing calibration times. Representations derived from the saliency maps matched the P300 spatio-temporal distribution, further validating the proposed decoding approach. This study, by specifically addressing the aspects of lightweight design, transfer learning, and interpretability, can contribute to advance the development of deep learning algorithms for P300-based BCIs

BCI applications based on artificial intelligence oriented to deep learning techniques

A Brain-Computer Interface, BCI, can decode the brain signals corresponding to the intentions of individuals who have lost neuromuscular connection, to reestablish communication to control external devices. To this aim, BCI acquires brain signals as Electroencephalography (EEG) or Electrocorticography (ECoG), uses signal processing techniques and extracts features to train classifiers for providing proper control instructions. BCI development has increased in the last decades, improving its performance through the use of different signal processing techniques for feature extraction and artificial intelligence approaches for classification, such as deep learning-oriented classifiers. All of these can assure more accurate assistive systems but also can enable an analysis of the learning process of signal characteristics for the classification task. Initially, this work proposes the use of a priori knowledge and a correlation measure to select the most discriminative ECoG signal electrodes. Then, signals are processed using spatial filtering and three different types of temporal filtering, followed by a classifier made of stacked autoencoders and a softmax layer to discriminate between ECoG signals from two types of visual stimuli. Results show that the average accuracy obtained is 97% (+/- 0.02%), which is similar to state-of-the-art techniques, nevertheless, this method uses minimal prior physiological and an automated statistical technique to select some electrodes to train the classifier. Also, this work presents classifier analysis, figuring out which are the most relevant signal features useful for visual stimuli classification. The features and physiological information such as the brain areas involved are compared. Finally, this research uses Convolutional Neural Networks (CNN) or Convnets to classify 5 categories of motor tasks EEG signals. Movement-related cortical potentials (MRCPs) are used as a priori information to improve the processing of time-frequency representation of EEG signals. Results show an increase of more than 25% in average accuracy compared to a state-of-the-art method that uses the same database. In addition, an analysis of CNN or ConvNets filters and feature maps is done to and the most relevant signal characteristics that can help classify the five types of motor tasks.DoctoradoDoctor en Ingeniería Eléctrica y Electrónic

Supported diagnosis of adhd from eeg signals based on hidden markov models and probability product kernels

Attention deficit hyperactivity disorder (ADHD), most often present in childhood, may persist in adult life, hampering personal development. However, ADHD diagnosis is a real challenge since it highly depends on the clinical observation of the patient, the parental and scholar information, and the specialist expertise. Despite demanded objective diagnosis aids from biosignals, the physiological biomarkers lack robustness and significance under the non-stationary and non-linear electroencephalographic dynamics. Therefore, this work presents a supported diagnosis methodology for ADHD from the dynamic characterization of EEG based on hidden Markov models (HMM) and probability product kernels (PPK). Based on the symptom of impulsivity, the proposed approach trains an HMM for each subject from EEG signals in failed inhibition tasks. In the first instance, PPK measures the similarity between subjects through the inner product between their trained HMMs. Then, given the computational costs, fast computation of PPK for HMM facilitates parameter tuning of kernel similarity. Finally, the Kernel Principal Component Analysis (KPCA) projects the PPK to a lower dimensional space, allowing the interpretability of the results. Thus, a support vector machine supports the diagnosis of ADHD as a classification task using PPK as the inner product operator. The methodology compared classification results on EEG signals with all channels, channels of interest (COI), and analysis in the Theta, Alpha, and Beta frequency bands. The results show an accuracy rate of 97.0% in the Beta band in COI, which supports the assumption that this frequency rhythm may be correlated to differences between ADHD and controls regarding attentional allocation during the execution of the cognitive task.El trastorno por déficit de atención e hiperactividad (TDAH), que suele presentarse en la infancia, puede persistir en la vida adulta, obstaculizando el desarrollo personal. Sin embargo, el diagnóstico del TDAH es un verdadero reto, ya que depende en gran medida de la observación clínica del paciente, de la información de los padres y de los estudiosos, y de la experiencia de los especialistas. A pesar de la demanda de ayudas para el diagnóstico objetivo a partir de bioseñales, los biomarcadores fisiológicos carecen de robustez y significación bajo la dinámica electroencefalográfica no estacionaria y no lineal. Por lo tanto, este trabajo presenta una metodología de diagnóstico apoyada para el TDAH a partir de la caracterización dinámica del EEG basada en modelos ocultos de Markov (HMM) y productos de kernel de probabilidad (PPK). Basándose en el síntoma de impulsividad, el enfoque propuesto entrena un HMM para cada sujeto a partir de las señales del EEG en tareas de inhibición fallidas. En primer lugar, el PPK mide la similitud entre los sujetos a través del producto interno entre sus HMMs entrenados. Luego, dados los costes computacionales, el cálculo rápido de PPK para los HMM facilita el ajuste de los parámetros de similitud del kernel. Por último, el Análisis de Componentes Principales del Kernel (KPCA) proyecta el PPK a un espacio de menor dimensión, lo que permite la interpretabilidad de los resultados. Así, una máquina de vectores de apoyo apoya el diagnóstico del TDAH como una tarea de clasificación utilizando el PPK como operador de producto interno. La metodología comparó los resultados de clasificación en señales de EEG con todos los canales, canales de interés (COI), y análisis en las bandas de frecuencia Theta, Alpha, y Beta. Los resultados muestran una tasa de precisión del 97,0% en la banda Beta en COI, lo que apoya la suposición de que este ritmo de frecuencia puede estar correlacionado con las diferencias entre el TDAH y los controles en cuanto a la asignación atencional durante la ejecución de la tarea cognitiva.MaestríaMagíster en Ingeniería EléctricaContents 1 List of Symbols and Abbreviations 5 1.1 Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.2 Abbrevations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 Introduction 7 2.1 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 Justification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.3 State of the art . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.4 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.4.1 General objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.4.2 Specific objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3 Develop a multichannel time series classification methodology taking into account signal dynamics 13 3.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.1.1 Similarity between time series . . . . . . . . . . . . . . . . . . . . . . 13 3.2 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.2.1 EEG Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.2.2 HMM training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.2.3 Parameter tuning and Classification . . . . . . . . . . . . . . . . . . . 17 3.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 4 Develop a time series classification methodology that takes into account spectral information and reduces the computational cost of training. 21 4.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 4.1.1 Fast computation of PPK for HMM . . . . . . . . . . . . . . . . . . . 22 4.2 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 4.2.1 Synthetic Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 4.2.2 Training and Parameter tuning and classification . . . . . . . . . . . 24 4.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 1 CONTENTS 4.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 5 Develop a methodology for visualizing stochastic representations to facilitate the interpretability of inference machines 32 5.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 5.1.1 Model interpretability . . . . . . . . . . . . . . . . . . . . . . . . . . 32 5.1.2 Low-dimensional HMM visualization . . . . . . . . . . . . . . . . . . 33 5.1.3 Low-dimensional state visualization . . . . . . . . . . . . . . . . . . 34 5.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 5.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 6 Conclusions 4

Shallow convolutional network excel for classifying motor imagery EEG in BCI applications

Many studies applying Brain-Computer Interfaces (BCIs) based on Motor Imagery (MI) tasks for rehabilitation have demonstrated the important role of detecting the Event-Related Desynchronization (ERD) to recognize the user’s motor intention. Nowadays, the development of MI-based BCI approaches without or with very few calibration stages session-by-session for different days or weeks is still an open and emergent scope. In this work, a new scheme is proposed by applying Convolutional Neural Networks (CNN) for MI classification, using an end-to-end Shallow architecture that contains two convolutional layers for temporal and spatial feature extraction. We hypothesize that a BCI designed for capturing event-related desynchronization/synchronization (ERD/ERS) at the CNN input, with an adequate network design, may enhance the MI classification with fewer calibration stages. The proposed system using the same architecture was tested on three public datasets through multiple experiments, including both subject-specific and non-subject-specific training. Comparable and also superior results with respect to the state-of-the-art were obtained. On subjects whose EEG data were never used in the training process, our scheme also achieved promising results with respect to existing non-subject-specific BCIs, which shows greater progress in facilitating clinical applications

Motor decoding from the posterior parietal cortex using deep neural networks

Objective. Motor decoding is crucial to translate the neural activity for brain-computer interfaces (BCIs) and provides information on how motor states are encoded in the brain. Deep neural networks (DNNs) are emerging as promising neural decoders. Nevertheless, it is still unclear how different DNNs perform in different motor decoding problems and scenarios, and which network could be a good candidate for invasive BCIs. Approach. Fully-connected, convolutional, and recurrent neural networks (FCNNs, CNNs, RNNs) were designed and applied to decode motor states from neurons recorded from V6A area in the posterior parietal cortex (PPC) of macaques. Three motor tasks were considered, involving reaching and reach-to-grasping (the latter under two illumination conditions). DNNs decoded nine reaching endpoints in 3D space or five grip types using a sliding window approach within the trial course. To evaluate decoders simulating a broad variety of scenarios, the performance was also analyzed while artificially reducing the number of recorded neurons and trials, and while performing transfer learning from one task to another. Finally, the accuracy time course was used to analyze V6A motor encoding. Main results. DNNs outperformed a classic Naive Bayes classifier, and CNNs additionally outperformed XGBoost and Support Vector Machine classifiers across the motor decoding problems. CNNs resulted the top-performing DNNs when using less neurons and trials, and task-to-task transfer learning improved performance especially in the low data regime. Lastly, V6A neurons encoded reaching and reach-to-grasping properties even from action planning, with the encoding of grip properties occurring later, closer to movement execution, and appearing weaker in darkness. Significance. Results suggest that CNNs are effective candidates to realize neural decoders for invasive BCIs in humans from PPC recordings also reducing BCI calibration times (transfer learning), and that a CNN-based data-driven analysis may provide insights about the encoding properties and the functional roles of brain regions