EEG-based image classification using an efficient geometric deep network based on functional connectivity

To ensure that the FC-GDN is properly calibrated for the EEG-ImageNet dataset, we subject it to extensive training and gather all of the relevant weights for its parameters. Making use of the FC-GDN pseudo-code. The dataset is split into a "train" and "test" section in Kfold cross-validation. Ten-fold recommends using ten folds, with one fold being selected as the test split at each iteration. This divides the dataset into 90% training data and 10% test data. In order to train all 10 folds without overfitting, it is necessary to apply this procedure repeatedly throughout the whole dataset. Each training fold is arrived at after several iterations. After training all ten folds, results are analyzed. For each iteration, the FC-GDN weights are optimized by the SGD and ADAM optimizers. The ideal network design parameters are based on the convergence of the trains and the precision of the tests. This study offers a novel geometric deep learning-based network architecture for classifying visual stimulation categories using electroencephalogram (EEG) data from human participants while they watched various sorts of images. The primary goals of this study are to (1) eliminate feature extraction from GDL-based approaches and (2) extract brain states via functional connectivity. Tests with the EEG-ImageNet database validate the suggested method's efficacy. FC-GDN is more efficient than other cutting-edge approaches for boosting classification accuracy, requiring fewer iterations. In computational neuroscience, neural decoding addresses the problem of mind-reading. Because of its simplicity of use and temporal precision, Electroencephalographys (EEG) are commonly employed to monitor brain activity. Deep neural networks provide a variety of ways to detecting brain activity. Using a Function Connectivity (FC) - Geometric Deep Network (GDN) and EEG channel functional connectivity, this work directly recovers hidden states from high-resolution temporal data. The time samples taken from each channel are utilized to represent graph signals on a topological connection network based on EEG channel functional connectivity. A novel graph neural network architecture evaluates users' visual perception state utilizing extracted EEG patterns associated to various picture categories using graphically rendered EEG recordings as training data. The efficient graph representation of EEG signals serves as the foundation for this design. Proposal for an FC-GDN EEG-ImageNet test. Each category has a maximum of 50 samples. Nine separate EEG recorders were used to obtain these images. The FC-GDN approach yields 99.4% accuracy, which is 0.1% higher than the most sophisticated method presently availabl

Psychophysiology-based QoE assessment : a survey

We present a survey of psychophysiology-based assessment for quality of experience (QoE) in advanced multimedia technologies. We provide a classification of methods relevant to QoE and describe related psychological processes, experimental design considerations, and signal analysis techniques. We summarize multimodal techniques and discuss several important aspects of psychophysiology-based QoE assessment, including the synergies with psychophysical assessment and the need for standardized experimental design. This survey is not considered to be exhaustive but serves as a guideline for those interested to further explore this emerging field of research

Deep Learning Approaches for Seizure Video Analysis: A Review

Seizure events can manifest as transient disruptions in the control of movements which may be organized in distinct behavioral sequences, accompanied or not by other observable features such as altered facial expressions. The analysis of these clinical signs, referred to as semiology, is subject to observer variations when specialists evaluate video-recorded events in the clinical setting. To enhance the accuracy and consistency of evaluations, computer-aided video analysis of seizures has emerged as a natural avenue. In the field of medical applications, deep learning and computer vision approaches have driven substantial advancements. Historically, these approaches have been used for disease detection, classification, and prediction using diagnostic data; however, there has been limited exploration of their application in evaluating video-based motion detection in the clinical epileptology setting. While vision-based technologies do not aim to replace clinical expertise, they can significantly contribute to medical decision-making and patient care by providing quantitative evidence and decision support. Behavior monitoring tools offer several advantages such as providing objective information, detecting challenging-to-observe events, reducing documentation efforts, and extending assessment capabilities to areas with limited expertise. The main applications of these could be (1) improved seizure detection methods; (2) refined semiology analysis for predicting seizure type and cerebral localization. In this paper, we detail the foundation technologies used in vision-based systems in the analysis of seizure videos, highlighting their success in semiology detection and analysis, focusing on work published in the last 7 years. Additionally, we illustrate how existing technologies can be interconnected through an integrated system for video-based semiology analysis.Comment: Accepted in Epilepsy & Behavio

Investigation of neural activity in Schizophrenia during resting-state MEG : using non-linear dynamics and machine-learning to shed light on information disruption in the brain

Environ 25% de la population mondiale est atteinte de troubles psychiatriques qui sont typiquement associés à des problèmes comportementaux, fonctionnels et/ou cognitifs et dont les corrélats neurophysiologiques sont encore très mal compris. Non seulement ces dysfonctionnements réduisent la qualité de vie des individus touchés, mais ils peuvent aussi devenir un fardeau pour les proches et peser lourd dans l’économie d’une société. Cibler les mécanismes responsables du fonctionnement atypique du cerveau en identifiant des biomarqueurs plus robustes permettrait le développement de traitements plus efficaces. Ainsi, le premier objectif de cette thèse est de contribuer à une meilleure caractérisation des changements dynamiques cérébraux impliqués dans les troubles mentaux, plus précisément dans la schizophrénie et les troubles d’humeur. Pour ce faire, les premiers chapitres de cette thèse présentent, en intégral, deux revues de littératures systématiques que nous avons menées sur les altérations de connectivité cérébrale, au repos, chez les patients schizophrènes, dépressifs et bipolaires. Ces revues révèlent que, malgré des avancées scientifiques considérables dans l’étude de l’altération de la connectivité cérébrale fonctionnelle, la dimension temporelle des mécanismes cérébraux à l’origine de l’atteinte de l’intégration de l’information dans ces maladies, particulièrement de la schizophrénie, est encore mal comprise. Par conséquent, le deuxième objectif de cette thèse est de caractériser les changements cérébraux associés à la schizophrénie dans le domaine temporel. Nous présentons deux études dans lesquelles nous testons l’hypothèse que la « disconnectivité temporelle » serait un biomarqueur important en schizophrénie. Ces études explorent les déficits d’intégration temporelle en schizophrénie, en quantifiant les changements de la dynamique neuronale dite invariante d’échelle à partir des données magnétoencéphalographiques (MEG) enregistrés au repos chez des patients et des sujets contrôles. En particulier, nous utilisons (1) la LRTCs (long-range temporal correlation, ou corrélation temporelle à longue-distance) calculée à partir des oscillations neuronales et (2) des analyses multifractales pour caractériser des modifications de l’activité cérébrale arythmique. Par ailleurs, nous développons des modèles de classification (en apprentissage-machine supervisé) pour mieux cerner les attributs corticaux et sous-corticaux permettant une distinction robuste entre les patients et les sujets sains. Vu que ces études se basent sur des données MEG spontanées enregistrées au repos soit avec les yeux ouvert, ou les yeux fermées, nous nous sommes par la suite intéressés à la possibilité de trouver un marqueur qui combinerait ces enregistrements. La troisième étude originale explore donc l’utilité des modulations de l’amplitude spectrale entre yeux ouverts et fermées comme prédicteur de schizophrénie. Les résultats de ces études démontrent des changements cérébraux importants chez les patients schizophrènes au niveau de la dynamique d’invariance d’échelle. Elles suggèrent une dégradation du traitement temporel de l’information chez les patients, qui pourrait être liée à leurs symptômes cognitifs et comportementaux. L’approche multimodale de cette thèse, combinant la magétoencéphalographie, analyses non-linéaires et apprentissage machine, permet de mieux caractériser l’organisation spatio-temporelle du signal cérébrale au repos chez les patients atteints de schizophrénie et chez des individus sains. Les résultats fournissent de nouvelles preuves supportant l’hypothèse d’une « disconnectivité temporelle » en schizophrénie, et étendent les recherches antérieures, en explorant la contribution des structures cérébrales profondes et en employant des mesures non-linéaires avancées encore sous-exploitées dans ce domaine. L’ensemble des résultats de cette thèse apporte une contribution significative à la quête de nouveaux biomarqueurs de la schizophrénie et démontre l’importance d’élucider les altérations des propriétés temporelles de l’activité cérébrales intrinsèque en psychiatrie. Les études présentées offrent également un cadre méthodologique pouvant être étendu à d’autres psychopathologie, telles que la dépression.Psychiatric disorders affect nearly a quarter of the world’s population. These typically bring about debilitating behavioural, functional and/or cognitive problems, for which the underlying neural mechanisms are poorly understood. These symptoms can significantly reduce the quality of life of affected individuals, impact those close to them, and bring on an economic burden on society. Hence, targeting the baseline neurophysiology associated with psychopathologies, by identifying more robust biomarkers, would improve the development of effective treatments. The first goal of this thesis is thus to contribute to a better characterization of neural dynamic alterations in mental health illnesses, specifically in schizophrenia and mood disorders. Accordingly, the first chapter of this thesis presents two systematic literature reviews, which investigate the resting-state changes in brain connectivity in schizophrenia, depression and bipolar disorder patients. Great strides have been made in neuroimaging research in identifying alterations in functional connectivity. However, these two reviews reveal a gap in the knowledge about the temporal basis of the neural mechanisms involved in the disruption of information integration in these pathologies, particularly in schizophrenia. Therefore, the second goal of this thesis is to characterize the baseline temporal neural alterations of schizophrenia. We present two studies for which we hypothesize that the resting temporal dysconnectivity could serve as a key biomarker in schizophrenia. These studies explore temporal integration deficits in schizophrenia by quantifying neural alterations of scale-free dynamics using resting-state magnetoencephalography (MEG) data. Specifically, we use (1) long-range temporal correlation (LRTC) analysis on oscillatory activity and (2) multifractal analysis on arrhythmic brain activity. In addition, we develop classification models (based on supervised machine-learning) to detect the cortical and sub-cortical features that allow for a robust division of patients and healthy controls. Given that these studies are based on MEG spontaneous brain activity, recorded at rest with either eyes-open or eyes-closed, we then explored the possibility of finding a distinctive feature that would combine both types of resting-state recordings. Thus, the third study investigates whether alterations in spectral amplitude between eyes-open and eyes-closed conditions can be used as a possible marker for schizophrenia. Overall, the three studies show changes in the scale-free dynamics of schizophrenia patients at rest that suggest a deterioration of the temporal processing of information in patients, which might relate to their cognitive and behavioural symptoms. The multimodal approach of this thesis, combining MEG, non-linear analyses and machine-learning, improves the characterization of the resting spatiotemporal neural organization of schizophrenia patients and healthy controls. Our findings provide new evidence for the temporal dysconnectivity hypothesis in schizophrenia. The results extend on previous studies by characterizing scale-free properties of deep brain structures and applying advanced non-linear metrics that are underused in the field of psychiatry. The results of this thesis contribute significantly to the identification of novel biomarkers in schizophrenia and show the importance of clarifying the temporal properties of altered intrinsic neural dynamics. Moreover, the presented studies offer a methodological framework that can be extended to other psychopathologies, such as depression

Computational and Perceptual Characterization of Auditory Attention

Humans are remarkably capable at making sense of a busy acoustic environment in real-time, despite the constant cacophony of sounds reaching our ears. Attention is a key component of the system that parses sensory input, allocating limited neural resources to elements with highest informational value to drive cognition and behavior. The focus of this thesis is the perceptual, neural, and computational characterization of auditory attention. Pioneering studies exploring human attention to natural scenes came from the visual domain, spawning a number of hypotheses on how attention operates among the visual pathway, as well as a considerable amount of computational work that attempt to model human perception. Comparatively, our understanding of auditory attention is yet very elementary, particularly pertaining to attention automatically drawn to salient sounds in the environment, such as a loud explosion. In this work, we explore how human perception is affected by the saliency of sound, characterized across a variety of acoustic features, such as pitch, loudness, and timbre. Insight from psychoacoustical data is complemented with neural measures of attention recorded directly from the brain using electroencephalography (EEG). A computational model of attention is presented, tracking the statistical regularities of incoming sound among a high-dimensional feature space to build predictions of future feature values. The model determines salient time points that will attract attention by comparing its predictions to the observed sound features. The high degree of agreement between the model and human experimental data suggests predictive coding as a potential mechanism of attention in the auditory pathway. We investigate different modes of volitional attention to natural acoustic scenes with a "cocktail-party" simulation. We argue that the auditory system can direct attention in at least three unique ways (globally, based on features, and based on objects) and that perception can be altered depending on how attention is deployed. Further, we illustrate how the saliency of sound affects the various modes of attention. The results of this work improve our understanding of auditory attention, highlighting the temporally evolving nature of sound as a significant distinction between audition and vision, with a focus on using natural scenes that engage the full capability of human attention

Breaking Down the Barriers To Operator Workload Estimation: Advancing Algorithmic Handling of Temporal Non-Stationarity and Cross-Participant Differences for EEG Analysis Using Deep Learning

This research focuses on two barriers to using EEG data for workload assessment: day-to-day variability, and cross- participant applicability. Several signal processing techniques and deep learning approaches are evaluated in multi-task environments. These methods account for temporal, spatial, and frequential data dependencies. Variance of frequency- domain power distributions for cross-day workload classification is statistically significant. Skewness and kurtosis are not significant in an environment absent workload transitions, but are salient with transitions present. LSTMs improve day- to-day feature stationarity, decreasing error by 59% compared to previous best results. A multi-path convolutional recurrent model using bi-directional, residual recurrent layers significantly increases predictive accuracy and decreases cross-participant variance. Deep learning regression approaches are applied to a multi-task environment with workload transitions. Accounting for temporal dependence significantly reduces error and increases correlation compared to baselines. Visualization techniques for LSTM feature saliency are developed to understand EEG analysis model biases

Visual Saliency and Image Reconstruction from EEG Signals via an Effective Geometric Deep Network-Based Generative Adversarial Network

Data Availability Statement: The EEG-ImageNet dataset used in this study is publicly available in this address: https://tinyurl.com/eeg-visual-classification (accessed on 10 October 2022).Copyright © 2022 by the authors. Reaching out the function of the brain in perceiving input data from the outside world is one of the great targets of neuroscience. Neural decoding helps us to model the connection between brain activities and the visual stimulation. The reconstruction of images from brain activity can be achieved through this modelling. Recent studies have shown that brain activity is impressed by visual saliency, the important parts of an image stimuli. In this paper, a deep model is proposed to reconstruct the image stimuli from electroencephalogram (EEG) recordings via visual saliency. To this end, the proposed geometric deep network-based generative adversarial network (GDN-GAN) is trained to map the EEG signals to the visual saliency maps corresponding to each image. The first part of the proposed GDN-GAN consists of Chebyshev graph convolutional layers. The input of the GDN part of the proposed network is the functional connectivity-based graph representation of the EEG channels. The output of the GDN is imposed to the GAN part of the proposed network to reconstruct the image saliency. The proposed GDN-GAN is trained using the Google Colaboratory Pro platform. The saliency metrics validate the viability and efficiency of the proposed saliency reconstruction network. The weights of the trained network are used as initial weights to reconstruct the grayscale image stimuli. The proposed network realizes the image reconstruction from EEG signals.This research received no external funding

Recent Advances in Signal Processing

The signal processing task is a very critical issue in the majority of new technological inventions and challenges in a variety of applications in both science and engineering fields. Classical signal processing techniques have largely worked with mathematical models that are linear, local, stationary, and Gaussian. They have always favored closed-form tractability over real-world accuracy. These constraints were imposed by the lack of powerful computing tools. During the last few decades, signal processing theories, developments, and applications have matured rapidly and now include tools from many areas of mathematics, computer science, physics, and engineering. This book is targeted primarily toward both students and researchers who want to be exposed to a wide variety of signal processing techniques and algorithms. It includes 27 chapters that can be categorized into five different areas depending on the application at hand. These five categories are ordered to address image processing, speech processing, communication systems, time-series analysis, and educational packages respectively. The book has the advantage of providing a collection of applications that are completely independent and self-contained; thus, the interested reader can choose any chapter and skip to another without losing continuity

Neural processing of semantic content in movies

Naturalistic stimuli, such as movies, contain interacting, multimodal and semantic features and allow for free exploration through eye movements. The full extent of neural responses to features such as motion, film cuts and eye movement behavior has not been established. The main hypothesis of this thesis is that complex multimodal and semantic stimuli in naturalistic movies engage a widespread ensemble of locations across the entire brain. To address this question I analyzed simultaneous intracranial and eyetracking data from over 6,000 electrodes across 23 patients with intractable epilepsy. Responses to fast eye movements – saccades – and film cuts are widespread across the entire brain, while responses to motion are restricted to visual brain areas. Higher-order brain areas respond differentially to semantic and low-level changes across film cuts and saccades. Movies have also recently been used in combination with resting state scans to investigate the utility of functional connectivity as a potential biomarker for psychiatric disorders. Functional connectivity in fMRI data measured during resting state and movie conditions is reliable, subject-specific and related to phenotype. However, it is unclear whether functional connectivity of EEG also possesses these qualities, which are required for the clinical use of neural biomarkers. I hypothesize that functional connectivity networks measured in EEG data recorded during movie watching are a predictor of psychiatric phenotypes similar to functional connectivity of fMRI. I demonstrate that functional connectivity of EEG is reliable, subject-specific and related to phenotypes. However, the patterns of functional connectivity differ in EEG and fMRI, suggesting the measures capture complementary information. In summary, these results demonstrate that the semantic content in movies allows one to study neural processing in naturalistic settings. In addition, EEG functional connectivity recorded during resting state and movie condition is reliabe, subject-specific and related to phenotype