Exposing Internal Attentional Brain States using Single-Trial EEG Analysis with Combined Imaging Modalities

The goal of this dissertation is to explore the neural correlates of endogenous task-related attentional modulations. Natural fluctuations in task engagement are challenging to study, primarily because they are by nature not event related and thus cannot be controlled experimentally. Here we exploit well-accepted links between attention and various measures of neural activity while subjects perform simple target detection tasks that leave their minds free to wander. We use multimodal neuroimaging, specifically simultaneous electroencephalograpy and functional magnetic resonance imaging (EEG-fMRI) and EEG-pupillometry, with data-driven machine learning methods and study activity across the whole brain. We investigate BOLD fMRI correlates of EEG variability spanning each trial, enabling us to unravel a cascade of attention-related activations and determine their temporal ordering. We study activity during auditory and visual paradigms independently, and we also combine data to investigate supra modal attention systems. Without aiming to study known attention-related functional brain networks, we found correlates of attentional modulations in areas representative of the default mode network (DMN), ventral attention network (VAN), locus coeruleus norepinephrine (LC-NE) system, and regions implicated in generation of the extensively-studied P300 EEG response to target stimuli. Our results reveal complex interactions between known attentional systems, and do so non-invasively to study normal fluctuations of task engagement in the human brain

Spatiotemporal dynamics of attention networks revealed by representational similarity analysis of EEG and fMRI

The fronto-parietal attention networks have been extensively studied with functional magnetic resonance imaging (fMRI), but spatiotemporal dynamics of these networks are not well understood. We measured event-related potentials (ERPs) with electroencephalography (EEG) and collected fMRI data from identical experiments where participants performed visual and auditory discrimination tasks separately or simultaneously and with or without distractors. To overcome the low temporal resolution of fMRI, we used a novel ERP-based application of multivariate representational similarity analysis (RSA) to parse time-averaged fMRI pattern activity into distinct spatial maps that each corresponded, in representational structure, to a short temporal ERP segment. Discriminant analysis of ERP-fMRI correlations revealed 8 cortical networks-2 sensory, 3 attention, and 3 other-segregated by 4 orthogonal, temporally multifaceted and spatially distributed functions. We interpret these functions as 4 spatiotemporal components of attention: modality-dependent and stimulus-driven orienting, top-down control, mode transition, and response preparation, selection and execution.Peer reviewe

EEG To FMRI Synthesis: Is Deep Learning a Candidate?

Advances on signal, image and video generation underly major breakthroughs on generative medical imaging tasks, including Brain Image Synthesis. Still, the extent to which functional Magnetic Ressonance Imaging (fMRI) can be mapped from the brain electrophysiology remains largely unexplored. This work provides the first comprehensive view on how to use state-of-the-art principles from Neural Processing to synthesize fMRI data from electroencephalographic (EEG) data. Given the distinct spatiotemporal nature of haemodynamic and electrophysiological signals, this problem is formulated as the task of learning a mapping function between multivariate time series with highly dissimilar structures. A comparison of state-of-the-art synthesis approaches, including Autoencoders, Generative Adversarial Networks and Pairwise Learning, is undertaken. Results highlight the feasibility of EEG to fMRI brain image mappings, pinpointing the role of current advances in Machine Learning and showing the relevance of upcoming contributions to further improve performance. EEG to fMRI synthesis offers a way to enhance and augment brain image data, and guarantee access to more affordable, portable and long-lasting protocols of brain activity monitoring. The code used in this manuscript is available in Github and the datasets are open source

Multimodal Investigation of Brain Network Systems: From Brain Structure and Function to Connectivity and Neuromodulation

The field of cognitive neuroscience has benefited greatly from multimodal investigations of the human brain, which integrate various tools and neuroimaging data to understand brain functions and guide treatments for brain disorders. In this dissertation, we present a series of studies that illustrate the use of multimodal approaches to investigate brain structure and function, brain connectivity, and neuromodulation effects. Firstly, we propose a novel landmark-guided region-based spatial normalization technique to accurately quantify brain morphology, which can improve the sensitivity and specificity of functional imaging studies. Subsequently, we shift the investigation to the characteristics of functional brain activity due to visual stimulations. Our findings reveal that the task-evoked positive blood-oxygen-level dependent (BOLD) response is accompanied by sustained negative BOLD responses in the visual cortex. These negative BOLD responses are likely generated through subcortical neuromodulatory systems with distributed ascending projections to the cortex. To further explore the cortico-subcortical relationship, we conduct a multimodal investigation that involves simultaneous data acquisition of pupillometry, electroencephalography (EEG), and functional magnetic resonance imaging (fMRI). This investigation aims to examine the connectivity of brain circuits involved in the cognitive processes of salient stimuli. Using pupillary response as a surrogate measure of activity in the locus coeruleus-norepinephrine system, we find that the pupillary response is associated with the reorganization of functional brain networks during salience processing. In addition, we propose a cortico-subcortical integrated network reorganization model with potential implications for understanding attentional processing and network switching. Lastly, we employ a multimodal investigation that involves concurrent transcranial magnetic stimulation (TMS), EEG, and fMRI to explore network perturbations and measurements of the propagation effects. In a preliminary exploration on brain-state dependency of TMS-induced effects, we find that the propagation of left dorsolateral prefrontal cortex TMS to regions in the lateral frontoparietal network might depend on the brain-state, as indexed by the EEG prefrontal alpha phase. Overall, the studies in this dissertation contribute to the understanding of the structural and functional characteristics of brain network systems, and may inform future investigations that use multimodal methodological approaches, such as pupillometry, brain connectivity, and neuromodulation tools. The work presented in this dissertation has potential implications for the development of efficient and personalized treatments for major depressive disorder, attention deficit hyperactivity disorder, and Alzheimer's disease

Spatiotemporal Dynamics of Attention Networks Revealed by Representational Similarity Analysis of EEG and fMRI

The fronto-parietal attention networks have been extensively studied with functional magnetic resonance imaging (fMRI), but spatiotemporal dynamics of these networks are not well understood. We measured event-related potentials (ERPs) with electroencephalography (EEG) and collected fMRI data from identical experiments where participants performed visual and auditory discrimination tasks separately or simultaneously and with or without distractors. To overcome the low temporal resolution of fMRI, we used a novel ERP-based application of multivariate representational similarity analysis (RSA) to parse time-averaged fMRI pattern activity into distinct spatial maps that each corresponded, in representational structure, to a short temporal ERP segment. Discriminant analysis of ERP-fMRI correlations revealed 8 cortical networks—2 sensory, 3 attention, and 3 other—segregated by 4 orthogonal, temporally multifaceted and spatially distributed functions. We interpret these functions as 4 spatiotemporal components of attention: modality-dependent and stimulus-driven orienting, top-down control, mode transition, and response preparation, selection and execution.</p

Single-Trial EEG-fMRI Reveals the Generation Process of the Mismatch Negativity

Although research on the mismatch negativity (MMN) has been ongoing for 40 years, the generation process of the MMN remains largely unknown. In this study, we used a single-trial electro-encephalography (EEG)-functional magnetic resonance imaging (fMRI) coupling method which can analyze neural activity with both high temporal and high spatial resolution and thus assess the generation process of the MMN. We elicited the MMN with an auditory oddball paradigm while recording simultaneous EEG and fMRI. We divided the MMN into five equal-durational phases. Utilizing the single-trial variability of the MMN, we analyzed the neural generators of the five phases, thereby determining the spatiotemporal generation process of the MMN. We found two distinct bottom-up prediction error propagations: first from the auditory cortex to the motor areas and then from the auditory cortex to the inferior frontal gyrus (IFG). Our results support the regularity-violation hypothesis of MMN generation

Safety and data quality of EEG recorded simultaneously with multi-band fMRI

Simultaneously recorded electroencephalography and functional magnetic resonance imaging (EEG-fMRI) is highly informative yet technically challenging. Until recently, there has been little information about EEG data quality and safety when used with newer multi-band (MB) fMRI sequences. Here, we measure the relative heating of a MB protocol compared with a standard single-band (SB) protocol considered to be safe. We also evaluated EEG quality recorded concurrently with the MB protocol on humans. We compared radiofrequency (RF)-related heating at multiple electrodes and magnetic field magnitude, B_(1+RMS), of a MB fMRI sequence with whole-brain coverage (TR = 440 ms, MB factor = 4) against a previously recommended, safe SB sequence using a phantom outfitted with a 64-channel EEG cap. Next, 9 human subjects underwent eyes-closed resting state EEG-fMRI using the MB sequence. Additionally, in three of the subjects resting state EEG was recorded also during the SB sequence and in an fMRI-free condition to directly compare EEG data quality across scanning conditions. EEG data quality was assessed by the ability to remove gradient and cardioballistic artifacts along with a clean spectrogram. The heating induced by the MB sequence was lower than that of the SB sequence by a factor of 0.73 ± 0.38. This is consistent with an expected heating ratio of 0.64, calculated from the square of the ratio of B_(1+RMS) values of the sequences. In the resting state EEG data, gradient and cardioballistic artifacts were successfully removed using traditional template subtraction. All subjects showed an individual alpha peak in the spectrogram with a posterior topography characteristic of eyes-closed EEG. The success of artifact rejection for the MB sequence was comparable to that in traditional SB sequences. Our study shows that B_(1+RMS) is a useful indication of the relative heating of fMRI protocols. This observation indicates that simultaneous EEG-fMRI recordings using this MB sequence can be safe in terms of RF-related heating, and that EEG data recorded using this sequence is of acceptable quality after traditional artifact removal techniques