Multimodal imaging of human brain activity: rational, biophysical aspects and modes of integration

Until relatively recently the vast majority of imaging and electrophysiological studies of human brain activity have relied on single-modality measurements usually correlated with readily observable or experimentally modified behavioural or brain state patterns. Multi-modal imaging is the concept of bringing together observations or measurements from different instruments. We discuss the aims of multi-modal imaging and the ways in which it can be accomplished using representative applications. Given the importance of haemodynamic and electrophysiological signals in current multi-modal imaging applications, we also review some of the basic physiology relevant to understanding their relationship

Functional Source Separation for EEG-fMRI Fusion: Application to Steady-State Visual Evoked Potentials

Neurorobotics is one of the most ambitious fields in robotics, driving integration of interdisciplinary data and knowledge. One of the most productive areas of interdisciplinary research in this area has been the implementation of biologically-inspired mechanisms in the development of autonomous systems. Specifically, enabling such systems to display adaptive behavior such as learning from good and bad outcomes, has been achieved by quantifying and understanding the neural mechanisms of the brain networks mediating adaptive behaviors in humans and animals. For example, associative learning from aversive or dangerous outcomes is crucial for an autonomous system, to avoid dangerous situations in the future. A body of neuroscience research has suggested that the neurocomputations in the human brain during associative learning involve re-shaping of sensory responses. The nature of these adaptive changes in sensory processing during learning however are not yet well enough understood to be readily implemented into on-board algorithms for robotics application. Toward this overall goal, we record the simultaneous electroencephalogram (EEG) and functional magnetic resonance imaging (fMRI), characterizing one candidate mechanism, i.e., large-scale brain oscillations. The present report examines the use of Functional Source Separation (FSS) as an optimization step in EEG-fMRI fusion that harnesses timing information to constrain the solutions that satisfy physiological assumptions. We applied this approach to the voxel-wise correlation of steady-state visual evoked potential (ssVEP) amplitude and blood oxygen level-dependent imaging (BOLD), across both time series. The results showed the benefit of FSS for the extraction of robust ssVEP signals during simultaneous EEG-fMRI recordings. Applied to data from a 3-phase aversive conditioning paradigm, the correlation maps across the three phases (habituation, acquisition, extinction) show converging results, notably major overlapping areas in both primary and extended visual cortical regions, including calcarine sulcus, lingual cortex, and cuneus. In addition, during the acquisition phase when aversive learning occurs, we observed additional correlations between ssVEP and BOLD in the anterior cingulate cortex (ACC) as well as the precuneus and superior temporal gyrus

When Is Simultaneous Recording Necessary? A Guide for Researchers Considering Combined EEG-fMRI.

Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) provide non-invasive measures of brain activity at varying spatial and temporal scales, offering different views on brain function for both clinical and experimental applications. Simultaneous recording of these measures attempts to maximize the respective strengths of each method, while compensating for their weaknesses. However, combined recording is not necessary to address all research questions of interest, and experiments may have greater statistical power to detect effects by maximizing the signal-to-noise ratio in separate recording sessions. While several existing papers discuss the reasons for or against combined recording, this article aims to synthesize these arguments into a flow chart of questions that researchers can consider when deciding whether to record EEG and fMRI separately or simultaneously. Given the potential advantages of simultaneous EEG-fMRI, the aim is to provide an initial overview of the most important concepts and to direct readers to relevant literature that will aid them in this decision

Functional Source Separation for EEG-fMRI Fusion: Application to Steady-State Visual Evoked Potentials

Neurorobotics is one of the most ambitious fields in robotics, driving integration of interdisciplinary data and knowledge. One of the most productive areas of interdisciplinary research in this area has been the implementation of biologically-inspired mechanisms in the development of autonomous systems. Specifically, enabling such systems to display adaptive behavior such as learning from good and bad outcomes, has been achieved by quantifying and understanding the neural mechanisms of the brain networks mediating adaptive behaviors in humans and animals. For example, associative learning from aversive or dangerous outcomes is crucial for an autonomous system, to avoid dangerous situations in the future. A body of neuroscience research has suggested that the neurocomputations in the human brain during associative learning involve re-shaping of sensory responses. The nature of these adaptive changes in sensory processing during learning however are not yet well enough understood to be readily implemented into on-board algorithms for robotics application. Toward this overall goal, we record the simultaneous electroencephalogram (EEG) and functional magnetic resonance imaging (fMRI), characterizing one candidate mechanism, i.e., large-scale brain oscillations. The present report examines the use of Functional Source Separation (FSS) as an optimization step in EEG-fMRI fusion that harnesses timing information to constrain the solutions that satisfy physiological assumptions. We applied this approach to the voxel-wise correlation of steady-state visual evoked potential (ssVEP) amplitude and blood oxygen level-dependent imaging (BOLD), across both time series. The results showed the benefit of FSS for the extraction of robust ssVEP signals during simultaneous EEG-fMRI recordings. Applied to data from a 3-phase aversive conditioning paradigm, the correlation maps across the three phases (habituation, acquisition, extinction) show converging results, notably major overlapping areas in both primary and extended visual cortical regions, including calcarine sulcus, lingual cortex, and cuneus. In addition, during the acquisition phase when aversive learning occurs, we observed additional correlations between ssVEP and BOLD in the anterior cingulate cortex (ACC) as well as the precuneus and superior temporal gyrus

Reference-free removal of EEG-fMRI ballistocardiogram artifacts with harmonic regression

Combining electroencephalogram (EEG) recording and functional magnetic resonance imaging (fMRI) offers the potential for imaging brain activity with high spatial and temporal resolution. This potential remains limited by the significant ballistocardiogram (BCG) artifacts induced in the EEG by cardiac pulsation-related head movement within the magnetic field. We model the BCG artifact using a harmonic basis, pose the artifact removal problem as a local harmonic regression analysis, and develop an efficient maximum likelihood algorithm to estimate and remove BCG artifacts. Our analysis paradigm accounts for time-frequency overlap between the BCG artifacts and neurophysiologic EEG signals, and tracks the spatiotemporal variations in both the artifact and the signal. We evaluate performance on: simulated oscillatory and evoked responses constructed with realistic artifacts; actual anesthesia-induced oscillatory recordings; and actual visual evoked potential recordings. In each case, the local harmonic regression analysis effectively removes the BCG artifacts, and recovers the neurophysiologic EEG signals. We further show that our algorithm outperforms commonly used reference-based and component analysis techniques, particularly in low SNR conditions, the presence of significant time-frequency overlap between the artifact and the signal, and/or large spatiotemporal variations in the BCG. Because our algorithm does not require reference signals and has low computational complexity, it offers a practical tool for removing BCG artifacts from EEG data recorded in combination with fMRI.National Institutes of Health (U.S.) (Award DP1-OD003646)National Institutes of Health (U.S.) (Award TR01-GM104948)National Institutes of Health (U.S.) (Grant R44NS071988)National Institute of Neurological Diseases and Stroke (U.S.) (Grant Grant R44NS071988

Concurrent fNIRS and EEG for brain function investigation: A systematic, methodology-focused review

Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) stand as state-of-the-art techniques for non-invasive functional neuroimaging. On a unimodal basis, EEG has poor spatial resolution while presenting high temporal resolution. In contrast, fNIRS offers better spatial resolution, though it is constrained by its poor temporal resolution. One important merit shared by the EEG and fNIRS is that both modalities have favorable portability and could be integrated into a compatible experimental setup, providing a compelling ground for the development of a multimodal fNIRS-EEG integration analysis approach. Despite a growing number of studies using concurrent fNIRS-EEG designs reported in recent years, the methodological reference of past studies remains unclear. To fill this knowledge gap, this review critically summarizes the status of analysis methods currently used in concurrent fNIRS-EEG studies, providing an up-to-date overview and guideline for future projects to conduct concurrent fNIRS-EEG studies. A literature search was conducted using PubMed and Web of Science through 31 August 2021. After screening and qualification assessment, 92 studies involving concurrent fNIRS-EEG data recordings and analyses were included in the final methodological review. Specifically, three methodological categories of concurrent fNIRS-EEG data analyses, including EEG-informed fNIRS analyses, fNIRS-informed EEG analyses, and parallel fNIRS-EEG analyses, were identified and explained with detailed description. Finally, we highlighted current challenges and potential directions in concurrent fNIRS-EEG data analyses in future research

EEG-fMRI Based Information Theoretic Characterization of the Human Perceptual Decision System

The modern metaphor of the brain is that of a dynamic information processing device. In the current study we investigate how a core cognitive network of the human brain, the perceptual decision system, can be characterized regarding its spatiotemporal representation of task-relevant information. We capitalize on a recently developed information theoretic framework for the analysis of simultaneously acquired electroencephalography (EEG) and functional magnetic resonance imaging data (fMRI) (Ostwald et al. (2010), NeuroImage 49: 498–516). We show how this framework naturally extends from previous validations in the sensory to the cognitive domain and how it enables the economic description of neural spatiotemporal information encoding. Specifically, based on simultaneous EEG-fMRI data features from n = 13 observers performing a visual perceptual decision task, we demonstrate how the information theoretic framework is able to reproduce earlier findings on the neurobiological underpinnings of perceptual decisions from the response signal features' marginal distributions. Furthermore, using the joint EEG-fMRI feature distribution, we provide novel evidence for a highly distributed and dynamic encoding of task-relevant information in the human brain

Imaging of epileptic activity using EEG-correlated functional MRI.

This thesis describes the method of EEG-correlated fMRI and its application to patients with epilepsy. First, an introduction on MRI and functional imaging methods in the field of epilepsy is provided. Then, the present and future role of EEG-correlated fMRI in the investigation of the epilepsies is discussed. The fourth chapter reviews the important practicalities of EEG-correlated fMRI that were addressed in this project. These included patient safety, EEG quality and MRI artifacts during EEG-correlated fMRI. Technical solutions to enable safe, good quality EEG recordings inside the MR scanner are presented, including optimisation of the EEG recording techniques and algorithms for the on-line subtraction of pulse and image artifact. In chapter five, a study applying spike-triggered fMRI to patients with focal epilepsy (n = 24) is presented. Using statistical parametric mapping (SPM), cortical Blood Oxygen Level-Dependent (BOLD) activations corresponding to the presumed generators of the interictal epileptiform discharges (IED) were identified in twelve patients. The results were reproducible in repeated experiments in eight patients. In the remaining patients no significant activation (n = 10) was present or the activation did not correspond to the presumed epileptic focus (n = 2). The clinical implications of this finding are discussed. In a second study it was demonstrated that in selected patients, individual (as opposed to averaged) IED could also be associated with hemodynamic changes detectable with fMRI. Chapter six gives examples of combination of EEG-correlated fMRI with other modalities to obtain complementary information on interictal epileptiform activity and epileptic foci. One study compared spike-triggered fMRI activation maps with EEG source analysis based on 64-channel scalp EEG recordings of interictal spikes using co-registration of both modalities. In all but one patient, source analysis solutions were anatomically concordant with the BOLD activation. Further, the combination of spike- triggered fMRI with diffusion tensor and chemical shift imaging is demonstrated in a patient with localisation-related epilepsy. In chapter seven, applications of EEG-correlated fMRI in different areas of neuroscience are discussed. Finally, the initial imaging findings with the novel technique for the simultaneous and continuous acquisition of fMRI and EEG data are presented as an outlook to future applications of EEG-correlated fMRI. In conclusion, the technical problems of both EEG-triggered fMRI and simultaneous EEG-correlated fMRI are now largely solved. The method has proved useful to provide new insights into the generation of epileptiform activity and other pathological and physiological brain activity. Currently, its utility in clinical epileptology remains unknown