Multimodal imaging of human brain activity: rational, biophysical aspects and modes of integration

Until relatively recently the vast majority of imaging and electrophysiological studies of human brain activity have relied on single-modality measurements usually correlated with readily observable or experimentally modified behavioural or brain state patterns. Multi-modal imaging is the concept of bringing together observations or measurements from different instruments. We discuss the aims of multi-modal imaging and the ways in which it can be accomplished using representative applications. Given the importance of haemodynamic and electrophysiological signals in current multi-modal imaging applications, we also review some of the basic physiology relevant to understanding their relationship

EEG correlated functional MRI and postoperative outcome in focal epilepsy

Background: The main challenge in assessing patients with epilepsy for resective surgery is localising seizure onset. Frequently, identification of the irritative and seizure onset zones requires invasive EEG. EEG correlated functional MRI (EEG-fMRI) is a novel imaging technique which may provide localising information with regard to these regions. In patients with focal epilepsy, interictal epileptiform discharge (IED) correlated blood oxygen dependent level (BOLD) signal changes were observed in approximately 50% of patients in whom IEDs are recorded. In 70%, these are concordant with expected seizure onset defined by non-invasive electroclinical information. Assessment of clinical validity requires post-surgical outcome studies which have, to date, been limited to case reports of correlation with intracranial EEG. The value of EEG-fMRI was assessed in patients with focal epilepsy who subsequently underwent epilepsy surgery, and IED correlated fMRI signal changes were related to the resection area and clinical outcome. Methods: Simultaneous EEG-fMRI was recorded in 76 patients undergoing presurgical evaluation and the locations of IED correlated preoperative BOLD signal change were compared with the resected area and postoperative outcome. Results: 21 patients had activations with epileptic activity on EEG-fMRI and 10 underwent surgical resection. Seven of 10 patients were seizure free following surgery and the area of maximal BOLD signal change was concordant with resection in six of seven patients. In the remaining three patients, with reduced seizure frequency post-surgically, areas of significant IED correlated BOLD signal change lay outside the resection. 42 of 55 patients who had no IED related activation underwent resection. Conclusion: These results show the potential value of EEG-fMRI in presurgical evaluation

Noncanonical spike-related BOLD responses in focal epilepsy

Till now, most studies of the Blood Oxygen Level-Dependent (BOLD) response to interictal epileptic discharges (IED) have assumed that its time course matches closely to that of brief physiological stimuli, commonly called the canonical event-related haemodynamic response function (canonical HRF). Analyses based on that assumption have produced significant response patterns that are generally concordant with prior electroclinical data. In this work, we used a more flexible model of the event-related response, a Fourier basis set, to investigate the presence of other responses in relation to individual IED in 30 experiments in patients with focal epilepsy. We found significant responses that had a noncanonical time course in 37% of cases, compared with 40% for the conventional, canonical HRF-based approach. In two cases, the Fourier analysis suggested activations where the conventional model did not. The noncanonical activations were almost always remote from the presumed generator of epileptiform activity. In the majority of cases with noncanonical responses, the noncanonical responses in single-voxel clusters were suggestive of artifacts. We did not find evidence for IED-related noncanonical HRFs arising from areas of pathology, suggesting that the BOLD response to IED is primarily canonical. Noncanonical responses may represent a number of phenomena, including artefacts and propagated epileptiform activity

Modelling large motion events in fMRI studies of patients with epilepsy

EEG-correlated fMRI can provide localisation information on the generators of epileptiform discharges in patients with focal epilepsy. To increase the technique's clinical potential, it is important to consider ways of optimising the yield of each experiment while minimizing the risk of false-positive activation. Head motion can lead to severe image degradation and result in false-positive activation and is usually worse in patients than in healthy subjects. We performed general linear model fMRI data analysis on simultaneous EEG–fMRI data acquired in 34 cases with focal epilepsy. Signal changes associated with large inter-scan motion events (head jerks) were modelled using modified design matrices that include ‘scan nulling’ regressors. We evaluated the efficacy of this approach by mapping the proportion of the brain for which F-tests across the additional regressors were significant. In 95% of cases, there was a significant effect of motion in 50% of the brain or greater; for the scan nulling effect, the proportion was 36%; this effect was predominantly in the neocortex. We conclude that careful consideration of the motion-related effects in fMRI studies of patients with epilepsy is essential and that the proposed approach can be effective

The spatio-temporal mapping of epileptic networks: Combination of EEG–fMRI and EEG source imaging

Simultaneous EEG–fMRI acquisitions in patients with epilepsy often reveal distributed patterns of Blood Oxygen Level Dependant (BOLD) change correlated with epileptiform discharges. We investigated if electrical source imaging (ESI) performed on the interictal epileptiform discharges (IED) acquired during fMRI acquisition could be used to study the dynamics of the networks identified by the BOLD effect, thereby avoiding the limitations of combining results from separate recordings. Nine selected patients (13 IED types identified) with focal epilepsy underwent EEG–fMRI. Statistical analysis was performed using SPM5 to create BOLD maps. ESI was performed on the IED recorded during fMRI acquisition using a realistic head model (SMAC) and a distributed linear inverse solution (LAURA). ESI could not be performed in one case. In 10/12 remaining studies, ESI at IED onset (ESIo) was anatomically close to one BOLD cluster. Interestingly, ESIo was closest to the positive BOLD cluster with maximal statistical significance in only 4/12 cases and closest to negative BOLD responses in 4/12 cases. Very small BOLD clusters could also have clinical relevance in some cases. ESI at later time frame (ESIp) showed propagation to remote sources co-localised with other BOLD clusters in half of cases. In concordant cases, the distance between maxima of ESI and the closest EEG–fMRI cluster was less than 33 mm, in agreement with previous studies. We conclude that simultaneous ESI and EEG–fMRI analysis may be able to distinguish areas of BOLD response related to initiation of IED from propagation areas. This combination provides new opportunities for investigating epileptic networks

The role of spatial frequency information for ERP components sensitive to faces and emotional facial expression

To investigate the impact of spatial frequency on emotional facial expression analysis, ERPs were recorded in response to low spatial frequency (LSF), high spatial frequency (HSF), and unfiltered broad spatial frequency (BSF) faces with fearful or neutral expressions, houses, and chairs. In line with previous findings, BSF fearful facial expressions elicited a greater frontal positivity than BSF neutral facial expressions, starting at about 150 ms after stimulus onset. In contrast, this emotional expression effect was absent for HSF and LSF faces. Given that some brain regions involved in emotion processing, such as amygdala and connected structures, are selectively tuned to LSF visual inputs, these data suggest that ERP effects of emotional facial expression do not directly reflect activity in these regions. It is argued that higher order neocortical brain systems are involved in the generation of emotion-specific waveform modulations. The face-sensitive N170 component was neither affected by emotional facial expression nor by spatial frequency information

Aberrant activity in conceptual networks underlies N400 deficits and unusual thoughts in schizophrenia.

BackgroundThe N400 event-related potential (ERP) is triggered by meaningful stimuli that are incongruous, or unmatched, with their semantic context. Functional magnetic resonance imaging (fMRI) studies have identified brain regions activated by semantic incongruity, but their precise links to the N400 ERP are unclear. In schizophrenia (SZ), N400 amplitude reduction is thought to reflect overly broad associations in semantic networks, but the abnormalities in brain networks underlying deficient N400 remain unknown. We utilized joint independent component analysis (JICA) to link temporal patterns in ERPs to neuroanatomical patterns from fMRI and investigate relationships between N400 amplitude and neuroanatomical activation in SZ patients and healthy controls (HC).MethodsSZ patients (n = 24) and HC participants (n = 25) performed a picture-word matching task, in which words were either matched (APPLE→apple) by preceding pictures, or were unmatched by semantically related (in-category; IC, APPLE→lemon) or unrelated (out of category; OC, APPLE→cow) pictures, in separate ERP and fMRI sessions. A JICA "data fusion" analysis was conducted to identify the fMRI brain regions specifically associated with the ERP N400 component. SZ and HC loading weights were compared and correlations with clinical symptoms were assessed.ResultsJICA identified an ERP-fMRI "fused" component that captured the N400, with loading weights that were reduced in SZ. The JICA map for the IC condition showed peaks of activation in the cingulate, precuneus, bilateral temporal poles and cerebellum, whereas the JICA map from the OC condition was linked primarily to visual cortical activation and the left temporal pole. Among SZ patients, fMRI activity from the IC condition was inversely correlated with unusual thought content.ConclusionsThe neural networks associated with the N400 ERP response to semantic violations depends on conceptual relatedness. These findings are consistent with a distributed network underlying neural responses to semantic incongruity including unimodal visual areas as well as integrative, transmodal areas. Unusual thoughts in SZ may reflect impaired processing in transmodal hub regions such as the precuneus, leading to overly broad semantic associations

The pain matrix reloaded: a salience detection system for the body

Neuroimaging and neurophysiological studies have shown that nociceptive stimuli elia salience detection system for the bodycit responses in an extensive cortical network including somatosensory, insular and cingulate areas, as well as frontal and parietal areas. This network, often referred to as the "pain matrix", is viewed as representing the activity by which the intensity and unpleasantness of the perception elicited by a nociceptive stimulus are represented. However, recent experiments have reported (i) that pain intensity can be dissociated from the magnitude of responses in the "pain matrix", (ii) that the responses in the "pain matrix" are strongly influenced by the context within which the nociceptive stimuli appear, and (iii) that non-nociceptive stimuli can elicit cortical responses with a spatial configuration similar to that of the "pain matrix". For these reasons, we propose an alternative view of the functional significance of this cortical network, in which it reflects a system involved in detecting, orienting attention towards, and reacting to the occurrence of salient sensory events. This cortical network might represent a basic mechanism through which significant events for the body's integrity are detected, regardless of the sensory channel through which these events are conveyed. This function would involve the construction of a multimodal cortical representation of the body and nearby space. Under the assumption that this network acts as a defensive system signaling potentially damaging threats for the body, emphasis is no longer on the quality of the sensation elicited by noxious stimuli but on the action prompted by the occurrence of potential threats

Within-trial effects of stimulus-reward associations

While a globally energizing influence of motivation has long been appreciated in psychological research, a series of more recent studies has described motivational influences on specific cognitive operations ranging from visual attention, to cognitive control, to memory formation. In the majority of these studies, a cue predicts the potential to win money in a subsequent task, thus allowing for modulations of proactive task preparation. Here we describe some recent studies using tasks that communicate reward availability without such cues by directly associating specific task features with reward. Despite abolishing the cue-based preparation phase, these studies show similar performance benefits. Given the clear difference in temporal structure, a central question is how these behavioral effects are brought about, and in particular whether control processes can rapidly be enhanced reactively. We present some evidence in favor of this notion. Although additional influences, for example sensory prioritization of reward-related features, could contribute to the reward-related performance benefits, those benefits seem to strongly rely on enhancements of control processes during task execution. Still, for a better mechanistic understanding of reward benefits in these two principal paradigms (cues vs. no cues), more work is needed that directly compares the underlying processes. We anticipate that reward benefits can be brought about in a very flexible fashion depending on the exact nature of the reward manipulation and task, and that a better understanding of these processes will not only be relevant for basic motivation research, but that it can also be valuable for educational and psychopathological contexts

EEG-informed fMRI reveals spatiotemporal characteristics of perceptual decision making

Single-unit and multiunit recordings in primates have already established that decision making involves at least two general stages of neural processing: representation of evidence from early sensory areas and accumulation of evidence to a decision threshold from decision-related regions. However, the relay of information from early sensory to decision areas, such that the accumulation process is instigated, is not well understood. Using a cued paradigm and single-trial analysis of electroencephalography (EEG), we previously reported on temporally specific components related to perceptual decision making. Here, we use information derived from our previous EEG recordings to inform the analysis of fMRI data collected for the same behavioral task to ascertain the cortical origins of each of these EEG components. We demonstrate that a cascade of events associated with perceptual decision making takes place in a highly distributed neural network. Of particular importance is an activation in the lateral occipital complex implicating perceptual persistence as a mechanism by which object decision making in the human brain is instigated