Multimodal Integration: fMRI, MRI, EEG, MEG

This chapter provides a comprehensive survey of the motivations, assumptions and pitfalls associated with combining signals such as fMRI with EEG or MEG. Our initial focus in the chapter concerns mathematical approaches for solving the localization problem in EEG and MEG. Next we document the most recent and promising ways in which these signals can be combined with fMRI. Specically, we look at correlative analysis, decomposition techniques, equivalent dipole tting, distributed sources modeling, beamforming, and Bayesian methods. Due to difculties in assessing ground truth of a combined signal in any realistic experiment difculty further confounded by lack of accurate biophysical models of BOLD signal we are cautious to be optimistic about multimodal integration. Nonetheless, as we highlight and explore the technical and methodological difculties of fusing heterogeneous signals, it seems likely that correct fusion of multimodal data will allow previously inaccessible spatiotemporal structures to be visualized and formalized and thus eventually become a useful tool in brain imaging research

Predictive decoding of neural data

In the last five decades the number of techniques available for non-invasive functional imaging has increased dramatically. Researchers today can choose from a variety of imaging modalities that include EEG, MEG, PET, SPECT, MRI, and fMRI. This doctoral dissertation offers a methodology for the reliable analysis of neural data at different levels of investigation. By using statistical learning algorithms the proposed approach allows single-trial analysis of various neural data by decoding them into variables of interest. Unbiased testing of the decoder on new samples of the data provides a generalization assessment of decoding performance reliability. Through consecutive analysis of the constructed decoder\u27s sensitivity it is possible to identify neural signal components relevant to the task of interest. The proposed methodology accounts for covariance and causality structures present in the signal. This feature makes it more powerful than conventional univariate methods which currently dominate the neuroscience field. Chapter 2 describes the generic approach toward the analysis of neural data using statistical learning algorithms. Chapter 3 presents an analysis of results from four neural data modalities: extracellular recordings, EEG, MEG, and fMRI. These examples demonstrate the ability of the approach to reveal neural data components which cannot be uncovered with conventional methods. A further extension of the methodology, Chapter 4 is used to analyze data from multiple neural data modalities: EEG and fMRI. The reliable mapping of data from one modality into the other provides a better understanding of the underlying neural processes. By allowing the spatial-temporal exploration of neural signals under loose modeling assumptions, it removes potential bias in the analysis of neural data due to otherwise possible forward model misspecification. The proposed methodology has been formalized into a free and open source Python framework for statistical learning based data analysis. This framework, PyMVPA, is described in Chapter 5

The relationship between MEG and fMRI

In recent years functional neuroimaging techniques such as fMRI, MEG, EEG and PET have provided researchers with a wealth of information on human brain function. However none of these modalities can measure directly either the neuro-electrical or neuro-chemical processes that mediate brain function. This means that metrics directly reflecting brain ‘activity’ must be inferred from other metrics (e.g. magnetic fields (MEG) or haemodynamics (fMRI)). To overcome this limitation, many studies seek to combine multiple complementary modalities and an excellent example of this is the combination of MEG (which has high temporal resolution) with fMRI (which has high spatial resolution). However, the full potential of multi-modal approaches can only be truly realised in cases where the relationship between metrics is known. In this paper, we explore the relationship between measurements made using fMRI and MEG. We describe the origins of the two signals as well as their relationship to electrophysiology. We review multiple studies that have attempted to characterise the spatial relationship between fMRI and MEG, and we also describe studies that exploit the rich information content of MEG to explore differing relationships between MEG and fMRI across neural oscillatory frequency bands. Monitoring the brain at “rest” has become of significant recent interest to the neuroimaging community and we review recent evidence comparing MEG and fMRI metrics of functional connectivity. A brief discussion of the use of magnetic resonance spectroscopy (MRS) to probe the relationship between MEG/fMRI and neurochemistry is also given. Finally, we highlight future areas of interest and offer some recommendations for the parallel use of fMRI and MEG

Computational methods for Bayesian estimation of neuromagnetic sources

The electromagnetic inverse problem in human brain research consists of determining underlying source currents in the brain based on measurements outside the head. Solution to the inverse problem is ambiguous, necessitating the use of prior information and modeling assumptions for obtaining reasonable inverse estimates. In this study, we create new and improve existing computational methods for estimating neuromagnetic sources in the human brain. One straightforward way of incorporating presumptions to this problem is to formulate it in a probabilistic Bayesian manner. Bayesian statistics is largely based on modeling uncertainties associated with parameters constituting the model by representing them with probability distributions. In this work, existing neuroscientific knowledge and information from anatomical and functional magnetic resonance imaging are used as prior assumptions in model implementation. The neuromagnetic inverse problem is resolved with two different approaches. First, we perform the analysis using distributed source current modeling and infer some arbitrary parameter choices and the source currents from the measurement data by using numerical sampling methods. We apply similar strategies to cortically constrained current dipole localization and suggest using functional magnetic resonance imaging data for guiding the sampling algorithm. The models are tested with simulated and measured data. The presented methods are rather automatic, yielding plausible and robust inverse estimates of cortical current sources. With the spatiotemporal dipole localization model, the inclusion of functional magnetic resonance imaging data improves performance of the numerical sampling method. However, apparent multimodality of the parameter posterior distribution causes complications especially with empirical data. We suggest using loose cortical orientation constraints for smoothing down the complicated posterior distribution instead of marginal improvements to the sampling scheme. This might help to overcome the somewhat limited mixing properties of the sampling algorithm and ease the inconvenient multimodality of the posterior distribution.Ihmisaivojen tutkimukseen liittyvällä sähkömagneettisella käänteisongelmalla tarkoitetaan aivojen virtalähteiden paikantamista pään ulkopuolisten mittausten perusteella. Ongelmaan ei ole yksikäsitteistä ratkaisua, joten mallintamisessa on käytettävä ennakko-oletuksia järkevien ratkaisujen tuottamiseksi. Tässä tutkimuksessa kehitämme uusia ja parannamme olemassaolevia laskennallisia menetelmiä aivoissa syntyvien magneettikenttiä tuottavien lähteiden paikantamiseksi. Kenties yksinkertaisin tapa lisätä ennakko-oletuksia tähän ongelmaan on käyttää bayesilaista mallintamista. Bayesilainen tilastotiede perustuu pitkälti parametrien epävarmuuksien mallintamiseen ja esittämiseen todennäköisyysjakaumin. Työn mallien muodostamisessa käytetään apuna aivojen toiminnallisesta ja rakenteellisesta magneettikuvauksesta saatavaa neurotieteellistä ennakkotietoa. Sähkömagneettisen käänteisongelman ratkaisuun käytämme kahta eri menetelmää. Aluksi analysoimme aivojen pinnalle muodostettuja virtalähdejakaumamalleja ja pyrimme laskennallisia otantamenetelmiä käyttäen arvioimaan virtojen sekä muuten etukäteen mielivaltaisesti valittavien parametrien arvoja mittausaineistosta. Sovellamme samantyyppistä otantamenetelmää malliin, missä dipolaarisia virtalähteitä rajoittaa aivojen kuorikerroksen anatomia ja fysiologia. Ehdotamme lisäksi toiminnallisen magneettikuvauksen tuottaman mittausaineiston käyttöä otantamenetelmän apuna. Malleja testataan sekä simuloidulla että kokeellisella mittausaineistolla. Kehitetyt menetelmät ovat hyvin automaattisia ja tuottavat järkeviä ratkaisuja magneettisten mittausten lähteiksi. Dipolaaristen virtalähteiden paikallis-ajalliseen määrittämiseen käytetyn otantamenetelmän suorituskyky parantuu toiminnallisesta magneettikuvauksesta saatavan tiedon avulla. Mallin parametrien todennäköisyysjakauma on kuitenkin selvästi monihuippuinen aiheuttaen ongelmia erityisesti kokeellisen mittausaineiston kanssa. Otantamenetelmän parannusten sijaan ehdotamme väljempien aivojen kuorikerroksen anatomiaan perustuvien rajoitteiden käyttöä, jolloin itse parametrien todennäköisyysjakauma saattaa muuttua helpommin käsiteltäväksi. Tämä parantanee myös nykyisen otantamenetelmän tehokkuutta tässä ongelmassa ja helpottaa siten monihuippuisten jakaumien jatkokäsittelyä.reviewe

Multimodal functional neuroimaging: new insights from novel head modeling methodologies

2009/2010Neuroimaging plays a critically important role in neuroscience research and management of neurological and mental disorders. Modern neuroimaging techniques rely on various “source” signals that change across different spatial and temporal scales in accompany with neuronal activity. Nowadays, several types of noninvasive neuroimaging modalities are available based on biophysical signals related to either brain electrophysiology or hemodynamics/metabolism. In this dissertation, advanced model-based neuroimaging methods for the estimation of cortical brain activity from combined high-resolution electroencephalography (EEG), multimodal Magnetic Resonance Imaging (MRI) and functional Magnetic Resonance Imaging (fMRI) data are presented. The present dissertation begins with a review of the current state-of-the-art in the major neuroimaging techniques. Particular attention has been devoted to EEG modelling since such signals propagate (virtually) instantaneously from the activated neuronal tissues via volume conduction to the recording sites on/above the scalp surface. The instantaneous nature of EEG indicates an intrinsically high temporal resolution and precision, which make it well suited for studying brain functions on the neuronal time scale. The collective nature suggests low spatial resolution and specificity, which impede mapping brain functions in great regional details. However, this is regardless of recent advancements in electromagnetic source imaging, which has led to great strides in improving the EEG/MEG spatial resolution to a centimetre scale or even smaller. These methods entail: 1) modeling the brain electrical activity; 2) modeling the head volume conduction process so as to link the modeled electrical activity to EEG; and 3) reconstructing the brain electrical activity from recorded EEG data. For this aim, a subject's multicompartment head model (scalp, skull, CSF, brain cortex, white matter) is constructed from either individual magnetic resonance images or approximated geometry models. We compared different spherical and realistic head modelling techniques in estimating EEG forward solutions from current dipole sources distributed on a standard cortical space reconstructed from Montreal Neurological Institute (MNI) MRI data. Computer simulations are presented for three different four-shell head models, two with realistic geometry, either surface-based (BEM) or volume-based (FDM), and the corresponding sensor-fitted spherical-shaped model. Point Spread Function (PSF) and Lead Field (LF) cross-correlation analyses were performed for 26 symmetric dipole sources to quantitatively assess models’ accuracy in EEG source reconstruction. Both statistical and imaging analysis point to the realistic geometry as a relevant factor of improvement, particularly important when considering sources placed in the temporal or in the occipital cortex. In these situations, using a realistic head model will allow a better spatial discrimination of neural sources when compared to the spherical model. Moreover a brief overview of Diffusion Weighted Imaging and Diffusion Tensor Imaging is also given, as their application in modelling refinement is increasing the accuracy and the complexity of the brain models. Both fMRI and EEG represent brain activity in terms of a reliable anatomical localization and a detailed temporal evolution of neural signals. Simultaneous EEG-fMRI recordings offer the possibility to greatly enrich the significance and the interpretation of the single modality results because the same neural processes are observed from the same brain at the same time. Nonetheless, the different physical nature of the measured signals by the two techniques renders the coupling not always straightforward, especially in cognitive experiments where spatially localized and distributed effects coexist and evolve temporally at different temporal scales. The purpose of the last chapter is to illustrate the combination of simultaneously recorded EEG and fMRI signals exploiting the principles of EEG distributed source modelling. We define a common source space for fMRI and EEG signal projection and novel framework for the spatial and temporal comparative analysis. We use simultaneous EEG-fMRI in order to explore the relationship between the envelope of spontaneous neuronal oscillations in the alpha frequency band (8-13 Hz) recorded with EEG during eyes closed rest and spontaneous fluctuations of the fMRI BOLD signal. We showed on a single-subject analysis how the presented approach, when combined to an accurate realistic head modelling, is able to localize the alpha rhythmic modulation in the occipital visual area and the parieto-occipital sulcus. This finding is in line with recent studies, asserting that, within these regions, time-frequency analysis and phase-synchronization analysis indicated increased alpha power and alpha-band phase-synchronization in eyes-closed condition versus eyes-open condition. Given the lack in the scientific literature of group-analysis experimental studies performed with realistic modelling approach in this field, this topic will be further investigated in future work.XXII Ciclo198

EpiGauss : caracterização espacio-temporal da actividade cerebral em epilepsia

Doutoramento em Engenharia ElectrotécnicaA epilepsia é uma patologia cerebral que afecta cerca de 0,5% da população mundial. Nas epilepsias focais, o principal objectivo clínico é a localização da zona epileptogénica (área responsável pelas crises), uma informação crucial para uma terapêutica adequada. Esta tese é centrada na caracterização da actividade cerebral electromagnética do cérebro epiléptico. As contribuições nesta área, entre a engenharia e neurologia clínica, são em duas direcções. Primeiro, mostramos que os conceitos associados às pontas podem ser imprecisos e não ter uma definição objectiva, tornando necessária uma reformulação de forma a definir uma referência fiável em estudos relacionados com a análise de pontas. Mostramos que as características das pontas em EEG são estatisticamente diferentes das pontas em MEG. Esta constatação leva a concluir que a falta de objectividade na definição de ponta na literatura pode induzir utilizações erradas de conceitos associados ao EEG na análise de MEG. Também verificamos que o uso de conjuntos de detecções de pontas efectuadas por especialistas (MESS) como referência pode fornecer resultados enganadores quando apenas baseado em critérios de consenso clínico, nomeadamente na avaliação da sensibilidade e especificidade de métodos computorizados de detecção de pontas Em segundo lugar, propomos o uso de métodos estatísticos para ultrapassar a falta de precisão e objectividade das definições relacionadas com pontas. Propomos um novo método de neuroimagem suportado na caracterização de geradores electromagnéticos – EpiGauss – baseado na análise individual dos geradores de eventos do EEG que explora as suas estruturas espacio-temporais através da análise de “clusters”. A aplicação de análise de “clusters” à análise geradores de eventos do EEG tem como objectivo usar um método não supervisionado, para encontrar estruturas espacio-temporais dps geradores relevantes. Este método, como processo não supervisionado, é orientado a utilizadores clínicos e apresenta os resultados sob forma de imagens médicas com interpretação similar a outras técnicas de imagiologia cerebral. Com o EpiGauss, o utilizador pode determinar a localização estatisticamente mais provável de geradores, a sua estabilidade espacial e possíveis propagações entre diferente áreas do cérebro. O método foi testado em dois estudos clínicos envolvendo doentes com epilepsia associada aos hamartomas hipotalâmicos e o outro com doentes com diagnóstico de epilepsia occipital. Em ambos os estudos, o EpiGauss foi capaz de identificar a zona epileptogénica clínica, de forma consistente com a história e avaliação clínica dos neurofisiologistas, fornecendo mais informação relativa à estabilidade dos geradores e possíveis percursos de propagação da actividade epileptogénica contribuindo para uma melhor caracterização clínica dos doentes. A conclusão principal desta tese é que o uso de técnicas não supervisionadas, como a análise de “clusters”, associadas as técnicas não-invasivas de EMSI, pode contribuir com um valor acrescido no processo de diagnóstico clínico ao fornecer uma caracterização objectiva e representação visual de padrões complexos espaciotemporais da actividade eléctrica epileptogénica.Epilepsy is a brain pathology that affects 0.5% of the world population. In focal epilepsies, the main clinical objective is the localization of the epileptogenic zone (brain area responsible for the epileptic seizures – EZ), a key information to decide an adequate therapeutic approach. This thesis is centred on electromagnetic activity characterization of the epileptic brain. Our contribution to this boundary area between engineering and clinical neurology is two-folded. First we show that spike related clinical concepts can be unprecise and some do not have objective definitions making necessary a reformulation in order to have a reliable reference in spike related studies. We show that EEG spike wave quantitative features are statistically different from their MEG counterparts. This finding leads to the conclusion that the lack of objective spike feature definitions in the literature can induce the wrong usage of EEG feature definition in MEG analysis. We also show that the use of multi-expert spike selections sets (MESS) as gold standard, although clinically useful, may be misleading whenever defined solely in terms of clinical agreement criteria, namely as references for automatic spike detection algorithms in sensitivity and specificity method analysis. Second, we propose the use of statistical methods to overcome some lack of precision and objectivity in spike related definitions. In this context, we propose a new ElectroMagnetic Source Imaging (EMSI) method – EpiGauss – based on cluster analysis that explores both spatial and temporal information contained in individual events sources analysis characterisation. This automatic cluster method for the analysis of spike related electric generators based in EEG is used to provide an unsupervised tool to find their relevant spatio-temporal structures. This method enables a simple unsupervised procedure aimed for clinical users and presents its results in an intuitive representation similar to other brain imaging techniques. With EpiGauss, the user is able to determine statistically probable source locations, their spatial stability and propagation patterns between different brain areas. The method was tested in two different clinical neurophysiology studies, one with a group of Hypothalamic Hamartomas and another with a group of Occipital Epilepsy patients. In both studies EpiGauss identified the clinical epileptogenic zone, consistent with the clinical background and evaluation of neurophysiologists, providing further information on stability of source locations and their probable propagation pathways that enlarges their clinical interpretation. This thesis main conclusion is that the use of unsupervised techniques, such as clustering, associated with EMSI non-invasive techniques, can bring an added value in clinical diagnosis process by providing objective and visual representation of complex epileptic brain spatio-temporal activity patterns

On mapping epilepsy : magneto- and electroencephalographic characterizations of epileptic activities

Epilepsy is one of the most common neurological disorder, affecting up to 10 individuals per 1000 persons. The disorder have been known for several thousand years, with the first clinical descriptions dating back to ancient times. Nonetheless, characterization of the dynamics underlying epilepsy remains largely unknown. Understanding these patophysiological processes requires unifying both a neurobiological perspective, as well as a technically advanced neuroimaging perspective. The incomplete insight into epilepsy dynamics is reflected by the insufficient treatment options. Approximately 30% of all patients do not respond to anti-epileptic drugs (AEDs) and thus suffers from recurrent seizures despite adequate pharmacological treatments. These pharmacoresistant patients often undergo epilepsy surgery evaluations. Epilepsy surgery aims to resect the part of the brain that generates the epileptic seizure activity (seizure onset zone, SOZ). Nonetheless, up to 50% of all patients relapse after surgery. This can be due to incomplete mapping of both the SOZ and of other structures that might be involved in seizure initiation and propagation. Such cortical and subcortical structures are collectively referred to as the epileptic network. Historically, epilepsy was considered to be either a generalized disorder involving the entire brain, or a highly localized, focal, disorder. The modern technological development of both structural and functional neuroimaging has drastically altered this view. This development has made significant contributions to the now prevailing view that both generalized and focal epilepsies arise from more or less widespread pathological network pathways. Visualization of these pathways play an important role in the presurgical planning. Thus, both improved characterization and understanding of such pathways are pivotal in improvement of epilepsy diagnostics and treatments. It is evident that epilepsy research needs to stand on two legs: Both improved understanding of pathological, neurobiological and neurophysiological process, and improved neuroimaging instrumentation. Epilepsy research do not only span from visualization to understanding of neurophysiological processes, but also from cellular, neuronal, microscopic processes, to dynamical, large-scale network processes. It is well known that neurons involved in epileptic activities exhibit specific, pathological firing patterns. Genetic mutations resulting in neuronal ion channel defects can cause severe, and even lethal, epileptic syndromes in children, clearly illustrating a role for neuron membrane properties in epilepsy. However, cellular processes themselves cannot explain how epileptic seizures can involve, and propagate across, large cortical areas and generate seizure-specific symptomatologies. A strict cellular perspective can neither explain epilepsy-associated pathological interactions between larger distant regions in between seizures. Instead, the dynamical effects of cellular synchronization across both mesoscopic and macroscopic scales also need to be considered. Today, the only means to study such effects in human subjects are by combinations of neuroimaging modalities. However, as all measurement techniques, these exhibit individual limitations that affect the kind of information that can be inferred from these. Thus, once more we reach the conclusion that epilepsy research needs to rest upon both a neurophysiological/neurobiological leg, and a technical/instrumentational leg. In accordance with this necessity of a dual approach to epilepsy, this thesis covers both neurophysiological aspects of epileptic activity development, as well as functional neuroimaging instrumentation development with focus on epileptic activity detection and localization. Part 1 (neurophysiological part) is concerned with the neurophysiological dynamical changes that underlie development of so called interictal epileptiform discharges (IEDs) with special focus on the role of low-frequency oscillations. To this aim, both conventional magnetoencephalography (MEG) and intracranial electroencephalography (iEEG) with neurostimulation is analyzed. Part 2 (instrumentation part) is concerned with development of cutting-edge, novel on-scalp magnetoencephalography (osMEG) within clinical epilepsy evaluations and research with special focus on IEDs. The theses cover both modeling of osMEG characteristics, as well as the first-ever osMEG recording of a temporal lobe epilepsy patient

A Novel Synergistic Model Fusing Electroencephalography and Functional Magnetic Resonance Imaging for Modeling Brain Activities

Study of the human brain is an important and very active area of research. Unraveling the way the human brain works would allow us to better understand, predict and prevent brain related diseases that affect a significant part of the population. Studying the brain response to certain input stimuli can help us determine the involved brain areas and understand the mechanisms that characterize behavioral and psychological traits. In this research work two methods used for the monitoring of brain activities, Electroencephalography (EEG) and functional Magnetic Resonance (fMRI) have been studied for their fusion, in an attempt to bridge together the advantages of each one. In particular, this work has focused in the analysis of a specific type of EEG and fMRI recordings that are related to certain events and capture the brain response under specific experimental conditions. Using spatial features of the EEG we can describe the temporal evolution of the electrical field recorded in the scalp of the head. This work introduces the use of Hidden Markov Models (HMM) for modeling the EEG dynamics. This novel approach is applied for the discrimination of normal and progressive Mild Cognitive Impairment patients with significant results. EEG alone is not able to provide the spatial localization needed to uncover and understand the neural mechanisms and processes of the human brain. Functional Magnetic Resonance imaging (fMRI) provides the means of localizing functional activity, without though, providing the timing details of these activations. Although, at first glance it is apparent that the strengths of these two modalities, EEG and fMRI, complement each other, the fusion of information provided from each one is a challenging task. A novel methodology for fusing EEG spatiotemporal features and fMRI features, based on Canonical Partial Least Squares (CPLS) is presented in this work. A HMM modeling approach is used in order to derive a novel feature-based representation of the EEG signal that characterizes the topographic information of the EEG. We use the HMM model in order to project the EEG data in the Fisher score space and use the Fisher score to describe the dynamics of the EEG topography sequence. The correspondence between this new feature and the fMRI is studied using CPLS. This methodology is applied for extracting features for the classification of a visual task. The results indicate that the proposed methodology is able to capture task related activations that can be used for the classification of mental tasks. Extensions on the proposed models are examined along with future research directions and applications