Applying neural networks for improving the MEG inverse solution

Magnetoencephalography (MEG) and electroencephalography (EEG) are appealing non-invasive methods for recording brain activity with high temporal resolution. However, locating the brain source currents from recordings picked up by the sensors on the scalp introduces an ill-posed inverse problem. The MEG inverse problem one of the most difficult inverse problems in medical imaging. The current standard in approximating the MEG inverse problem is to use multiple distributed inverse solutions – namely dSPM, sLORETA and L2 MNE – to estimate the source current distribution in the brain. This thesis investigates if these inverse solutions can be "post-processed" by a neural network to provide improved accuracy on source locations. Recently, deep neural networks have been used to approximate other ill-posed inverse medical imaging problems with accuracy comparable to current state-of- the-art inverse reconstruction algorithms. Neural networks are powerful tools for approximating problems with limited prior knowledge or problems that require high levels of abstraction. In this thesis a special case of a deep convolutional network, the U-Net, is applied to approximate the MEG inverse problem using the standard inverse solutions (dSPM, sLORETA and L2 MNE) as inputs. The U-Net is capable of learning non-linear relationships between the inputs and producing predictions about the site of single-dipole activation with higher accuracy than the L2 minimum-norm based inverse solutions with the following resolution metrics: dipole localization error (DLE), spatial dispersion (SD) and overall amplitude (OA). The U-Net model is stable and performs better in aforesaid resolution metrics than the inverse solutions with multi-dipole data previously unseen by the U-Net

EpiGauss : caracterização espacio-temporal da actividade cerebral em epilepsia

Doutoramento em Engenharia ElectrotécnicaA epilepsia é uma patologia cerebral que afecta cerca de 0,5% da população mundial. Nas epilepsias focais, o principal objectivo clínico é a localização da zona epileptogénica (área responsável pelas crises), uma informação crucial para uma terapêutica adequada. Esta tese é centrada na caracterização da actividade cerebral electromagnética do cérebro epiléptico. As contribuições nesta área, entre a engenharia e neurologia clínica, são em duas direcções. Primeiro, mostramos que os conceitos associados às pontas podem ser imprecisos e não ter uma definição objectiva, tornando necessária uma reformulação de forma a definir uma referência fiável em estudos relacionados com a análise de pontas. Mostramos que as características das pontas em EEG são estatisticamente diferentes das pontas em MEG. Esta constatação leva a concluir que a falta de objectividade na definição de ponta na literatura pode induzir utilizações erradas de conceitos associados ao EEG na análise de MEG. Também verificamos que o uso de conjuntos de detecções de pontas efectuadas por especialistas (MESS) como referência pode fornecer resultados enganadores quando apenas baseado em critérios de consenso clínico, nomeadamente na avaliação da sensibilidade e especificidade de métodos computorizados de detecção de pontas Em segundo lugar, propomos o uso de métodos estatísticos para ultrapassar a falta de precisão e objectividade das definições relacionadas com pontas. Propomos um novo método de neuroimagem suportado na caracterização de geradores electromagnéticos – EpiGauss – baseado na análise individual dos geradores de eventos do EEG que explora as suas estruturas espacio-temporais através da análise de “clusters”. A aplicação de análise de “clusters” à análise geradores de eventos do EEG tem como objectivo usar um método não supervisionado, para encontrar estruturas espacio-temporais dps geradores relevantes. Este método, como processo não supervisionado, é orientado a utilizadores clínicos e apresenta os resultados sob forma de imagens médicas com interpretação similar a outras técnicas de imagiologia cerebral. Com o EpiGauss, o utilizador pode determinar a localização estatisticamente mais provável de geradores, a sua estabilidade espacial e possíveis propagações entre diferente áreas do cérebro. O método foi testado em dois estudos clínicos envolvendo doentes com epilepsia associada aos hamartomas hipotalâmicos e o outro com doentes com diagnóstico de epilepsia occipital. Em ambos os estudos, o EpiGauss foi capaz de identificar a zona epileptogénica clínica, de forma consistente com a história e avaliação clínica dos neurofisiologistas, fornecendo mais informação relativa à estabilidade dos geradores e possíveis percursos de propagação da actividade epileptogénica contribuindo para uma melhor caracterização clínica dos doentes. A conclusão principal desta tese é que o uso de técnicas não supervisionadas, como a análise de “clusters”, associadas as técnicas não-invasivas de EMSI, pode contribuir com um valor acrescido no processo de diagnóstico clínico ao fornecer uma caracterização objectiva e representação visual de padrões complexos espaciotemporais da actividade eléctrica epileptogénica.Epilepsy is a brain pathology that affects 0.5% of the world population. In focal epilepsies, the main clinical objective is the localization of the epileptogenic zone (brain area responsible for the epileptic seizures – EZ), a key information to decide an adequate therapeutic approach. This thesis is centred on electromagnetic activity characterization of the epileptic brain. Our contribution to this boundary area between engineering and clinical neurology is two-folded. First we show that spike related clinical concepts can be unprecise and some do not have objective definitions making necessary a reformulation in order to have a reliable reference in spike related studies. We show that EEG spike wave quantitative features are statistically different from their MEG counterparts. This finding leads to the conclusion that the lack of objective spike feature definitions in the literature can induce the wrong usage of EEG feature definition in MEG analysis. We also show that the use of multi-expert spike selections sets (MESS) as gold standard, although clinically useful, may be misleading whenever defined solely in terms of clinical agreement criteria, namely as references for automatic spike detection algorithms in sensitivity and specificity method analysis. Second, we propose the use of statistical methods to overcome some lack of precision and objectivity in spike related definitions. In this context, we propose a new ElectroMagnetic Source Imaging (EMSI) method – EpiGauss – based on cluster analysis that explores both spatial and temporal information contained in individual events sources analysis characterisation. This automatic cluster method for the analysis of spike related electric generators based in EEG is used to provide an unsupervised tool to find their relevant spatio-temporal structures. This method enables a simple unsupervised procedure aimed for clinical users and presents its results in an intuitive representation similar to other brain imaging techniques. With EpiGauss, the user is able to determine statistically probable source locations, their spatial stability and propagation patterns between different brain areas. The method was tested in two different clinical neurophysiology studies, one with a group of Hypothalamic Hamartomas and another with a group of Occipital Epilepsy patients. In both studies EpiGauss identified the clinical epileptogenic zone, consistent with the clinical background and evaluation of neurophysiologists, providing further information on stability of source locations and their probable propagation pathways that enlarges their clinical interpretation. This thesis main conclusion is that the use of unsupervised techniques, such as clustering, associated with EMSI non-invasive techniques, can bring an added value in clinical diagnosis process by providing objective and visual representation of complex epileptic brain spatio-temporal activity patterns

Conjugate Gradient Method Applied to Cortical Imaging in EEG/ERP

International audienc

Spatio-Temporal Approaches to Denoising and Feature Extraction in Rapid Image Triage

Ph.DDOCTOR OF PHILOSOPH

Disentangling neuronal pre- and post-response activation in the acquisition of goal-directed behavior through the means of co-registered EEG-fMRI

Behavior is considered goal-directed when the actor integrates information about the subsequent outcome of an action (Balleine & O'Doherty, 2010; Dickinson & Balleine, 1994; Kiesel & Koch, 2012), potentially enabling the anticipation of consequences of an action. Thus, it requires prior acquisition of knowledge about the current contingencies between behavioral responses and their outcomes under certain stimulus conditions (J. Hoffmann & Engelkamp, 2013). This association chain enables events lying in the future to be mentally represented and assessed in terms of value and achievability. However, while neural correlates of instructed goal-directed action integration processes have already been examined in a functional magnetic resonance imaging (fMRI) study using this paradigm (Ruge & Wolfensteller, 2015), there has been no information if those processes are also reflected in Electroencephalography (EEG) and if so which specific EEG parameters are modulated by them. This dissertation set out to investigate neurocognitive mechanisms of instructed outcome response learning utilizing two different imaging methods, namely EEG and fMRI. Study 1 was an exploratory study to answer the question what kinds of learning-related EEG correlates were to expect. The O-R outcome integration specific EEG correlates identified in Study 1 served as regressors in a unified general linear model (EEG-informed fMRI analysis) in the co-registered EEG-fMRI study (Study 2). One of the key questions in this study was if the EEG signal could help to differentiate between BOLD pre-response activation associated with processes related to response preparation or initiation and activation associated with post-response outcome integration processes. The foundation to both studies of this work was an experimental paradigm of instructed S-R-O learning, which included a learning and a test phase. Stimuli were four abstract visual patterns that differed in each block. Each visual stimulus required a distinct manual response and was predictably followed by a distinct auditory outcome. Instructions were delivered via a “guided implementation” procedure in which the instruction was embedded within the first three successful behavioral implementation trials. In these first three trials, the visual stimulus was followed by an imperative stimulus highlighting the correct response. The guided implementation phase was followed by an unguided implementation phase where the correct response now had to be retrieved from memory. Behaviorally, the strength of acquired O-R associations can be analyzed via O-R compatibility effects measured in a subsequent outcome-priming test phase (Greenwald, 1970). In this test phase a previously learned outcome becomes an imperative stimulus that requires either the response, which produced that outcome in the preceding learning phase (O-R compatible), or a response, which produced a different outcome (O-R incompatible). The experimental design was embedded into an EEG recording setup in study 1 while study 2 comprised a simultaneous EEG-fMRI recording setup in which EEG scalp potentials were continuously recorded during the experimental session inside the MR scanner bore. Study 1 revealed various ERP markers correlated with outcome response learning. An ERP post-response anterior negativity following auditory outcomes was increasingly attenuated as a function of the acquired association strength. This suggests that previously reported action-induced sensory attenuation effects under extensively trained free choice conditions can be established within few repetitions of specific R-O pairings under forced choice conditions. Furthermore, an even more rapid development of a post-response but pre-outcome fronto-central positivity, which was reduced for high R-O learners, might indicate the rapid deployment of preparatory attention towards predictable outcomes. Finally, the study identified a learning-related stimulus-locked activity modulation within the visual P1-N1 latency range, which was thought to reflect the multi-sensory integration of the perceived antecedent visual stimulus with the anticipated auditory outcome. In general, study 2 was only partially able to replicate the EEG activity dynamics related to the formation of bidirectional R-O associations that were observed in study 1. Primarily, it was able to confirm the modulation in EEG negativity in the visual P1-N1 latency range over the learning course. The EEG-informed analysis revealed that learning-related modulations of the P1-N1 complex are functionally coupled to activation in the orbitofrontal cortex (OFC). More specifically, growing attenuation of the EEG negativity increase from early to late SRO repetition levels in high R-O learners was associated with an increase in activation in the OFC. An additional exploratory EEG analysis identified a recurring post outcome effect at central electrode sites expressed in a stronger negativity in late compared to early learning stages. This effect was present in both studies and showed no correlation with any of the behavioral markers of learning. The EEG-informed fMRI analysis resulted in a pattern of distinct functional couplings of this parameter with different brain regions, each correlated with different behavioral markers of S-R-O learning. First of all, increased coupling between the late EEG negativity and activation in the supplementary motor area (SMA) was positively correlated with the O-R compatibility effect. Thus, high R-O learners exhibited a stronger coupling than low R-O learners. Secondly, increased couplings between the late EEG negativity and activation in the somatosensory cortex as well as the dorsal caudate, on the other hand, were positively correlated with individual reaction time differences between early and late stages of learning. Regarding activation patterns prior to the behavioral response the results indicate that the OFC could serve as a (multimodal) hub for integrating stimulus information and information about its associated outcome in an early pre-stage of action selection and initiation. Learnt S-O contingencies would facilitate initiating the motor program of the action of choice. Hence, the earlier an outcome is anticipated (based on stimulus outcome associations), the better it will be associated with its response, eventually leading to stronger O-R compatibility effects later on. Thus, one could speculate that increased activation in response to S-R-O mappings possibly embodies a marker for the ongoing transition from mere stimulus-based behavior to a goal-directed behavior throughout the learning course. Post-response brain activation revealed a seemingly two-fold feedback integration stream of O-R contingencies. On one hand the SMA seems to be engaged in bidirectional encoding processes of O-R associations. The results promote the general idea that the SMA is involved in the acquisition of goal-directed behavior (Elsner et al., 2002; Melcher, Weidema, Eenshuistra, Hommel, & Gruber, 2008; Melcher et al., 2013). Together with prior research (Frimmel, Wolfensteller, Mohr, & Ruge, 2016) this notion can be generalized not only to extensive learning phases but also to learning tasks in which goal-directed behavior is acquired in only few practice trials. However, there is an ongoing debate on whether SMA activation can be clearly linked to sub-processes prior or subsequent to an agent’s action (Nachev, Kennard, & Husain, 2008). The results of this work provide additional evidence favoring an involvement of the SMA only following a performed action in response to an imperative stimulus and even more, subsequent to the perception of its ensuing effect. This may give rise to the interpretation that the SMA is associated with linking the motor program of the performed action to the sensory program of the perceived effect, hence establishing and strengthening O-R contingencies. Furthermore, the analysis identified an increased coupling of a late negativity in the EEG signal and activation in the dorsal parts of the caudate as well as the somatosensory cortex. The dorsal caudate has not particularly been brought into connection with O-R learning so far. I speculate that the coupling effect in this part of the caudate reflects an ongoing process of an early automatization of the acquired behavior. It has already be shown in a similar paradigm that behavior can be automatized within only few repetitions of novel instructed S-R mappings (Mohr et al., 2016).:Table of contents Table of contents II List of Figures IV List of Tables VI List of Abbreviations VII 1 Summary 1 1.1 Introduction 1 1.2 Study Objectives 2 1.3 Methods 3 1.4 Results 4 1.5 Discussion 4 2 Theoretical Background 7 2.1 Introduction 7 2.2 Theories of acquiring goal-directed behavior 9 2.2.1 Instrumental learning 9 2.2.1.1 Behavioral aspects 9 2.2.1.2 Neurophysiological correlates 14 2.2.2 Acquisition of goal-directed behavior according to ideomotor theory 16 2.2.2.1 Behavioral aspects 16 2.2.2.2 Neurophysiological correlates 22 2.3 Summary 25 2.4 Methodological background 26 2.4.1 Electroencephalography (EEG) 26 2.4.2 Functional magnetic resonance imaging (fMRI) 28 2.4.3 Co-registered EEG-fMRI 29 3 General objectives and research questions 34 4 Study 1 – Learning-related brain-electrical activity dynamics associated with the subsequent impact of learnt action-outcome associations 36 4.1 Introduction 36 4.2 Methods 39 4.3 Results 47 4.4 Discussion 60 5 Study 2 - Within trial distinction of O-R learning-related BOLD activity with the means of co-registered EEG information 64 5.1 Introduction 64 5.2 Methods 66 5.3 Results 86 5.4 Discussion 101 6 Concluding general discussion 109 6.1 Brief assessment of study objectives 109 6.2 Novel insights into rapid instruction based S-R-O learning? 109 6.2.1 Early stimulus outcome information retrieval indicates the transition from stimulus based behavior to goal-directed action 110 6.2.2 Post-response encoding and consolidation of O-R contingencies enables goal-directedness of behavior 112 6.3 Critical reflection of the methodology and outlook 116 6.3.1 Strengths and limitations of this work 116 6.3.2 Data quality assessment 117 6.3.3 A common neural foundation for EEG and fMRI? 119 6.3.4 How can co-registered EEG-fMRI contribute to a better understanding of the human brain? 121 6.4 General Conclusion 123 7 References 124 Danksagung Erklärun

Contributions en optimisation topologique : extension de la méthode adjointe et applications au traitement d'images

De nos jours, l'optimisation topologique a été largement étudiée en optimisation de structure, problème majeur en conception de systèmes mécaniques pour l'industrie et dans les problèmes inverses avec la détection de défauts et d'inclusions. Ce travail se concentre sur les approches de dérivées topologiques et propose une généralisation plus flexible de cette méthode rendant possible l'investigation de nouvelles applications. Dans une première partie, nous étudions des problèmes classiques en traitement d'images (restauration, inpainting), et exposons une formulation commune à ces problèmes. Nous nous concentrons sur la diffusion anisotrope et considérons un nouveau problème : la super-résolution. Notre approche semble meilleure comparée aux autres méthodes. L'utilisation des dérivées topologiques souffre d'inconvénients : elle est limitée à des problèmes simples, nous ne savons pas comment remplir des trous ... Dans une seconde partie, une nouvelle méthode visant à surmonter ces difficultés est présentée. Cette approche, nommée voûte numérique, est une extension de la méthode adjointe. Ce nouvel outil nous permet de considérer de nouveaux champs d'application et de réaliser de nouvelles investigations théoriques dans le domaine des dérivées topologiques.Nowadays, topology optimization has been extensively studied in structural optimization which is a major interest in the design of mechanical systems in the industry and in inverse problems with the detection of defects or inclusions. This work focuses on the topological derivative approach and proposes a more flexible generalization of this method making it possible to address new applications. In a first part, we study classical image processing problems (restoration, inpainting), and give a common framework to theses problems. We focus on anisotropic diffusion and consider a new problem: super-resolution. Our approach seems to be powerful in comparison with other methods. Topological derivative method has some drawbacks: it is limited to simple problems, we do not know how to fill holes, ... In a second part, to overcome these difficulties, an extension of the adjoint method is presented. Named the numerical vault, it allows us to consider new fields of applications and to explore new theoretical investigations in the area of topological derivative