Hansen, Lars Kai

Hansen, Sofie Therese

We propose the following generalization of the Variational Garrote for sequential EEG imaging: A Markov prior to promote sparse, but temporally smooth source dynamics. We derive a set of modied Variational Garrote updates and analyze the role of the prior's hyperparameters. An experimental evaluation is given in simulated data and in a benchmark EEG data set

Online Research Database In Technology

                       General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.  • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal   If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.     Downloaded from orbit.dtu.dk on: Dec 20, 2017EEG Sequence Imaging: A Markov Prior for the Variational GarroteHansen, Sofie Therese; Hansen, Lars KaiPublished in:Proceedings of the 3rd NIPS Workshop on Machine Learning and Interpretation in Neuroimaging 2013Publication date:2013Link back to DTU OrbitCitation (APA):Hansen, S. T., & Hansen, L. K. (2013). EEG Sequence Imaging: A Markov Prior for the Variational Garrote. InProceedings of the 3rd NIPS Workshop on Machine Learning and Interpretation in Neuroimaging 2013EEG Sequence Imaging:A Markov Prior for the Variational GarroteSofie Therese Hansen and Lars Kai HansenSection for Cognitive Systems, DTU Compute, Technical University of Denmark,Kgs. Lyngby, Denmarksofha@dtu.dk and lkai@dtu.dk, home page: www.compute.dtu.dkAbstract. We propose the following generalization of the VariationalGarrote for sequential EEG imaging: A Markov prior to promote sparse,but temporally smooth source dynamics. We derive a set of modifiedVariational Garrote updates and analyze the role of the prior’s hyperpa-rameters. An experimental evaluation is given in simulated data and ina benchmark EEG data set.Keywords: Source reconstruction, Bayesian inference, the VariationalGarrote, EEG, sparse Bayesian learning.1 Sparse Sequence ReconstructionThe dynamics of electroencephalographic (EEG) sources is an active researchfield, see. e.g., [1–6]. We are interested in the spatio-temporal source distribu-tion under well-defined brain activation. The contribution of the present paperis to expand upon a promising new algorithm, the so-called Variational Garrote(VG), first proposed in [7] and recently applied to EEG brain imaging in [8] andexpanded to a fixed sparsity temporal model in [9]. Our goal and main contri-bution in this presentation is to relax the fixed sparsity model by introducing aflexible Markov prior forming a new algorithm that we refer to as MarkoVG.The forward relation between cortical sources and electrode potentials islinear and we will here assume the forward propagation model known, althoughattempts have been made of estimating it from data, see e.g. [5]. Using a ’spikeand slab’ like representation, the linear relation between observations acrosstime, Y ∈ RK×T , the forward model A ∈ RK×N and the source matrix X ∈RN×T is modified in the Variational Garrote (VG) [7] asYk,t =N∑n=1Ak,nXn,t + Ek,tV G=⇒ Yk,t =N∑n=1Ak,nSn,tXn,t + Ek,t , (1)where Sn,t is a 0, 1 binary variable controlling the spatio-temporal support ofbrain activity Xn,t (i.e., at the dipolar location n at time t). The variable Ek,tis assumed to be i.i.d. normal noise with zero mean and unknown variance 1/β.Aiming for temporally smooth and spatially sparse configurations we assign asimple Markov model prior for the binary variables of a specific dipole locationn, represented by a transition matrix Γi,j = Prob(Sn,t = i|Sn,t−1 = j) with twofree parameters, e.g., of the formΓ =[Γ0,0 Γ0,1Γ1,0 Γ1,1]=[1− Γ1,0 Γ0,1Γ1,0 1− Γ0,1]. (2)The stationary distribution of Γ is given by (Γ0,1/(Γ1,0+Γ0,1), Γ1,0/(Γ1,0+Γ0,1)),thus the ratioΓ0,1Γ1,0controls the prior sparsity.The VG approach is based on approximate variational inference. Here we de-rive the modified update rules for the variational approximation. With uniformpriors on X and β we obtain a variational free energy F (q,X, β) which is mini-mized to obtain the optimal variational distribution q, the source estimates X,and the noise parameter β. To reduce computation we use the dual formulation[7] introducing Zk,t =∑nAk,nMn,tXn,t and Lagrange multipliers λk,tF =−KT2logβ2pi+β2∑t,k(Yk,t − Zk,t)2 + Kβ2∑t,nMn,t(1−Mn,t)X2n,tχn,n (3)−∑n,t[Mn,t logΓ1,0Γ0,0+Mn,t−1 logΓ0,1Γ0,0+ (Mn,tMn,t−1) logΓ0,0Γ1,1Γ0,1Γ1,0](4)+NT log1Γ0,0+∑n,t[Mn,t log(Mn,t) + (1−Mn,t) log(1−Mn,t)] (5)+∑t,kλk,t(Zk,t −∑nAk,nMn,tXn,t), (6)where χ = ATA/K. The variational estimates satisfy the following equation set(with σ(x) = (1 + exp(−x))−1)Xn,t =1Kβ1(1−Mn,t)χn,n∑kλk,tAk,n , Zk,t = Yk,t − 1βλk,t , (7)β =1TK∑t,k,cλk,tλc,tCk,c,t, Ck,c,t≡δk,c+ 1K∑nMn,t(1−Mn,t)χn,nAk,nAc,n (8)λc,t = βYˆk,t with∑cCk,c,tYˆk,t = Yk,t , (9)Mn,t = σ(Kβ2χn,nX2n,t + γ1 + γ2 (Mn,t−1 +Mn,t+1)), (10)solved by iteration. Here the combination of Markov parameters; γ1 = log(Γ1,0Γ0,1Γ 20,0),γ2 = log(Γ0,0Γ1,1Γ0,1Γ1,0)determine the sparsity and smoothness of the solution: Theparameter γ2 is seen to control the degree of temporal smoothness, while γ1 cor-responds to Kappen’s sparsity control parameter (with negative values favoringsparse solutions). If γ2 = 0 we recover the original VG algorithm.2 Experimental EvaluationIn the following a simulation example will serve to illustrate the properties ofMarkoVG. We simulate a data set of size N = 150, K = 25 and T = 25. Theweight distribution is controlled as three active sources with sine wave like timecourses active at different time windows. The data are corrupted by normaladditive noise (SNR= 5 dB). An example of the generated sources can be seenin the left panel of Fig. 1, while the right panel shows the estimated sourcesusing MarkoVG. Here the parameter Γ0,1 = 0.02 is fixed, while the ratioΓ0,1Γ1,0is estimated through three-fold cross-validation among possible values rangingfrom 10−5 to 10, in 50 steps. For each step 25 iterations are performed.We find that the relevant weights are recovered while one irrelevant weight(at time t = 11) is mistakenly judged as being relevant. Swift convergence of Mn,tis seen in Fig. 2 for both the ’true’ locations (depicted in blue, green and redcorresponding to Fig. 1) and for one ’false’ location, chosen as the location withnon-zero activity at time t = 11 (depicted in gray). The color indicates the valueof Mn,t; brightest or darkest indicate Mn,t=0 or 1 (minimal or maximal marginalposterior probability of activation in location n). In Fig. 2, MarkoVG is furtherevaluated using the source retrieval score (F1-measure) and the mean squarederror (MSE) on the estimated sources. A total of 100 randomly generated datasets similar to that shown in Fig. 1 are drawn. The performance measures are seento converge quickly towards their optimum. Note that the iterations minimizethe Free energy whose optimum is not simply related to source retrieval (F1) norMSE. Varying both γ1 and γ2 in a grid to inspect the influence of the smoothnessand sparsity parameters on the F1-measure and MSE we obtain Fig. 3. Note ’hot’colors indicate high values (better performance in F1-measure and worse for theMSE). A (γ1,γ2)-region exists with high source retrieval ability and low MSE.5 10 15 20 25−1−0.500.51Time samples, tFeature values5 10 15 20 25−1−0.500.51Time samples, tFeature valuesFig. 1: The source time functions, true (left) and estimated (right). The active sourcesare in blue (n=1), green (n=2) and red (n=3) respectively, and the non-active sources(n=4:150) are all represented as black dotted lines. Three-fold cross-validation is usedto find an optimum level of Γ0,1/Γ1,0. For each investigated level 25 iterations areapplied. SNR= 5 dB.0 10 20 30 40 500.50.60.70.80.91Source retrieval index, F1No. of iterations00.511.52x 10−3MSE on estimated featuresFig. 2: Left: Convergence of Mn,t during 25 iterations for the 3 planted active sources(blue, green and red) and the strongest false source (gray scale). Example correspondsto that shown in Fig. 1. Right: Evaluation of MarkoVG with 100 repetitions of datasets similar to that shown in Fig. 1, all with SNR around 5 dB. Performance shown asmean ± standard deviation of source retrieval F1-measure and MSE on the weights.Temporal smoothness, γ2Sparsity, γ 1  −10 0 10 20−25−20−15−10−500.20.40.60.81Temporal smoothness, γ2Sparsity, γ 1  −10 0 10 20−25−20−15−10−500.20.40.60.81Fig. 3: Search for optimal smoothness and sparsity. Left: mean F1, right: mean MSE,across 30 repetitions of data sets similar to Fig. 1(left). For each parameter combination25 iterations are applied. SNR= 5 dB.The existence of this region indicates that the MarkovVG representation indeedallows us to find sparse sources with limited bias on the source magnitudes.The performance of MarkoVG is next examined on benchmark EEG datawhich is part of a multi-modal face response data set, available through theSPM website1. The data used here are collected from a single subject and usedto demonstrate the modulation in brain activity when seeing faces vs. scrambledfaces. The EEG signals were acquired with a 128 channel ActiveTwo system anddownsampled from 2048 Hz to 200 Hz and averaged across trials, more specifi-cations on the setup can be found in the SPM manual [10]. For reference weshow the results using Friston et al.’s multiple sparse priors model (MSP) [3] as1 http://www.fil.ion.ucl.ac.uk/spm/data/mmfaces/Fig. 4: The time evolution of the two strongest sources found in face response. Inthe inset of the cortex (posterior view) arrows point to the corresponding locations180 ms post-stimuli. Left: Sources obtained using SPM’s multiple sparse priors method.The color coding indicates that the solution obtained is rather dense. Right: Sourcesobtained using Zhang et al.’s T-MSBL method. As in MSP, estimate is rather dense.Fig. 5: The time evolution of the two strongest sources found in face response. Inthe inset of the cortex (posterior view) arrows point to the corresponding locations180 ms post-stimuli. Left: Solution obtained using the temporally expanded VG (teVG)scheme. The color coding indicates more sparse solution than that obtained by MSPand T-MSBL. Right: Estimate by MarkoVG. As teVG, MarkoVG is also spatially verysparse. The time courses are, however, now sparse, thus the difference between faceand scrambled face processing has been localized to focused shorter time intervals.adopted in the SPM software and the result of Zhang et al.’s multiple measure-ments vector sparse Bayesian learning model, T-MSBL [4] in Fig. 4 left and rightpanels, respectively. These solutions should be compared to two versions of theVG: Time expanded VG (teVG) [9] and MarkoVG, both shown in Fig. 5. TheMSP and T-MSBL estimates of the sources responsible for the difference betweenface and scrambled faces are very smooth in time, in fact resembling standardERPs. This is also the case for the window-wise constant support model teVG,c.f., Fig. 5 (left), while the more flexible support recovery method MarkoVGfinds a smaller number of active sites for the difference signal in Fig. 5 (right).Also we note that teVG and MarkoVG in general find spatially sparser solutions,viz., the more extended gray areas in Fig. 4.3 ConclusionWe have proposed MarkoVG assigning a Markov prior for promotion of tem-porally smooth sources in the Variational Garrote. We derived the modifiedvariational update rules and identified the role of the Markov prior parame-ters. We showed that MarkoVG converges fast, as VG also does, and we foundthat sources are reliably estimated both in terms of location and source strengthmean square error. In a benchmark EEG data set we showed that MarkoVG pro-duced more focused activation than multiple sparse priors and temporal sparseBayesian learning, both of which are more similar to our earlier VG generaliza-tion, teVG, which assumes constant temporal support in specified windows.AcknowledgementWe thank the Lundbeck Foundation for support throughthe Center for Integrated Molecular Brain Imaging (CIMBI).References1. Gorodnitsky, I. F., Rao, B. D., Sparse signal reconstruction from limited data usingFOCUSS: a reweighted minimum norm algorithm. IEEE Transactions on SignalProcessing, 45(3), pp. 600-616 (1997).2. Wipf, D. P., Rao, B. D., An Empirical Bayesian Strategy for Solving the Simultane-ous Sparse Approximation Problem. IEEE Transactions on Signal Processing, 55(7),pp. 3704-3716 (2007).3. Friston, K. J., Harrison, L., Daunizeau, J., Kiebel, S. J., Phillips, C., Trujillo-Barreto, N., Henson, R. N., et al.: . Multiple sparse priors for the M/EEG inverseproblem, NeuroImage 39(3), pp. 1104-1120 (2008).4. Zhang, Z., Rao, B.D.: Sparse Signal Recovery with Temporally Correlated SourceVectors Using Sparse Bayesian Learning, IEEE Journal of Selected Topics in SignalProcessing, Special Issue on Adaptive Sparse Representation of Data and Applica-tions in Signal and Image Processing 5(5), pp. 912–926 (2011).5. Stahlhut, C., Mørup, M., Winther, O., Hansen, L.K.. Simultaneous EEG source andforward model reconstruction (sofomore) using a hierarchical bayesian approach.Journal of Signal Processing Systems, 65(3), pp. 431-444 (2011).6. Montoya-Martinez, J., Artes-Rodriguez, A., Hansen, L.K., Pontil, M. Structuredsparsity regularization approach to the EEG inverse problem. In Proc. 3rd Interna-tional Workshop on Cognitive Information Processing (CIP 2012), IEEE (2012).7. Kappen, H.:The Variational Garrote. arXiv preprint arXiv:1109.0486. Published athttp://arxiv.org/abs/1109.0486 (2011).8. Hansen, S.T., Stahlhut, C., Hansen, L.K.: Sparse Source EEG Imaging with theVariational Garrote, 3rd International Workshop on Pattern Recognition in Neu-roimaging (PRNI), pp. 106–109 (2013).9. Hansen, S.T., Stahlhut, C., Hansen, L.K.: Expansion of the Variational Garrote to aMultiple Measurement Vectors Model, Scandinavian Conference on AI 2013 (2013).10. Ashburner, J., Chen, C.-C., Moran, R., Henson, R. N., Glauche, V., Phillips, C.:SPM8 Manual, Available at http://www.fil.ion.ucl.ac.uk/spm/doc/ (2012).

EEG Sequence Imaging: A Markov Prior for the Variational Garrote

http://orbit.dtu.dk/ws/files/72764421/MLINIfinal.pdf

EEG Sequence Imaging: A Markov Prior for the Variational Garrote

Abstract

Similar works

Full text

Available Versions

Online Research Database In Technology