International audienceMore and more eﬀort is done in BCI research to improve its usability for patients, with respect to its communication speed and transmission accuracy. In this contribution, we ex- periment with BCI speller based on P300 evoked potential. More precisely, the typical form of event-related potential (ERP) inspires us to devise classiﬁcation methods based on the simi- larity/dissimilarity in the time domain between single trials and one or several estimated ERP templates derived from sub ject recordings. The reliable estimation of template is diﬃcult in a single trial due to the low signal-to-noise ratio (SNR) of electroencephalographic (EEG) signals. We ﬁrst explicitly estimate the template using several averaging techniques: point- to-point averaging, cross-correlation alignment and dynamic time warping. Then we inexplic- itly estimate several ERP templates using learning vector quantization algorithm combined with an extreme learning machine. Finally classiﬁcation is realized based on the similar- ity/dissimilarity between the single trials and the template. Simulation is carried out using a BCI competition III data set acquired with the P300 speller paradigm. The experiments show that template-based classiﬁers can also obtain high accuracy

Bougrain, Laurent

Liang, Nanying

INRIA a CCSD electronic archive server

HAL Id: inria-00337070https://hal.inria.fr/inria-00337070Submitted on 6 Nov 2008HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.Averaging techniques for single-trial analysis of oddballevent-related potentialsNanying Liang, Laurent BougrainTo cite this version:Nanying Liang, Laurent Bougrain. Averaging techniques for single-trial analysis of oddball event-related potentials. 4th International Brain-Computer Interface workshop, Sep 2008, Graz, Austria.￿inria-00337070￿Averaging techniques for single-trial analysisof oddball event-related potentialsNanying Liang, Laurent BougrainINRIA Nancy - Grand Est, Francewww.loria.fr/~liangnnaAbstractMore and more effort is done in BCI research to improve its usability for patients, withrespect to its communication speed and transmission accuracy. In this contribution, we ex-periment with BCI speller based on P300 evoked potential. More precisely, the typical form ofevent-related potential (ERP) inspires us to devise classification methods based on the simi-larity/dissimilarity in the time domain between single trials and one or several estimated ERPtemplates derived from subject recordings. The reliable estimation of template is difficult ina single trial due to the low signal-to-noise ratio (SNR) of electroencephalographic (EEG)signals. We first explicitly estimate the template using several averaging techniques: point-to-point averaging, cross-correlation alignment and dynamic time warping. Then we inexplic-itly estimate several ERP templates using learning vector quantization algorithm combinedwith an extreme learning machine. Finally classification is realized based on the similar-ity/dissimilarity between the single trials and the template. Simulation is carried out usinga BCI competition III data set acquired with the P300 speller paradigm. The experimentsshow that template-based classifiers can also obtain high accuracy.1 IntroductionBrain-computer interface (BCI) system is a potentially powerful new communication and controloption for those with severe motor disabilities. BCI system translates brain activity into commandsfor a computer or other devices. Electroencephalography (EEG) is the most studied potential non-invasive interface mainly due to its fine temporal resolution, ease of use, portability and low set-up cost. Unfortunately, non-invasive implants produce a noisy signal because the skull dampenssignals. Another substantial barrier to use EEG as a brain-computer interface is the extensivetraining required before users can work the technology.Oddball paradigms are used by brain-computer interfaces to generate event-related potential(ERP) on targets selected by the user. The well-known P300 speller is based on this principle [1].The main problem is to be able to detect ERP in a noisy electroencephalographic signal recordedon human scalp. Literature proposes a set of methods to detect ERPs based on averaging. Linearor nonlinear alignment is used to deal with the variable latencies in ERPs. A good alignmentallows to obtain a better averaged ERP response, named “ERP template” here, and then to beable to detect ERP faster.Section 2 of this paper presents a set of methods of template-based classifiers. Section 3 presentsthe results of these techniques on a well-known third BCI2000 competition data set acquired froma P300 speller. In section 4, we conduct the conclusion and give recommendation for future works.2 MethodsMethods design for ERP detection use features extracted from the responses in the time domain.To extract features from the frequency domain after any transformation is difficult in this situation1because ERPs are short-time events with a local peak. Here, we discuss two types of methodsusing one or several templates estimated explicitly and inexplicitly in the time domain respectively.2.1 Classifiers based one ERP templateIn oddball paradigm, one deviant rare stimulus is presented to subjects amongst a train of standardones. Only the deviant stimulus induces an ERP. The most common evoked potential used in BCIis the P300. Its main characteristic is a positif latency that occurs around 300ms after the stimuluspresentation. This leads us to derive a set of classifiers. All of them first explicitly estimate oneERP template by averaging the ERP responses in the time domain, and then use the distancebetween the response and the ERP template as the discriminant criterion.point-to-point averaging (P2P) classifier: P2P classifier discriminates the ERP responsesfrom non-ERP responses in a straightforward way: first it estimates one ERP template by simplyaveraging the measurements of ERP responses point-to-point, and then calculates the Euclideandistance between responses and the ERP template. For the P300 speller, the ERP responsecorresponds with the one producing the minimum distance within a set of column or row responses.Then the other responses are considered as non-ERP responses.P2P classifier is simple for implementation. However the latency of the components of ERPvaries with the external factors (such as target-to-target interval). Simply averaging the measure-ments point-to-point could blur the component. We then seek the solution of alignment beforeaveraging. Such alignment includes linear alignment (such as, cross-correlation alignment) andnonlinear alignment (such as, dynamic time warping).cross-correlation averaging (CC) classifier: Linear alignments can be done by shifting themeasurements before the point-to-point averaging. One of the well-known linear alignments iscross-correlation. The best shift of one sequence in order to align it to the other ones is given bythe one producing the highest correlation [2, 3]. CC classifier first searches the best shift for pairsof ERP responses before averaging them point-to-point to obtain one template. Then it calculatesthe cross-correlation between the testing response and the ERP template. The predicted ERPresponse corresponds with the one giving the maximum correlation value with the template. Suchscheme accommodates variable linear latency. However, if one looks the ERP measurements moreclosely, it is found that the latency of the ERP component is nonlinear. In this case, the nonlinearalignment methods are better than the linear ones for template estimation.dynamic time warping averaging (DTW) classifier: Dynamic Time warping distance iswidely used in speech domain. It is able to deal with compression/expansion by comparing thedistance of each point of the first sequence to every point of the second one using the informationof the amplitude and/or the first derivative of amplitude of the signals [4]. Hence, DTW classifierfirst finds out the best nonlinear alignment path giving the closest distance, and then averages theERP responses according to the alignment path to obtain one ERP template [5, 6, 7]. Finally, itrecognizes the ERP responses as the one producing the minimum DTW distance.2.2 Methods with several templates of ERP & non-ERPOn one hand, considering the complexity of ERP responses, one template could be insufficientto capture the variability of ERP components. On the other hand, one would like to introducenon-ERP templates into decision making process done by the classifiers. Clustering methods canproduce several templates. The aim of clustering methods is to split a set of patterns into clustersas homogeneous as possible, i.e., patterns are gathered by similarity. A classical rule to assign anew pattern to a cluster is based on distances between the input pattern and templates. LVQ isthe most famous algorithm in this category.2learning vector quantization (LVQ): The first version of the learning vector quantizationalgorithm proposed by Kohonen [8] is a supervised clustering method designed for classification.Input weights (IW) of connections linking inputs to each neuron is a vector m with the samedimension than input x. Each neuron is assigned to a class according to the pre-defined layerweights (LW). A classic competitive step is used to determine which neuron is the closest oneto the current input with the formula: c = arg mini{||x − mi||}, and only the winner neuron isupdated according to the following rules:mc(t + 1) = mc(t) + α(t)[x(t) − mc(t)] if x and mc belong to the same classmc(t + 1) = mc(t) − α(t)[x(t) − mc(t)] if x and mc belong to different classeswhere α is learning rate. To apply this algorithm, one needs to decide the value of learning rateand the number of neurons assigned to each class. Further considering the multichannel recordingof EEG measurements, if one simply merges features from each channel, the dimension of IWincreases with the number of channels, and it thus causes memory problem in simulation. Wethen consider multichannel LVQ.multichannel learning vector quantization (mLVQ): For multichannel LVQs, one LVQmodel is created for each channel, and trained according to the above rules to update IWs. Thus,there are several ERP and non-ERP templates generated for each channels. In order to combinethe templates in an optimal way, we further introduce an update scheme for LWs: LW = (−||X −IW ||)†T , where X is the input matrix and T is the target matrix. The symbol † represents thepseudo-inverse operator. Such a solution of LW is known as minimum norm least square solution.It is originally found being used in extreme learning machine (ELM) in [9].3 Results3.1 Wadsworth BCI data setThe results presented here are based on the P300 speller data set from the BCI competition III[10] 1. The data set contains training and testing EEG data from two subjects. There are 85letters for training and 100 letters for testing. For each letter, the recording consists of 15 epochs,and within each epoch, there are 12 flashings. For each epoch, a random permutation is chosen tohighlight rows and columns. There is a 6x6 grid in this application containing 26 letters, 9 digitsand one dash character. We are interested in the measurement window starting from the onsetof the flashing and till one second, which consists of 240 samples at a sampling rate of 240Hz.In case the row/column is highlighted exactly where the target letter is located, we got the ERPresponse; otherwise the non-ERP response.3.2 Experimental Results3.2.1 ERP TemplatesFigure 1 shows the comparison between the ERP templates estimated by P2P averaging, CCand DTW alignment averaging, respectively. Templates are built using the first 128 trials of thetraining set of subject A. Indeed, the algorithms using CC and DTW prefer to pair trials in abinary tree structure. The latency of the P300 component in the estimated templates by CC andDTW alignment averaging methods is different from P2P averaging. This is the effect of aligningthe component in the time domain before averaging. And we found that the template estimatedby DTW was smoother than by cross-correlation.3.2.2 Classifier ComparisonSimulation is carried out to compare the performance of the template-based classifiers using P2P,CC, DTW and LVQ. The result of linear discriminant analysis (LDA) is also provided for its1The data set is available at: http://ida.first.fraunhofer.de/projects/bci/competition_iii/3Figure 1: Comparison of ERP templates (red solid curves) and averaged background EEG activi-ties (black dot curves) for responses recorded on channel Cz using training data from subject A.From top to bottom, templates are estimated by point-to-point averaging, cross-correlation anddynamic time warping alignment averaging, respectively.simplicity and good performance in the application of BCIs. Table 1 shows the testing accuracyfor subjects A and B. The results are based on the raw measurements from channel Cz (i.e., nopre-processing is involved). We used the full training set to estimate the ERP templates for P2P,CC and DTW; while for LVQ and LDA, they are used to estimate the parameters of the models.The full testing set is used to evaluate the performance of the classifiers. From the table, we cansee that LVQ and LDA achieve similar performance and are better than the classifiers using P2P,CC, and DTW.Table 2 shows the testing accuracy for subjects A and B based on measurements of all channels.Due to limit of memory and the large size of the input data, one needs to re-sample the signals.Thus, for Table 2, we first smooth the signal using a moving averaging filter with window size of13, and then re-sample it with a down-sampling factor of 13. Further, multichannel version of LVQis used here. We find that the performance of mLVQ and LDA are similar again and both increaseas more channels are used for classification; while the performance of the explicit template-basedclassifiers using P2P, CC and DTW is bad.From the above simulation results, we found that the performance of classifiers (such as LVQ),which take account of both ERP and non-ERP templates is better than those (such as P2P,CC, and DTW) which use only one ERP template. The optimization of the combination of thetemplates from different channels through assigning different weights is even more critical forimproving the accuracy.4 Conclusion and further workBCI applications based on P300 ERP detection, such as the P3speller, need statistical or machinelearning techniques with temporal features. ¿From a recording point of view, ERPs are graphic-elements. They have a specific waveform clearly different from EEG background activity. Thus,efficient modeling of the waveform can help to detect ERPs. Averaging techniques and clusteringmethods allow extracting one or several ERP templates. Averaging techniques using point-to-point averaging, cross-correlation and dynamic time warping are not efficient. They produceonly one ERP template and suffer from two difficulties: first, responses are too noisy to easilydistinguish ERP from non-ERP responses; second, according to the previous remark, they do not4MethodSubject Epoch P2P CC DTW LVQ LDA05 7% 8% 15% 17% 18%A 10 5% 26% 28% 28% 30%15 3% 21% 39% 43% 41%05 2% 6% 6% 8% 15%B 10 5% 9% 10% 13% 19%15 3% 16% 15% 21% 26%Table 1: Experimental results based on channel Cz for subjects A and B. The percentage of correctclassified letters contained in the testing set is shown regarding using the first 5 epochs, 10 epochsand all epochs.MethodSubject Epoch P2P CC DTW mLVQ LDA05 9% 7% 7% 47% 45%A 10 13% 6% 12% 77% 78%15 14% 7% 15% 87% 88%05 3% 2% 4% 72% 76%B 10 3% 3% 3% 91% 92%15 4% 4% 3% 96% 96%Table 2: Experimental results based on all channels for subjects A and B. The percentage ofcorrect classified letters contained in the testing set is shown regarding using the first 5 epochs,10 epochs and all epochs.take into account the specificities of non-ERP responses to catch small differences between noisyERP and non-ERP responses. The first difficulty exists for any classification method. But, ourmethod mLVQ obtains similar results as LDA which is a reference technique for the P3Speller. Itis because mLVQ takes into account the second remark. Thus, template-based classifiers can alsoobtain good results. Moreover, this approach can extract knowledge about the ERP waveform ofpatients.In this article, we also evaluated the promising dynamic time warping distance. This kindof similarity measure is widely used in speech domain and align efficiently two sequences even ifthey differ each other from compression and/or dilatation on some segments. However, from ouropinion, DTW suffers from the low SNR. The real EEG signals used in our simulation are toonoisy. Many artificial peaks appear in the recording and thus it is difficult to obtain a strongalignment. In order to mitigate this difficulty, one may reduce the window of investigation to[250ms, 650ms] and constrain the window size of permitted distortion to short time. It shouldhelp to focus the attention on the critical zone.In practice, it is also very interesting to reduce the number of channels in order to equip thepatient with a compact system with only several necessary electrodes. Thus, channel selection willbe investigated in a further work.References[1] L.A. Farwell and E. Donchin. Talking off the top of your head: toward a mental prosthesisutilizing event-related brain potentials. Electroencephalography and Clinical Neurophysiology,70(6):510–523, 1988.[2] C. D. Woody. Characterization of an adaptive filter for the analysis of variable latencyneuroelectric signals. Med. Biol. Eng., 5:539–553, 1967.5[3] D.G. Wastell. Statistical detection of individual evoked responses: an evaluation of woody’sadaptative filter. Electroencephalography and Clinical Neurophysiology, 42:835–839, 1977.[4] E. Keogh and M. Pazzani. Derivative dynamic time warping, 2001.[5] T. Picton, M. Hunt, R. Mowrey, R. Rodriguez, and J. Maru. Evaluation of brain-stemauditory evoked potentials using dynamic time warping. Electroencephalography and ClinicalNeurophysiology, 71(3):212–225, 1988.[6] K. Wang, H. Begleiter, and B. Porjesz. Warp-averaging event-related potentials. ClinicalNeurophysiology, 112(10):1917–1924, 2001.[7] S. Casarotto, A. M. Bianchi, S. Cerutti, and G. A. Chiarenza. Dynamic time warping in theanalysis of event-related potentials. IEEE Engineering in Medicine and Biology Magazine,24(1):68–77, 2005.[8] T. Kohonen. The self-organizing map. In Proceedings of the IEEE, volume 78, pages 1464–1480, 1990.[9] G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew. Extreme learning machine: A new learning schemeof feedforward neural networks. In Proceedings of International Joint Conference on NeuralNetworks, volume 2, pages 985–990, Budapest, Hungary, 2004.[10] D. J. Krusienski and G. Schalk. Wadsworth BCI Dataset (P300 Evoked Potentials). BCICompetition III Challenge, 2004.6

Averaging techniques for single-trial analysis of oddball event-related potentials

HAL-Rennes 1

https://hal.inria.fr/inria-00337070/document

Averaging techniques for single-trial analysis of oddball event-related potentials

Abstract

Similar works

Full text

Available Versions

INRIA a CCSD electronic archive server

HAL-Rennes 1