Inagaki Kiyoto

Nakayama Kenji

Kanazawa University Repository for Academic Resources

A brain computer interface based on neuralnetwork with efficient pre-processing著者 Nakayama Kenji, Inagaki Kiyotojournal orpublication titleProc. IEEE, ISPACS2006, Yonago, Japanpage range 673-676year 2006-12-01URL http://hdl.handle.net/2297/6856A Brain Computer Interface Based on NeuralNetwork with Efficient Pre-ProcessingKenji Nakayama Kiyoto Inagaki∗Graduate School of Natural Science and Technology, Kanazawa Univ.Kakuma-machi, Kanazawa, 920-1192, JapanTel: +81-76-234-4896, Fax: +81-76-234-4900E-mail: nakayama@t.kanazawa-u.ac.jpAbstract— Brain Computer Interface (BCI) is one of hopefulinterface technologies between human and machine. However,brain waves are very weak and there exist many kinds of noises.Therefore, what kinds of features are useful, how to extractthe useful features, how to suppress noises, and so on are veryimportant.On the other hand, neural networks are very useful technologyfor pattern classification. Especially, multilayer neural networkstrained through the error back-propagation algorithm have beenwidely used in a wide variety of field.In this paper, the neural network is applied to the BCI.Amplitude of the FFT of the brain waves are used for the inputdata. Several kinds of techniques are introduced in this paper.Segmentation along the time axis for fast response, nonlinearnormalization for emphasizing important information with smallmagnitude, averaging samples of the brain waves for suppressingnoise effects and reduction in the number of the samples forachieving a small size network, and so on are newly introduced.Simulation was carried out by using the brain waves, whichare available from the web site of Colorado state university.The number of mental tasks is five. Ten data sets for eachmental task are prepared. Among them, 9 data sets are used fortraining, and the rest one data set is used for testing. Selectionof the one data set for testing is changed and accuracy of thecorrect classifications are averaged over the possible selections.Approximately, 80% of correct classification of the brain wavesis obtained, which is higher than the conventional.I. INTRODUCTIONNowadays, several kinds of interfaces between human andcomputers or machines have been proposed and developed.For persons being in a healthy condition, the keyboard andthe mouse are useful and practical interface. On the otherhand, for handicapped persons, several interface techniques,which use available organs and functions, have been studiedand developed.Among the interfaces developed for the handicapped per-sons, Brain Computer Interface (BCI) has been attractiverecently. Brain waves are first analyzed and classified. Mentaltasks, which a subject images, are estimated. Furthermore,based on the estimated mental task, computers and machinesare controlled [9].For one application, it can be expected that heavy handi-capped persons, who cannot control any parts of their ownbody, control a wheelchair, computers and other machinesthrough the BCI [6]. Furthermore, in the virtual reality (VR)technology, it may be possible to control a person states in theVR world, and to have many kinds of experiences in the VRworld. For instance, training to avoid danger situations maybe possible by using the BCI technology.Approaches to the BCI technology includes nonlinear clas-sification by using spectrum power, adaptive auto-regressivemodel and linear classification, space patterns and linearclassification, hidden Markov models, and so on [4]. Further-more, application of neural networks have been also discussed[3],[5],[7],[10],[11].In this paper, the multilayer neural network is applied tothe BCI. Especially, efficient pre-processing techniques areintroduced in order to achieve high probability of correctmental task classification. Simulation were carried out by usingthe brain waves, which are available from the web site ofColorado state university. Estimation results of the proposedmethod are compared to the conventional methods [12],[13].II. MENTAL TASKS AND BRAIN WAVE MEASUREMENTA. Mental TasksIn this paper, the brain waves, which are available fromthe web site of Colorado state university [1], are used. Thefollowing five kinds of mental tasks are used as imaging.• Baseline (B)• Multiplication (M)• Letter-composing (L)• Rotation of 3-D object (R)• Counting numbers (C)B. Brain Wave MeasurementLocation of the electrodes to measure brain waves is shownin Fig.1. Seven channels including C3, C4, P3, P4, O1, O2,EOG, are used. EOG is used for measuring movement of theeyeballs.The brain waves are measured for 10 sec and sampled by250Hz for each mental task. Therefore, 10sec × 250Hz =2, 500 samples are obtained for one channel and one mentaltask. One data set includes seven cahnnels. One example ofthe brain wave is shown in Fig.2.III. PRE-PROCESSINGA. Segmentation along Time AxisIn order to make the BCI responce fast, the brain wavemeasured during 10sec is divided into the segments with673frontbackleft rightC3 C4P3 P4O2O1Fig. 1. Location of seven channel electrodes.0 2 4 6 8 10−20−15−10−505101520time[sec]amplitudeFig. 2. Example of brain waves.0.5sec length as shown in Fig.3. The segmentation is shiftedby 0.25sec. This means the segment with 0.5sec length can beobtained every 0.25sec.{ { { ... ...0.5 sec 0.5 sec 0.5 sec 0.5 sec{Fig. 3. Segmentation of brain wave along time axis.B. Amplitude of FFT of Brain WavesWhat kinds of features should be used to classify the brainwaves and to estimate the corresponding mental task is veryimportant. In order to avoid effects of time delay, which isnot essential, the brain wave is Fourier transformed and itsamplitude is used to express feature of the brain wave. Thesegment of the brain wave with 0.5 sec length includes 125samples. The amplitude of the FFT of the segment is shownin Fig.4.−125 0 1250100200300400500frequency[Hz]−125 0 125050100150200frequency[Hz]Fig. 4. Amplitude of FFT of segment with 0.5sec length. Left: 125 samples,Right: 20 samples of averaging.C. Reduction of Samples by AveragingIn order to make the neural network compact and to reduceeffects of the noises, the FFT samples in some interval areaveraged. The averages are used for the neural network input.The number of samples is reduced to 20. The amplitude ofthe FFT for the reduced samples is also shown in Fig.4.D. Nonlinear NormalizationThe amplitude of the FFT is widely distributed. Smallsamples also contain important information for classifyingthe mental tasks. However, in the neural networks, largeinputs play an important role. If large samples do not includeimportant information, correct classification will be difficult.For this reason, the nonlinear normalization as shown in Eq.(1)and Fig.5 is employed in this paper. The small samples areexpanded and the large samples are compressed.f(x) =log(x−min+1)log(max−min+1) (1) 0 1 MIN     MAXxf(x)Fig. 5. Nonlinear normalization. Horizontal axis is input and vertical axis isoutput.E. Input of Neural NetworkSince the amplitude response is symmetrical, only the righthand side is used. Furthermore, the amplitude response ofthe seven channels are simultaneously applied to the neuralnetwork. An example of the neural network input is shown inFig.6. In this figure, the left hand side shows the input beforethe nonlinear normalization, and the right hand side is thatafter normalization.10 20 30 40 50 60 70050100150200ch1       ch2       ch3       ch4        ch5        ch6       ch710 20 30 40 50 60 7000.20.40.60.81ch1       ch2       ch3       ch4        ch5        ch6       ch7Fig. 6. Input of neural network including 7-channels. Left hand side: Beforenormalization, Right hand side: After normalization.674IV. MENTAL TASK CLASSIFICATION BY USING NEURALNETWORKA multilayer neural network having a single hidden layeris used. Activation functions used in the hidden layer and theoutput layer are sigmoid functions. The number of input nodesis 10samples×7-channels=70. Five output neurons are usedfor five mental tasks. Therefore, only one output neuron willrespond to the applied input data. In the training phase, thetarget for the output is binary, like (1, 0, 0, 0, 0). In the testingphase, the maximum output becomes the winner and the cor-responding mental task is assigned. However, when the winnerhave small value, estimation becomes incorrect. Therefore, theanswer of the neural network is rejected, that is any mental taskcannot be estimated. The error back-propagation algorithm isemployed for adjusting the connection weights.V. SIMULATION CONDITIONSA. Training and Testing Brain WavesThe brain waves with 10sec length for five mental taskswere measured 10 times. Therefore, 10 data sets are available.Among them, 9 data sets are used for training and the rest onedata set is used for testing. Five data sets are selected as thetesting data set. Thus, 5 independent trials were carried out,and classification accuracy is evaluated based on the averageover 5 trials [2].B. Probability of Correct and Error ClassificationsEstimation of the mental tasks is evaluated based on prob-ability of correct classification (Pc) and error classification(Pe).Pc =NcNt(2)Pe =NeNt(3)Nt = Nc +Ne +Nr (4)Rate =PcPc + Pe(5)In the above equations, Nc is the number of correct classifi-cations, Ne is the number of error classifications, and Nr isthe number of rejections.C. Parameters in Neural Network Learning• The number of hidden neurons: 20• A learning rate: 0.2• Initial weights: Random numbers in -0.1∼+0.1• The threshold for rejection: 0.8• The number of iterations: 5000VI. SIMULATION RESULTSA. Effects of Number of SamplesBefore the segmentation, effects of the number of samples,that is the amplitude of FFT, is investigated. The successivesamples are averaged and this average is used to express theamplitude. The number of samples is reduced from 2,500to 250, 100, 50, 20. Since, the amplitude of FFT for realsignal is symmetrical, then a half number of them is actuallyused. Figure 7 shows the learning curves for mental taskclassification. In this figure, fcorrect classification ratef, thatis Pc, is the average over five mental tasks. From this result,the number of samples of 20 is best.00.20.40.60.810 1000 2000 3000 4000 5000 20 50100250iterationcorrect classification rateFig. 7. Learning curves of mental classification for training and testing data.The number of samples is varied from 250 to 20.Table I shows the number of correct and error classificationsand their probability.TABLE INUMBER OF CORRECT AND ERROR CLASSIFICATIONS AND THEIRPROBABILITY WITH 20 SAMPLES.Mental Task B M L R C Reject Pc PeBaseline 7 0 0 0 0 3 70.0 0.0Multiplication 0 8 0 0 0 2 80.0 0.0Letter-composing 0 0 7 0 0 3 70.0 0.0Rotation 0 0 0 8 0 2 80.0 0.0Counting 0 0 0 0 9 1 90.0 0.0B. Effects of SegmentationAim of the segmentation is to make the BCI response fast.On the other hand, the length is limited to 0.5 sec for example.In this section, effects of this short length is investigated. Asdescribed in the previous section, The 10 sec length of thebrain waves is divided into 0.5 sec, which has 125 samples.The learning curves are shown in Fig.8.0.00.20.40.60.81.00 1000 2000 3000 4000 5000iterationLearningTestcorrect classification rateFig. 8. Learning curves for segmentation.Furthermore, the probability of correct and error classifica-tions and their rate, that is Pc, Pe and Rate, are shown inTabel II. The probabilies were averaged over the iterationsfrom 4,001 to 5,000. From these results, accuracy of thesegmentation is almost the same as that of using all data.675TABLE IIPROBABILITY OF CORRECT AND ERROR CLASSIFICATIONS AND THIRRATES FOR SEGMENTATION.TRaining data Testing dataPc Pe Rate Pc Pe RateSegmentation 99.7 0.1 0.99 79.7 10.5 0.88No Segmentation 100.0 0.0 1.00 78.0 0.0 1.00C. Effects of Nonlinear NormalizationEffects of the nonlinear normalization given by Eq.(1) onthe mental task classification accuracy is investigated. Forreference, linear normalization, by which the sample valuesare linearly normalized from 0 to 1, is also used for reference.The segmentation is used, and 125 samples are reduced to 20samples by averaging.The learning curves are shown in Fig.90.00.20.40.60.81.00 2000 4000 6000 8000 10000non-linearlineariterationcorrect classification rateFig. 9. Learning curves for nonlinear normalization.Furthermore, the probability of correct and error classifi-cations and their rate are shown in Table III. From theseresults, the nonlinear normalization can make convergence ofthe learning fast, and the probability can be also improved.TABLE IIIPROBABILITY OF CORRECT AND ERROR CLASSIFICATIONS AND THIRRATES FOR NONLINEAR NORMALIZATION.TRaining data Testing dataNormalization Pc Pe Rate Pc Pe RateNonlinear 99.7 0.1 0.99 79.7 10.5 0.88Linear 81.8 1.9 0.98 68.4 9.8 0.88D. Threshold for RejectionWhen the winner output has small value, the estimationby the neural network is not accurate. Therefore, the result isrejected. In this section, effects of the threshold for rejection isinvestigated. The probability of correct and error classificationsand their rates are listed in Table IV. By setting the thresholdto be low, Pc can be improved, however, Pe is also increased atthe same time. In some applications, Pe should be minimizedwhile the rejection can permitted to some extent. In thesecases, high threshold is prefered. Like this, the optimumthreshold is highly dependent on applications, to which theBCI system is applied.E. Comparison to Conventional MethodsThe same brain waves were used in [3]. The coefficientsof a 6-th order auto-regression model are used for the neuralTABLE IVPROBABILITY OF CORRECT AND ERROR CLASSIFICATIONS AND THIRRATES BY CHANGING THRESHOLD.Threshold Pc Pe Rate0.8 80.3 10.0 0.890.6 81.7 11.9 0.870.4 82.9 13.2 0.860.2 83.7 14.2 0.860.0 84.5 15.5 0.84network input. The data length of 10 sec is also divided into0.5 sec segments. The probability of correct classifications isfrom 30% to 60% for four subjects. Therefore, the proposedmethod, which employs the amplitude response of the FFT,the nonlinear normalization, the averaging and the rejection,can provide higher probability of correct mental classification.VII. CONCLUSIONA neural network has been applied to the BCI problem.In order to improve accuracy of mental task classification,several kinds of pre-processing to generate the input data ofthe neural network. Compared with the conventional methods,the probability of correct classification has been increased.REFERENCES[1] Colorado State University: http://www.cs.colostate.edu/eeg/[2] C. Anderson and Z. Sijercic, “Classification of EEG Signals from FourSubjects During Five Mental Tasks, “EANN’96, ed. by Bulsari, A.B.,Kallio, S., and Tsaptsinos, D., Systems Engineering Association, PL 34,FIN-20111 Turku 11, Finland, pp. 407-414, 1996.[3] J. R. Millan, J. Mourino, F. Babiloni, F. Cincotti, M. Varsta, and J.Heikkonen,”Local neural classifier for EEG-based recognition of metaltasks,”IEEE-INNS-ENNS Int. Joint Conf. Neural Networks, July 2000.[4] G. Pfurtscheller and C. Neuper, “ Motor imagery and direct brain-computer communication, “ Proc. IEEE, vol. 89, no. 7, pp. 1123-1134,July 2001.[5] K. R. Muller, C. W. Anderson, and G. E. Birch,” Linear and non-linearmethods for brain-computer interfaces,”IEEE Trans. Neural Sys. Rehab.Eng., vol. 11, no. 2, pp. 165-169, 2003.[6] B. Obermaier, G. R. Muller, and G. Pfurtscheller, “Virtual keyboardcontrolled by spontaneous EEG activity, “ IEEE Trans. Neural Sys. Rehab.Eng., vol. 11, no. 4, pp.422-426, Dec. 2003.[7] J. R. Millan, “ On the need for on-line learning in brain-computerinterfaces, “ Proc. IJCNN, pp. 2877-2882, 2004.[8] G. E. Fabiani, D. J. McFarland, J. R. Wolpaw, and G. Pfurtscheller,“Conversion of EEG activity into cursor movement by a brain-computerinterface (BCI), “IEEE Trans. Neural Sys. Rehab. Eng., vol. 12, no. 3,pp. 331-338, Sept. 2004.[9] G. Pfurtscheller, C. Neuper, C. Guger, W. Harkam, H. Ramoser, A.Schlo¨gl, B. Obermaier, and M. Pregenzer, “Current trends in Grazbraincomputer interface (BCI) research,” IEEE Trans. Rehab. Engng.,vol.8, pp.216-219, 2000.[10] B. Obermaier, C. Neuper, C. Guger, and G. Pfurtscheller, “Infor-mation transfer rate in a five-classes brain-computer interface,” IEEETrans.Neural Sys. Rehab. Eng., vol.9, no.3, pp.283-288, 2001.[11] C.W. Anderson, S.V. Devulapalli, and E.A. Stolz, “Determining mentalstate from EEG signals using neural networks,” Scientific Programming,Special Issue on Applications Analysis, Vol.4, No.3, pp.171-183, Fall,1995.[12] K.Inagaki and K.Nakayama, “Mental task classification based on brainwaves by using neural network”, IEICE, Technical Report, Vol.105,No.174, SIP2005-54, pp.25-30, July 2005. (in Japanese)[13] K.Inagaki and K.Nakayama,”On brain computer interface by usingmultilayer neural network”, IEICE, 20th Signal Processing Symposium,Dec. 2005 (in Japanese).676

A brain computer interface based on neural network with efficient pre-processing

https://kanazawa-u.repo.nii.ac.jp/?action=repository_action_common_download&item_id=7749&item_no=1&attribute_id=26&file_no=1

A brain computer interface based on neural network with efficient pre-processing

Abstract

Similar works

Full text

Available Versions

Kanazawa University Repository for Academic Resources