This paper presents a study on electroencephalogram (EEG)-based control of an electric wheelchair. The objective is to control the direction of an electric wheelchair using only EEG signals. In other words, this is an attempt to use brain signals to control mechanical devices such as wheelchairs. To achieve this goal, we have developed a recursive training algorithm to generate recognition patterns from EEG signals. Our experimental results demonstrate the utility of the proposed recursive training algorithm and the viability of accomplishing direction control of an electric wheelchair by only EEG signals

Hua O. Wang

Kazuo Tanaka

Kazuyuki Matsunaga

IEEE Transactions on Robotics

Creative Repository of Electro-Communications

国立大学法人電気通信大学 / The University of Electro-CommunicationsElectroencephalogram-based control of anelectric wheelchair著者（英） Kazuo Tanaka, Kazuyuki Matsunaga, Hua O. Wangjournal orpublication titleIEEE Transactions on Roboticsvolume 21number 4page range 762-766year 2005-08URL http://id.nii.ac.jp/1438/00009305/doi: 10.1109/TRO.2004.842350A TEST FOR IEEETRAN.CLS— [RUNNING ENHANCED CLASS V1.5] 1Electroencephalogram-Based Controlof an Electric WheelchairKazuo Tanaka, Member, IEEE, Kazuyuki Matsunaga and Hua O. Wang, Senior Member, IEEEAbstract—This paper presents a study on electroencephalogram(EEG)-based control of an electric wheelchair. The objective is tocontrol the direction of an electric wheelchair using only EEG sig-nals. In other words, this is an attempt to use brain signals tocontrol mechanical devices such as wheelchairs. To achieve thisgoal, we have developed a recursive training algorithm to gener-ate recognition patterns from EEG signals. Our experimental re-sults demonstrate the utility of the proposed recursive training al-gorithm and the viability of accomplishing direction control of anelectric wheelchair by only EEG signals.Index Terms— Electroencephalogram-based control, electricwheelchair, direction control, recursive training algorithm.I. INTRODUCTIONTHERE are numerous interfaces and communication meth-ods between human and machines. A typical human-machine interface is to utilize input devices such as keyboards,mouse, joysticks, etc. Recently, a number of biological sig-nals such as electromyogram (EMG) [1], electroencephalogram(EEG) [2], etc., have been employed as hands-free interfaces tomachines (e.g., see [3–5]). In particular, the so called brain-computer interface (BCI) [6–10] has received significant atten-tion. The BCI is a system that acquires and analyzes neural(brain) signals with the goal of creating a direct high bandwidthcommunication channel between the brain and the computer.Such systems are envisioned to have huge potentials for a wideranging areas of research and applications such as brain (neu-ral) signal acquisition and processing, bioengineering, and un-derstanding the underlying neuroscience, to name a few. Forsystems and controls research, advances on brain-machine in-terfaces offer intriguing opportunities and challenges, for in-stance, brain control of machines.There have been several studies using hands-free inputs incontrolling machines through brainwaves. For example, speedand direction control of a small mobile robot using brainwavesand small facial muscular movements was reported in [11]. Inthis study, beta wave amplitude of EEG, jaw clench and eyemuscle signal (electrooculogram, or EOG) are used to controlthe speed, forward/backward switching and the direction, re-spectively. In the system of [11], it is clear that the primary con-trol signals for the mobile robot are jaw clench and eye musclesignals. The brainwaves is limited to control the speed of themobile robot. The control objective was to move the mobilerobot but without any specified or desired target positions. TheKazuo Tanaka and Kazuyuki Matsunaga are with the Departmentof Mechanical Systems and Intelligent Systems, The University ofElectro-Communications, Chofu, Tokyo 182-8585 Japan (email:ktanaka@mce.uec.ac.jp; matunaga@rc.mce.uec.ac.jp).Hua O. Wang is with the Department of Aerospace and Mechanical Engi-neering, Boston University, Boston, MA 02215 USA (email: wangh@bu.edu).paper did not report the success and/or failure rates of reach-ing the target positions. Bare [12] and Felzer [13] investigatedwheelchair control using EOG and EMG, respectively. To thebest of our knowledge, there has been no report of wheelchaircontrol using only EEG in the literature.In this paper, we investigate and demonstrate direction con-trol of an electric wheelchair using only EEG signals. In addi-tion, we present and discuss the success rate of reaching giventarget positions. Electric wheelchairs are some of the most im-portant devices to assist physically handicapped persons. Ourapproach can be regarded as an advanced bio-control applica-tions of BCI. Our experimental results for EEG-based controlof an electric wheelchair show that the success rate of reachingtarget positions is about 80%.II. EXPERIMENTAL SYSTEMIn this section, we introduce the experimental system forEEG control of an wheelchair. Figure 1 and Table I illustratethe EEG experimental set up and the experimental conditions,respectively. The experimental system consists of an electrocap(electrodes), electroencephalography (amplifier), an electrodebox, an A/D converter and a computer. Figure 2 depicts elec-trode placement, where the thirteen points (circle) are selectedaccording to the ten-twenty international electrode system [14].We select both earlobes (A1 and A2) as reference electrodes.Electric WheelchairElectrocap(Electrode)PCAmplifierElectrode BoxFig. 1. Experimental System.In the experiments, six healthy subjects that sit in the elec-tric wheelchair are asked to think the direction (left or right)that they want to go. In this paper, the left and right directionthoughts are called left thinking and right thinking, respectively.Each subject is asked to perform 100 trials for the left thinkingA TEST FOR IEEETRAN.CLS— [RUNNING ENHANCED CLASS V1.5] 2TABLE IEXPERIMENTAL CONDITIONS.Electrode placement C3, C4, P3, P4, O1, O2, F7, F8T3, T4, T5, T6, Fz (Fig. 2)Reference electrodes A1, A2 (Fig.2)Sampling rate 1024 [Hz]Sampling time 1.0 [s]Bandpass filter 0.53 ∼ 30 [Hz]F7FzA2 A1T6F8P3 P4T5O1 O2F8C4T4T6C3F7 T3T5Fig. 2. Electrode placement (10-20 international electrode system).and 100 trials for the right thinking. Half of the data (referredto as generating data) collected from each subject are used togenerate a pattern for each subject. The remaining half of thecollected data (referred to as checking data) collected are usedto check the validity of the generated pattern. The detectiontime in each trial is 1 second. The sampling rate for detectingEEG is 1024 [HZ].To avoid artifacts from EMG (and also EOG), we use thebandpass filer (as shown in Table I) with a pass band of 0.53Hz to 30 Hz in the EEG detection. However, it is very diffi-cult to perfectly reject the artifacts from EMG (and also EOG)even with the utility of the bandpass filer. Therefore, in our ex-periments, all the subjects are requested to minimize the move-ments of their bodies as well as eyes. It is possible to reduceartifacts from EOG and EMG by the requirement. With respectto EOG artifact, it is confirmed from preliminary experimentsthat the frequency peaks of EOG signals are less than 0.5 [Hz]under the requirement.III. PATTERN GENERATION AND RECURSIVE TRAININGIn Section III-A we present a recognition pattern generationmethod using the generating data. Section III-B provides a re-cursive training procedure to optimize recognition patterns.In previous papers [15–17], we have recommended to gen-erate an individual recognition pattern for each subject sincedifferences among EEGs of subjects are quite large. In this pa-per, an individual recognition pattern will be generated for eachsubject as well.A. Pattern generation and pattern matchingIn this section, we describe a new recognition pattern gener-ation method using the generation data.Let brainwave signals detected from thirteen electrodes in thekth generating data (k = 1, 2, · · · , 100) befk1 [t], fk2 [t], · · · , fk13[t] t = 1, 2, · · · , 1024. (1)We represent FFT results for the time domain data (1) as (2).F k1 [n], Fk2 [n], · · · , F k13[n] n = 1, 2, · · · , 38. (2)The coefficient of correlation between electrodes i and j is ob-tained asRk(i,j) =38∑n=1(F ki [n]− F ki )(F kj [n]− F kj )(38− 1)SkFiSkFj, (3)where F ki and Fkj are the average values of F ki [n] and F kj [n],respectively. SkFi and SkFj are the standard deviations of F ki [n]and F kj [n], respectively. By calculating correlations among allthe electrodes, we obtain the following correlation coefficientmatrix:Rk =⎛⎜⎜⎜⎝Rk(1,1) Rk(1,2) · · · Rk(1,13)Rk(2,1) Rk(2,2) · · · Rk(2,13)... · · · . . . ...Rk(13,1) Rk(13,2) · · · Rk(13,13)⎞⎟⎟⎟⎠ . (4)Since the matrix is symmetric, we generate the pattern vec-tor P k by selecting the upper triangular elements excepting thediagonal elements. The pattern vector P k consists of 78 ele-ments.P k = (pk1 , pk2 , · · · , pk77, pk78)= (Rk(1,2), Rk(1,3), · · · , Rk(12,11), Rk(12,13)) (5)k = 1, 2, · · · , 100As the collected data include both the left thinking and theright thinking, the recignition patterns for left thinking and rightthinking (i.e., left pattern vector Plk and right pattern vectorPrk) are generated separately.A TEST FOR IEEETRAN.CLS— [RUNNING ENHANCED CLASS V1.5] 3The final left and right recognition patterns V l and V r areobtained by calculating the average for all the generating data.Here notations plkm and prkm denote the mth elements of thevectors Plk and Prk, respectively, where m = 1, 2, · · · , 78.Notations vlm and vrm denote the mth elements of the vectorsV l and V r, respectively.V l = (vl1, vl2, · · · , vl77, vl78)V r = (vr1, vr2, · · · , vr77, vr78), (6)wherevlm =15050∑k=1plkm,vrm =15050∑k=1prkm,m = 1, 2, · · · , 78.The pattern matching using the final left and right recognitionpatterns proceeds as follows. First, we generate a pattern vectorV x from the checking data (unknown data) in the same way asin (1)-(6). Next, we calculate the Euclidean distances betweenV x and V l as well as between V x and V r.Ll =√(V l − V x) (V l − V x)T (7)Lr =√(V r − V x) (V r − V x)T (8)Ll denotes the distance between an unknown pattern and thefinal left thinking pattern. Lr denotes the distance between anunknown pattern and the final right thinking pattern. Left orright thinking is selected according to the distances, i.e., theleft thinking is selected when Ll < Lr, and the right thinkingis selected when Ll > Lr.B. Recursive trainingIn this section we present a recursive training procedure tooptimize recognition patterns. In the recursive training algo-rithm, ρ denotes the number of the iterations. ρmax denotes themaximum iteration number. In this paper, ρmax = 10. EL(ρ)and ER(ρ) denote the recognition rates [%] for the left and rightthinkings using the checking data at the ρth iteration, respec-tively. V l(ρ), V r(ρ), Ll(ρ) and Lr(ρ) denote V l, V r, Ll andLr at the ρth iteration, respectively. As addressed above, anindividual recognition pattern is generated by applying the re-cursive training algorithm to each subject.[Recursive training algorithm](Step 1)ρ = 1.(Step 2)Record EEG through the 100 trials for the left thinking andthe 100 trials for the right thinking. Divide the recorded datainto generating data and checking data.(Step 3)Generate left thinking and right thinking pattern vectorsV l(ρ) and V r(ρ) using (1)-(6).(Step 4)Calculate Ll(ρ) and Lr(ρ) from the checking data via (7)and (8). According to the pattern matching results, calculatethe recognition rates EL(ρ) and ER(ρ) at the ρth iteration.(Step 5)If EL(ρ) > Emax and ER(ρ) > Emax then ρE = ρ and goto Step 8 else go to Step 6.(Step 6)If |EL(ρ) − EL(ρ−1)+···+EL(ρ−s+1)s | < ∆E then ρE = ρand go to Step 8 else go to Step 7.(Step 7)If ρ = ρmax then ρE = ρ and go to Step 8 else ρ = ρ + 1and go back to Step 2.(Step 8) The trained patterns for the left and right thinkingsare V l(ρE) and V r(ρE), respectively. The recognition rates forthe checking data are EL(ρE) and ER(ρE), respectively.In the electric wheelchair control, we use Emax = 70 [%],s = 3 and ∆E = 10 [%]. A final pattern for each subjectis generated through the recursive training. Table II shows therecognition rates EL(ρE) and ER(ρE) of six subjects (Sub-jects A-F) for the checking data. The average recognition rateis about 80 [%].TABLE IIRECOGNITION RATES THROUGH RECURSIVE TRAINING.Recognition Rate (%)Subject EL ERA 73 88B 85 47C 91 81D 76 88E 74 79F 74 85Ave. 79 (%) 78 (%)IV. ELECTRIC WHEELCHAIR CONTROLFigures 3 and 4 show the experiment workspace. The exper-iment workspace consists of 28 regions. Each region is 60 [cm]× 90 [cm]. The colored regions in Figure 4 show the target po-sitions (LEFT goal and RIGHT goal). According to the patternmatching results shown in Section III-A, the electric wheelchairis moved to the region A (the left direction) or the region B (theright direction) from the initial position (start). The subjects arerequired to approach the target positions by repeating the move-ment. The electric wheelchair can arrive at the target positionswhen the number of incorrect direction decisions is less thanor equal to one and the number of correct direction decisionsis three. Therefore the success rate of reaching the target posi-tions is about 31.2[%] if the direction (left or right) decision israndom.In the experiment, the wheelchair is stopped during the EEGdetection and the pattern matching since the processing time isseveral seconds. According to the pattern matching result, theA TEST FOR IEEETRAN.CLS— [RUNNING ENHANCED CLASS V1.5] 4LEFT GoalRIGHT GoalStartFig. 3. Experiment Workspace (Photo).StartRIGHT GoalLEFT Goal90cm60cmA BFig. 4. Experimental Workspace.electrical wheelchair is moved to the next region. The proce-dure is repeated until the wheelchair reaches target position oruntil the number of incorrect directions is more than one.Figures 5 and 6 show the recognition patterns for the left andright thinkings of the subjects C and D, respectively. In the fig-ures, the horizontal and vertical axes denote the element num-ber m and the elements (V lm(ρE) and V rm(ρE)), respectively.These patterns are generated through the recursive training de-scribed in Section III. The patterns of Subjects C and D are quitedifferent. This fact shows that it is quite reasonable to generatean individual recognition pattern for each subject.Using the generated recognition patterns, 20 control trials(the 10 LEFT goal trials and 10 RIGHT goal trials) for eachsubject are performed. Figure 7 shows an experimental result.Table III shows the success rates for reaching the target po-sitions. The average success rate is about 80%. This showsviability of EEG-based control for an electric wheelchair. Theexception if the low successful rate of Right for the subject B.0 10 20 30 40 50 60 70 8000.20.40.60.81                                                               mVlSubject C   Left0 10 20 30 40 50 60 70 8000.20.40.60.81                                                               mVrSubject C   RightFig. 5. Generated Recognition Pattern (Subject C).0 10 20 30 40 50 60 70 8000.20.40.60.81                                                               mVlSubject D   Left0 10 20 30 40 50 60 70 8000.20.40.60.81                                                               mVrSubject D   RightFig. 6. Generated Recognition Pattern (Subject D).TABLE IIISUCCESS RATES.Success Rate(%)Subject Left RightA 80 90B 90 20C 100 100D 80 100E 80 80F 70 90Ave. 83(%) 80(%)A TEST FOR IEEETRAN.CLS— [RUNNING ENHANCED CLASS V1.5] 5Research is ongoing to further improve the success rates of allsubjects.LEFT Goal RIGHT Goal(1) Inital positions(2) 1st movement(3) 2nd movement(4) 3rd movement (Goal positions)Fig. 7. Experiment Results.V. CONCLUSIONThis paper investigates control of an electric wheelchair byelectroencephalogram (EEG) signals. This represents an at-tempt to control machines via brain signals. In this research,the goal is to control the direction (left or right) of the electricwheelchair by only EEG signals. To do so, we have developeda recursive training algorithm to generate recognition patternsfrom EEG. The experimental results are quite encouraging anddemonstrate the utility of the pattern recognition algorithm andthe viability of controlling wheelchairs by only EEG signals.ACKNOWLEDGMENTWe would like to thank Mr. Koki Kubo of the University ofElectro-Communications for his contribution to this research.REFERENCES[1] A. T. C. Au and R. F. Kirsch, EMG-Based Prediction of Shoulder andElbow Kinematics in Able-Bodied and Spinal Cord Injured Individuals,IEEE Transactions on Rehabilitation Engineering, Vol.8, No.4, pp.471-480, December 2000.[2] J. Millan, et al., Noninvasive Brain-Actuated Control of a Mobile Robotby Human EEG, IEEE Transactions on Biomedical Engineering, Vol.51,No.6, pp.1026-1033, June (2004).[3] R. L. Abboubi, C. A. Glass, N. A. Newby, J. A. Flint and W. Craelius,A Biomimetic Controller for a Multifinger Prosthesis, IEEE Transactionson Rehabilitation Engineering, Vol.7, No.2, pp.121-129, June 1999.[4] J. A. Cozens, Robotic Assistance of an Active Upper Limb Exercise inNeurologically Impaired Patients, IEEE Transactions on RehabilitationEngineering, Vol.7, No.2, pp.254-256, June 1999.[5] V. D. Kalanovic, D. Popovic and N. T. Skaug, Feedback Error LearningNeural Network for Trans-Femoral Prosthesis, IEEE Transactions on Re-habilitation Engineering, Vol.8, No.1, pp.71-80, March 2000.[6] NIPS 2001 Brain Computer Interface Workshop, Whistler, BritishColumbia, Canada, December 7, 2001.[7] J. R. Wolpaw, N. Birbaumer, W. J. Heetderks, D. J. McFarland, P. H.Peckham, G. Schalk, E. Donchin, L. A. Quatrano, C. J. Robinson andT. M. Vaugham, Brain-Computer Interface Technology: A Review of theFirst International Meeting, IEEE Transactions on Rehabilitation Engi-neering, Vol.8, No.2, pp.164-173, June 2000[8] J. R. Wolpaw, D. J. McFarland and T. M. Vaughan, Brain-Computer In-terface Research at the Wadsworth Center, IEEE Transactions on Reha-bilitation Engineering, Vol.8, No.2, pp.222-226, June 2000.[9] G. Pfurtscheller, et al., Current Trends in Graz Brain-Computer Interface(BC) Research, IEEE Transactions on Rehabilitation Engineering, Vol.8,No.2, pp.216-219, June 2000.[10] J. R. Wolpaw, et al., Invited review:Brain-computer interfaces for commu-nication and control, Clinical Neurophysiology 113 pp.767-791 (2002).[11] W. A. Amai, J. C. Fahrenholtz and C. L. Leger, Hands-free Operationof a Small Mobile Robot, Sandia National Laboratories Report 2000.(http://vrai-group.epfl.ch/icra2000/papers/amai.pdf)[12] R. Bare, et al., E.O.G. guidance of a wheelchair using neural network,Proc. of the International Conference on Pattern Recognition (ICRP’00),pp.4668-4672, Barcelona, (2000).[13] T. Felzer and B. Freisleben, HaWCoS: the ”hands-free” wheelchair con-trol system, Proceedings of the Fifth International ACM Conference onAssistive Technologies, pp.127-134, Edinburgh, Scotland (2002).[14] H. H. Japer, The ten-twenty electrode system of the international federa-tion in electroencephalography and clinical neurophysiology, Electroen-cephalgr. Clin. Neurophysiol., Vol.10, pp.371-375 1958.[15] H. Ogawa, K. Nozaki, K. Tanaka and T. Taniguchi,Electroencephalographm-based Recognition, Proceedings of the10th Intelligent System Symposium, pp.283-286 (in Japanese), 2000.[16] S. Hori, N. Kanamori and K. Tanaka, ”Generation of Electroencephalo-graphm Recognition Patterns via Genetic Algorithm”, SICE, Vol.38,No.12, pp.1041–1043 (in Japanese), 2002.[17] K. Tanaka, K. Matsunaga, N. Kanamori, S. Hori and H. O. Wang,Electroencephalogram-based Control of a Mobile Robot, Proceedings2003 IEEE International Symposium on Computational Intelligence inRobotics and Automation, Kobe, July, pp.670-675 (2003)

Electroencephalogram-based control of an electric wheelchair

https://uec.repo.nii.ac.jp/?action=repository_action_common_download&item_id=9382&item_no=1&attribute_id=22&file_no=1

Electroencephalogram-based control of an electric wheelchair

Abstract

Similar works

Full text

Available Versions

Creative Repository of Electro-Communications