Presented at the 11th International Conference on Auditory Display (ICAD2005)It is essential for a visually impaired person to correctly identify the position of a sound source because such identification enables him/her to recognize his/her surroundings, including obstacles. We developed training equipment to help the visually impaired to improve their ability to identify the position of a sound source by applying a auditory display technique. Training for ten days with the system was conducted. As a result, the ability to identify a sound source position was improved

Iwaya, Yukio

Munekata, Tetsuya

Ohuchi, Makoto

Suzuki, Yoiti

Scholarly Materials And Research @ Georgia Tech

Proceedings of ICAD 05-Eleventh Meeting of the International Conference on Auditory Display, Limerick, Ireland, July 6-9, 2005TRAINING EFFECT OF A VIRTUAL AUDITORY GAME ON SOUND LOCALIZATIONABILITY OF THE VISUALLY IMPAIREDMakoto Ohuchi1,2,Yukio Iwaya1,Yôiti Suzuki1 and Tetsuya Munekata31Research Institute of Electrical Communication Tohoku University2Department of Information Science for Human WelfareTohoku Fukushi University3National Institute of Special Education(NISE)Japan.makoto@tfu-mail.tfu.ac.jpABSTRACTIt is essential for a visually impaired person to correctly iden-tify the position of a sound source because such identification en-ables him/her to recognize his/her surroundings, including obsta-cles. We developed training equipment to help the visually im-paired to improve their ability to identify the position of a soundsource by applying a auditory display technique. Training for tendays with the system was conducted. As a result, the ability toidentify a sound source position was improved.1. INTRODUCTIONThe authors have developed a virtual auditory display (VAD) whichis responsive to head movement. Furthermore, an action gamenamed “Hoy-Pippi” was developed with the VAD system for ap-plication to the welfare field. This game was actually played bypupils of a school for the blind [1] [2]. As this game became pop-ular with those who played it, we are confident of its potential as anew mode of entertainment for visually impaired persons.The characteristic feature of this game is “touching sound,”namely, positioning a hand at the position of a sound source. Inother words, in order to play this game, a player needs to recog-nize the position of a sound source, and must be able to correctlyposition his/her hand in that position.Then, we hypothesized that the ability to identify a positionof the sound source near the body would be improved by repeat-edly playing this game. For the visually impaired, improvementin the ability to identify a sound source position not only facili-tates perception of the position of the object which emits sound,but is useful in others ways. For example, it may enable them toturn their faces toward the conversation partner, as well as to per-ceive obstacles and to avoid them, and to recognize the space andenvironment in which they are standing. These are indispensablerequirements for improvement of the QoL (Quality of Life) of thevisually impaired. In this study, we trained ten listeners using thisgame for ten days, in order to confirm this hypothesis.2. OVERVIEW OF AUDITORY DISPLAYThe VAD convolves head related transfer functions (HRTFs) [3]according to the position of a sound in real time, the sound beingpresented through headphones (Figure 2). The head movement ofa listener is tracked by a 3D tracker system (Polhemus 3SPACEFASTRAK) to change HRTFs so that the position of the emittingthe sound object can be virtually fixed in a space. The listener canthen reduce the front-back error in a sound localization by headmovement, which results in improvement of the localization accu-racy. The receiver of the 3D tracker system was fixed to the top ofthe headband of the headphones. The HRTFs used for the exper-iment were the non-individualized, rather those of another personwere used.3. OUTLINE OF THE GAME SOFTWAREThe game software for training which the authors developed hasthe following specifications. If a game is started, a virtual crea-ture called Hoy-Pippi will appear in a random position, and shout“Hoy-Pippi.” Then, it flies around the head of a player for about 2seconds horizontally at the height of the player’s ears. If a playercan strike it with a rod having a 3D receiver made of polystyrenefoam, he/she can get a point(Figure 1). That is, it is a kind of molepanic game using 3D sound. The Hoy-Pippi appears 20 times pergame.Figure 1: A listener playing the game Hoy-Pippi.ICAD05-283Proceedings of ICAD 05-Eleventh Meeting of the International Conference on Auditory Display, Limerick, Ireland, July 6-9, 2005Host Computer (Windows 2000)CPU pentium4 2.2GHzHDD 120GBRAM   RIMM-512MBChip set  Intel 850memoryHRTFsoundsourceconvolutionheadangleD/AamplifierLow pass filter3D sensorAD/DA BoardHRTFssoundsourceTMS320C67013 SPACE FASTRAKEPSON Pro-1000DSP PCI-DSP670IFPC-SAD2080DPCIBusNF-electronic 3344STAXSRM-2012STAX SR-Lambda Basic System IIRod positionHead angle & positionFigure 2: The auditory display system.90cm0 153045Figure 3: Loudspeaker arrangement for the measurement of the ability to identify sound source positions.ICAD05-284Proceedings of ICAD 05-Eleventh Meeting of the International Conference on Auditory Display, Limerick, Ireland, July 6-9, 20054. MEASUREMENT OF THE ABILITY TO IDENTIFYSOUND SOURCE POSITIONS4.1. MethodTo measure the difference of the ability to identify a sound sourceposition between pre-training and post-training, the following ex-periment was conducted. As shown in Figure 3, seven loudspeak-ers were arranged around the circumference of a circle (horizontalplane) with a radius of 90 cm from the center of the listener’s head.A pink noise of 70 dBA is presented for 2 seconds from oneof the loudspeakers as a stimulus, and the listener points to theposition of the sound source with his/her right index finger. Thedifference between the position the listener points to and the centerof the correct speaker is counted as a sound localization error. Astimulus was randomly presented from each speaker 5 times, andthe average of the sound localization errors for each speaker wascalculated.This experiment was performed before and after training usingHoy-Pippi, and the effect of training was confirmed by comparingthose sound localization errors. Since listeners were sighted per-sons with normal hearing ability this time, they were blindfoldedbefore entering the laboratory.4.2. ListenersTen listeners participated in the experiment. Five of them werechosen at random and given training for 15 minutes every day forten days. They are called the “trained group.” The other five per-sons were a control group and underwent only a pre-test and apost-test without training. They are called the “non-trained group.”4.3. ResultsFigure 4 is a scatter diagram of the sound localization error forevery azimuth in the trained group. It can be seen that the soundlocalization error is decreased in almost all azimuths in the trainedgroup. Figure 5 is a scatter diagram of the sound localization errorin the non-trained group, which shows that there was scarcely anydifference between results of the pre- and post-test.Figures 6 and 7 show a comparison of the absolute value aver-age of the sound localization error of the pre-test and post-test inthe trained group and the non-trained group for every azimuth. Inthe trained group, the sound localization error of 15˚ to 90˚ showeda decrease after the training.The analyses of variance (ANOVAs) were performed with re-gard to the absolute value of the localization errors. Three factors,i.e., TRAINING (trained or non-trained), PRE (pre or post), andAZIMUTH (0˚ to 90˚) were treated as factors.As a result, the second order interaction among TRAINING,PRE, and AZIMUTH was significant (F(1,6)=2.63，p<.05). There-fore, analysis was sliced for each TRAINING condition. The in-teraction between the pre- vs. post-condition and the conditionof the sound source azimuths was significant only in the trainedgroup (F(1,6)=2.22，p<.05). In the non-trained group, however,this interaction was not significant. Errors for each azimuth werethen tested, which showed that the errors of the post-condition at60˚ (F(1,48)=5.45，p<.05) and 75˚ (F(1,48)=5.26，p<.05) were sig-nificantly smaller than those of the pre-condition. On the otherhand, in the non-trained group, no significant difference betweenazimuths was found. This result shows that the sound localizationerror of only the trained group decreased.Trained Group-60-40-2002040600 15 30 45 60 75 90Azimuth of Stimuli [deg]Sound localization error [deg]PrePostFigure 4: Scatter diagram of the sound localization error in thetrained group.Non Trained Group-60-40-2002040600 15 30 45 60 75 90Azimuth of Stimuli [deg]Sound localization error [deg]PrePostFigure 5: Scatter diagram of the sound localization error in thenon-trained group.Trained Group02468101214160 15 30 45 60 75 90Azimuth of Stimuli [deg]Sound localization error [deg]PrePostFigure 6: Comparison of the absolute value average of the soundlocalization error of the pre-test and post-test in the trained group.Non Trained Group02468101214160 15 30 45 60 75 90Azimuth of Stimuli [deg]Sound localization error [deg]PrePostFigure 7: Comparison of the absolute value average of the soundlocalization error of the pre-test and post-test in the non-trainedgroup.ICAD05-285Proceedings of ICAD 05-Eleventh Meeting of the International Conference on Auditory Display, Limerick, Ireland, July 6-9, 20055. DISCUSSIONIn the trained group, although the ability to identify the soundsource position after training improved in all the azimuths, therates of the improvement of 60˚ and 75˚ were especially remark-able. Since the sound localization ability for the azimuth nearthe median plane for humans is originally excellent [4], even withtraining, no remarkable improvement could be expected, but sincethe ability in the 60˚ to 75˚ azimuths is low, our experiment showedremarkable improvement by the training using Hoy-Pippi.However, when the localization error of the trained group andthat of the non-trained group in the pre-test were compared, that ofthe trained group was significantly large in the 60˚ azimuth, whilethat of the non-trained group was significantly large in the 90˚ az-imuth. This indicates the possibility that the trained group and thenon-trained group were not samples of the same quality statisti-cally. Thus, further experiments with the training of many morelisteners are required.In this research, although training for the ten days was per-formed, the authors did not investigate how many days were re-quired for the appearance of the training effect nor the type ofgrowth curve. However, analysis of the transition of the averagescore of the game Hoy-Pippi by ANOVAs showed that a signifi-cant difference appeared in the score on the 7th day of training.To obtain points in the Hoy-Pippi game, a player must be able torecognize the sound source position, as well as to be physicallyable to move the rod to that position. Therefore, it can be said thatthe ability of sound source position identification improved signif-icantly on the 7th day from the start of training.Finally, in this research study, only sighted subjects partici-pated and thus the results may not match those of the visually im-paired with generally good ability of sound source position iden-tification. However, this equipment should be extremely useful inattempts to improve the ability of sound source position identifica-tion in visually impaired children or the non-congenital blind.6. CONCLUSIONSA 3D sound game called “Hoy-Pippi” was developed with thetechnique of virtual auditory display. It was hypothesized that theability of sound source position identification improves with useof the game. Training with the game was conducted for the tendays. The listeners were ten sighted persons with normal hearing,five assigned to the trained group, and the other five assigned tothe non-trained group. The ability of identifying the sound sourceposition of each listener was measured before the training and afterthe training, results being compared after the training.As a result, the ability of sound source position identificationof only the trained group improved. This indicated that the 3Dsound game Hoy-Pippi was effective in improvement in the abilityof sound source position identification.In the future, the authors intend to modify the game as an in-teractive type on a network so that two players living in differentplaces can play each other. Moreover, the authors will further im-prove the application software and develop a training simulator ofa mental map for the visually impaired [5] [6] [7].7. ACKNOWLEDGMENTSThis research was partially supported by the Ministry of Educa-tion, Science, Sports and Culture, Grant-in-Aid for ExploratoryResearch, 14658054, 2002∼2004 and Consortium R&D projectsfor regional revitalization of Ministry of Economy Trade and In-dustry of Japanese Government, 2003∼2004.8. REFERENCES[1] M. Ohuchi, Y. Iwaya, Y. Suzuki, & T. Munekata, “A gamefor visually impaired children with a 3-D virtual auditory dis-play,” ICAD2003, pp. 309, 2003[2] M. Ohuchi, Y. Iwaya, Y. Suzuki, & T. Munekata, “Prototyp-ing of sound VR game for children with visual impairment,”The 23rd Asia-Pacific international seminar on special edu-cation, pp. 39, 2003[3] M. Morimoto & Y. Ando, “On the simulation of sound local-ization,” J.Acoust.Soc.Jpn., Vol.1, pp. 167-174, 1980[4] A. W. Mills, “On the minimum audible angle,” J. Acuust.Soc. Amer., vol.30, pp. 237-246 , 1958[5] J. Sánchez, “Interactive 3D sound hyperstories for blind chil-dren,” Proceedings of ACM-CHI , pp. 318-325, 1999[6] R. G. Golledge, M. Blades, R. Kitchin & D. Jacobson, “Un-derstanding Geographic Space Without The Use of Vision,”NSF Peport SBR95-14907(Final Report), 1999[7] O. Lahav & D. Mioduser, “Multisensory virtual environ-ment for supporting blind persons’ acquisition of spatialcognitive mapping, orientation, and mobility,” Proc.4rd IntlConf.Disability, Virtual Reality & Assoc.Tech., pp. 213-220, 2002ICAD05-286

Training effect of a virtual auditory game on sound localization ability of the visually impaired

https://smartech.gatech.edu/bitstream/1853/50191/1/OhuchiIwaya2005.pdf

Training effect of a virtual auditory game on sound localization ability of the visually impaired

Abstract

Similar works

Full text

Available Versions

Scholarly Materials And Research @ Georgia Tech