Brewster, S.A.

English

Mobile telephones, Personal Digital Assistants (PDAs) and handheld computers are one of the fastest growth areas of computing. One problem with these devices is that they have a limited amount of screen space: the screen cannot be large as the device must be able to fit into the hand or pocket to be easil

Stephen A. Brewster

CiteSeerX

Sound in the Interface to a Mobile Computer

Enlighten

      Brewster, S.A. (1999) Sound in the interface to a mobile computer. In, HCI International '99 (the 8th International Conference on Human-Computer Interaction), 22-26 August 1999., pages pp. 43-47, Munich, Germany.        http://eprints.gla.ac.uk/3285/     Glasgow ePrints Service http://eprints.gla.ac.uk Sound in the Interface to a Mobile ComputerStephen A. BrewsterGlasgow Interactive Systems Group,Department of Computing Science,University of Glasgow, Glasgow, G12 8RZ, UK.Tel: +44 (0)141 330 4966 / Fax: +44 (0) 141 330 4913 /Email: stephen@dcs.gla.ac.uk / Web: www.dcs.gla.ac.uk/~stephen1 IntroductionMobile telephones, Personal Digital Assistants (PDAs) and handheld computersare one of the fastest growth areas of computing. One problem with thesedevices is that they have a limited amount of screen space: the screen cannot belarge as the device must be able to fit into the hand or pocket to be easilycarried. As the screen is small it can become cluttered with information asdesigners try to cram on as much as possible. In many cases desktop widgets(buttons, menus, windows, etc.) have been taken straight from standardgraphical interfaces (where screen space is not a problem) and applied directlyto mobile devices. This has resulted in devices that are hard to use, with smalltext that is hard to read, cramped graphics and little contextual information.One way to solve the problem would be to substitute non-speech audio cues forvisual ones. Sound could be used to present information about widgets so thattheir size could be reduced. This would mean that the clutter on the displaycould be diminished and/or allow more information to be presented. This mustbe done in a way that maintains usability otherwise these smaller widgets willrender the device unusable.There has been little previous research directly in the area of sound in mobilecomputing devices. Some research has been done on different selectiontechniques using a pen and graphical display on mobile devices (Ren & Moriya,1997) but not using sound. Research at Glasgow has demonstrated theeffectiveness of sound in desktop widgets (Brewster, 1998) but not so far inmobile devices. The work reported here brings these together.2 ExperimentAn experiment was conducted to investigate the affect that sound would have onthe interaction with on-screen buttons in a mobile computing device. Sixteenparticipants from the Computing Science Department at Glasgow were used.The selection was made up of postgraduate and undergraduate students andstaff, nine male and seven female, all between the ages of 21 and 45.The experiment used a fully counterbalanced, two-condition, within-groupsdesign. The independent variables were button size and sound type. There weretwo conditions: standard (16x16 pixels) and small (8x8 pixels) buttons (seeFigure 1). Both of these sizes are commonly found on PDAs. There were two 7-minute treatments in each condition: visual only buttons and visual plus sound.Figure 1: Screenshots from the large and small buttons conditions.The experiment was run on a 3Com PalmIII handheld computer with input via astylus. The task the participants had to perform was similar to that of Brewsteret al. (1995). Participants had to enter a series of five digit strings (shown in thetarget window in Figure 1) using the numeric keypad. After each digit had beentapped the OK button had to be pressed to confirm it. The numbers appeared inthe window labelled ‘Current’.The dependent variables were subjective workload (using the standard NASATLX workload test) and number of strings entered. Together these gave a goodquantitative and qualitative measure of usability. Training was given beforeeach part of the experiment began and NASA TLX workload scales werecompleted after each treatment.The main hypotheses were: Subjective workload would be reduced in both conditions when soundswere present as participants would receive information they needed tooperate the buttons more effectively. This would be demonstrated by areduction in the NASA TLX workload scores. The efficiency of button use would increase in both conditions when thesounds were present because the sounds would allow users to hit thebuttons more easily and know when they had mis-clicked them. This wouldbe shown by an increase in the number of strings entered.The sounds used in the study were earcons (Blattner et al., 1989). These arestructured non-speech sounds that have been investigated in detail by Brewster(1998). Three earcons were used based around those of Brewster et al. (1995).Results from a previous experiment by Brewster & Cryer (1999) showed thatsophisticated sounds were more successful at improving usability than simpleones. The standard PalmIII keyclick sound was used to indicate pen release on abutton. A second sound indicated pen down on a button. This signified thatusers had hit the target (especially important for small targets) and was a higherpitched version of the basic sound. Finally, a sound was played if the user mis-hit a button – again common if targets are small and hard to hit (Brewster et al.,1995). This was a lower pitched version of the basic sound. The earcons wereall simple as the Palm’s audio capabilities were restricted.3 ResultsThe results of the subjective workload tests can be seen in Figure 2. Each wasscored out of 20 and T-tests were used for the statistical analysis. Results for thelarge buttons showed that in all cases (except for time pressure) the sonically-enhanced buttons reduced workload. This confirmed the hypothesis. Timepressure was identical for both treatments so no change was expected.For example, mental demand was significantly reduced (T15=3.81, p=0.001), aswas annoyance (T15=2.38, p=0.03) and overall preference was significantlyincreased for the large buttons with sound (T15=3.40, p=0.004). Due to lack ofspace all the results cannot be reported here.For the small buttons the results were again in favour of sound - workload wassignificantly reduced in all of the categories except for physical effort and timepressure. This again confirmed the hypothesis.For example, effort expended was significantly reduced (T15=5.80, p=0.00003)as was frustration experienced (T15=4.99, p=0.0002) and subjective performanceachieved was significantly increased (T15=4.01, p=0.0011) for the small buttonswith sound.The quantitative results can be seen in Figure 3. This shows the number of 5-digit strings that were entered in each of the treatments. The results show that inboth of the sound treatments performance was very significantly improved(large buttons: T15=9.22, p=0.0000001, small buttons: T15=7.79, p=0.000001).4 Discussion and ConclusionsThe results from the experiment show that sound can have important effects onusability in mobile computing devices. The qualitative results show that soundhad a big effect on workload for both button sizes. In almost all of the categoriesworkload was significantly reduced when sound was present. The results alsoshowed that the sounds did not annoy the participants, in fact they rated theannoyance as significantly less when sounds were present. They also rated thesounds as preferable to the silent buttons. This indicated that sound can improvethe qualitative experience users have with mobile devices.The quantitative results also back up the hypothesis that sounds will improveusability as participants were able to enter significantly more strings whensounds were present for both button sizes. The sounds helped users target thebuttons better, know when they had been pressed correctly and when they hadbeen mis-hit. This made them significantly easier to operate.These results give designers a simple way to increase the usability of buttons(one of the most common widgets) in mobile devices. This work also indicates02468101214161820MentalPhysicalTimeEffortAnnoyanceFrustrationPerformanceOverallpreferenceWorkload categoriesAverageworkloadscoreLarge buttons with soundLarge buttons no soundsmall buttons with soundsmall buttons no soundFigure 2: Average NASA TLX workload scores for the two conditions and twotreatments. For the first six categories higher scores mean higher workload, for thefinal two higher scores mean lower workload. Standard error bars are shown.that the research carried out on sound in desktop widgets (Brewster, 1998) canbe applied to hand-held devices and show the same benefits.This work was supported by EPSRC grant GR/L66373.5 ReferencesAll references by Brewster can be found at http://www.dcs.gla.ac.uk/~stephenBlattner, M., Sumikawa, D. & Greenberg, R. (1989). Earcons and icons: Theirstructure and common design principles. Human Computer Interaction, 4 (1)11-44.Brewster, S.A. (1998). The design of sonically-enhanced widgets. Interactingwith Computers, 11 (2), 211-235.Brewster, S.A. & Cryer, P.G. (1999). Maximising Screen-Space on MobileComputing Devices. To appear in Summary Proceedings of ACM CHI'99,Pittsburgh,: ACM Press Addison Wesley.Brewster, S.A., Wright, P.C., Dix, A.J. & Edwards, A.D.N. (1995). The sonicenhancement of graphical buttons. In Nordby, Helmersen, Gilmore & Arnesen(Eds.): Proceedings of IFIP Interact'95, (pp. 43-48), Lillehammer, Norway:Chapman & Hall.Ren, X. & Moriya, S. (1997). The best among six strategies for selecting aminute target and the determination of the minute maximum size of the targetson a pen-based computer. In Howard, Hammond & Lindfaard (Eds.),Proceedings of IFIP Interact'97, (pp. 85-92), Sydney, Australia: Chapman &Hall.010203040506070Large buttons w ithsoundLarge buttons nosoundSmall buttons w ithsoundSmall buttons nosoundCondition/TreatmentAveragenumberofstringsenteredFigure 3: Average number of strings entered. Standard error bars are shown.

All references by Brewster can be found at http://www.dcs.gla.ac.uk/~stephen

Maximising Screen-Space on Mobile Computing Devices.

The best among six strategies for selecting a minute target and the determination of the minute maximum size of the targets on a pen-based computer. In Howard, Hammond & Lindfaard (Eds.),

The design of sonically-enhanced widgets.

The sonic enhancement of graphical buttons.

Sound in the interface to a mobile computer

http://eprints.gla.ac.uk/3285/1/hcii299.pdf

Sound in the interface to a mobile computer

Abstract

Similar works

Full text

Available Versions

CiteSeerX

Enlighten