One of the classic problems with immersive environments is data entry; with a head-mounted display (HMD) the user can no longer see the keyboard. Although for many applications data entry is not a requirement, for some it is essential: communicating in collaborative environments, entering a filename to which work can be saved, or accessing system controls. Combining data gloves and a graphically represented keyboard with a predictive spelling paradigm, we describe an effective text entry technique for immersive environments; we explore the key issues when using such a technique, and report the results of preliminary usability testing

McCaul, Barry

Sutherland, Alistair

English

DCU Online Research Access Service

Predictive Text Entry in Immersive EnvironmentsBarry McCaul and Alistair SutherlandSchool of Computing,Dublin City University, Ireland.{barry.mccaul | alistair.sutherland}@computing.dcu.ieAbstractOne of the classic problems with immersiveenvironments is data entry; with a head-mounted display(HMD) the user can no longer see the keyboard. Although for many applications data entry is not a requirement, for some it is essential: communicating in collaborativeenvironments, entering a filename to which work can besaved, or accessing system controls. Combiningdatagloves and a graphically represented keyboard with a predictive spelling paradigm, we describe an effective text entry technique for immersive environments; we explorethe key issues when using such a technique, and reportthe results of preliminary usability testing.1. IntroductionVirtual Reality (VR) has progressed significantly sinceits conception, enabling previously impossibleapplications such as virtual prototyping, telepresence, and augmented reality. However, data entry remains one of the classic problems with this technology [4], and althoughthere have been previous attempts to address the issue,there is, as yet, no elegant solution to the problem.Existing techniques for data entry include speech,chorded keyboards, pen and tablet systems, and variousgloved techniques. Speech recognition systems offer anatural interface, but can require considerable training,suffer in noisy environments, and are inefficient for textediting and manipulation. Chorded keyboards, such as the Twiddler2 [5], allow one handed text entry, but beginnersare required to learn a complicated alphabet of chordsbefore they begin to type. Pen and tablet techniques, while natural, are limited by the handwriting speed, and can beslowed further by the need to learn and use adaptedalphabet characters to aid recognition accuracy.Datagloves are commonly used for interaction inimmersive environments, allowing the wearer to interactin a natural way with virtual objects. A text entrytechnique which uses datagloves is thus likely to be easilyincorporated into many immersive environments. Systems using datagloves for text entry include thePinch Keyboard [1], VType [2] and Finger-Joint GestureWearable Keypad [3]. While the Pinch Keyboard allowsusers to select individual letters on a virtual keyboard bymapping the user's hand in 3D space to select the desiredrow, both the Finger-Joint Gesture Wearable Keypad andVType use ambiguous keyboards, with a dictionary ofwords used to help predict the users intention.We suggest a method of data entry that combinesaspects from both the Pinch Keyboard [1] and VType [2]. Using two 5DT datagloves, a predictive text paradigm, agraphically represented keyboard, and intuitive interaction techniques, we developed a text entry technique that isboth simple and effective; a method that attempts to utilise user muscle memory, but ignores the position of the handand instead predicts which key the user actually wantsbased on finger flexure and the statistical redundancy ofthe English language.2. Predictive text entry in immersive environmentsThe central idea is that, when data entry is required in a virtual environment, the user is presented with a graphical representation of a keyboard, with each finger mapping to a column of keys. To type, the user simply flexes therelevant finger to select the correct column. After asequence of finger flexes the user is presented with thepredicted word. Users may rotate through alternativematching words to indicate the desired word if the initialprediction is incorrect.At any stage during the course of text entry there maybe several clashing words for a given series of keypresses. When there are clashing words, our system offersthe user an ordered list of the potential words rankedaccording to their probability. The most likely word ishighlighted ready for selection, but the user can highlightany of the words in the list by altering the tilt of their241IEEE Virtual Reality 2004 March 27-31, Chicago, IL USA 0-7803-8415-6/04/$20.00©2004 IEEE. Proceedings of the 2004 Virtual Reality (VR’04) 1087-8270/04 $ 20.00 IEEE wrist. On the left hand side, the user sees the words thatmap to the current sequence of keys they have pressed.These are highlighted by tilting the left hand and selectedby pressing the left thumb (Figure 1).Figure 13. Design and implementationWe focused on the three areas central to the use ofambiguous keyboards in immersive environments: thedisambiguation engine, the associated interactiontechniques, and the keyboard layout.Central to the use of the system is the disambiguationengine, which must predict the word intended by the userwith reasonable accuracy to avoid user frustration.Statistically, a QWERTY keyboard layout, with its uneven finger distribution, leads to a high clash count - where asequence of finger flexures map to more then onepotential word. However, using a language model (basedon a 4 million word corpora) and analysing the previouslytyped words, word prediction accuracy of 96% wasachieved.Further increases in prediction accuracy are, in fact,possible if keyboards are designed to minimise clashing.By analysing the frequency of each word in a dictionary, it is possible to create optimal keyboard layouts that have aprediction accuracy of over 99%. While these keyboardsare likely to seem foreign and unfamiliar to users, thepotential speed gains offered by high prediction accuracymay counter the extra time spent searching for keys.Word completion offers further opportunities forincreased text entry speed. Originally designed for peoplewith physical disabilities, who had difficulty using astandard keyboard for text input, word completionsystems generally wait for a user to enter the first letter ofa word, and then offer a list of potential completed words. When word completion is active the user sees a list ofcomplete words that they may choose from on the righthand side (Figure 1). These are highlighted by tilting theright hand and selected by pressing the right thumb.4. Usability testsInformal tests were carried out to evaluate variousaspects of the system, including keyboard layout and word prediction. It was hoped that the rich graphical nature ofVR would reduce the strain experienced by userssearching alternative, optimised keyboards, and facilitatequick scanning of complete word lists as they type,making word completion an extremely useful method oftext entry. Three keyboard layouts were tested: atraditional QWERTY; an alphabetic, similar to thosefound on many phones (Figure 1); and a layout optimisedto reduce clashing. All three layouts were tested, on 5participants, both with and without word completionactive, giving a total of 6 tests per user.5. Conclusions and further workOur results showed that our text entry technique waseasily understood, with beginners typing at an averagespeed of 9 words per minute (WPM) after just 12 minutestraining (More experienced users type at over 17 WPM).Keyboard layout caused no significant change to text entry speed; however, word completion provided significantspeed increases for all users, increasing WPM by andaverage of 33% when used.Further test are currently underway to evaluatealternative keyboard representations, and the effects ofvarious visual cues on speed and ease of use of thesystem.6. References[1] D. Bowman, V. Ly, and C. Campbell. Pinch keyboard:Natural text input for immersive environments. Technical report, Virginia Tech Dept. of Computer Science, 2001.[2] F. Evans, S. Skiena, and A. Varshney. VType: Entering text in a virtual world. Submitted to International Journal of Human-Computer Studies.[3] M. Goldstein and D. Chincolle. The finger-joint gesturewearable keypad, 1999. In Second Workshop on HumanComputer Interaction with Mobile Devices.[4] M. R. Mine, F. P. Brooks, Jr., and C. H. Sequin. Movingobjects in space: Exploiting proprioception in virtual-environment interaction. Computer Graphics, 31(AnnualConference Series):19-26, 1997.[5] Twiddler2. Handykey web site. Available from:http://www.handykey.com/site/twiddler2.html242Proceedings of the 2004 Virtual Reality (VR’04) 1087-8270/04 $ 20.00 IEEE 

Predictive text entry in immersive environments

Abstract

Similar works

Full text

Available Versions

DCU Online Research Access Service

Name not available

Irish Universities

Crossref