In this paper, we describe a project which uses PDAs to provide an interactive experience with a virtual environment. In particular, we focus on the navigational aspects of allowing the users to move through, and view, the environment. As this system will be deployed in a museum, it was crucial that the navigation be as intuitive as possible. To that end, we developed and evaluated two prototypes: one was based purely on gesture, whilst the other used a combination of gesture and keypad. For the purposes of our application, the combination of keypad and gesture provided the most effective

Marsden, Prof Gary

Tip, Nicholas

UCT Computer Science Research Document Archive

Navigation Control for Mobile Virtual Environments Gary Marsden & Nicholas Tip Department of Computer Science University of Cape Town South Africa +27 21 650 2666 gaz@cs.uct.ac.za   ABSTRACT In this paper, we describe a project which uses PDAs to provide an interactive experience with a virtual environment. In particular, we focus on the navigational aspects of allowing the users to move through, and view, the environment. As this system will be deployed in a museum, it was crucial that the navigation be as intuitive as possible. To that end, we developed and evaluated two prototypes: one was based purely on gesture, whilst the other used a combination of gesture and keypad. For the purposes of our application, the combination of keypad and gesture provided the most effective. Categories and Subject Descriptors H.5.2 [User Interfaces] General Terms Design, Experimentation, Human Factors, Keywords Virtual Reality, Input, Mobile Computing, PDA 1. INTRODUCTION At the University of Cape Town, we are engaged in a culture preservation project with the District Six Museum. District Six was a culturally diverse area of Cape Town, where people of different races lived fully integrated lives. This was an anathema to the government of the time, which forcibly removed the residents and leveled the area in the early 1960s. The area stands un-developed to this day. Since the establishment of a democratic government, however, there has been a concerted effort to preserve the memories and culture of District Six; the most visible of which is the District Six Museum. The nature of our collaboration with the museum is to create a virtual District Six, as it would have appeared before the evictions. Using old photographs, we are using photogrammetric techniques to reconstruct three-dimensional models. We are keen for as many people as possible to experience the virtual environment in a personal way. Inspired by the ‘Peephole’ work of Yee [6], we wondered about using PDAs to create a Peephole time-machine, allowing users to virtually view the old District Six whilst standing in the current District Six.  The motivation for embarking on this project is two-fold. Firstly, we wish to reflect the culture of District Six in as interactive a way as possible to show something worth celebrating from apartheid-era South Africa. Secondly, we wish to use the environment to elicit memories from those people who are old enough to remember living there. Using the voice recording function of the PDA, we will record oral histories from former residents. The philosophy behind the District Six museum is that anyone’s impression or interpretation of District Six is valid and they are keen to create this type of evolving archive.  Whilst the work on this project is potentially immense, this paper reports only on some initial studies to create an intuitive navigation interface, allowing PDA users to explore a virtual environment. 2. PAST WORK There have, of course, been many research projects exploiting the power of virtual reality in recreating environments that have been destroyed. One such large project in Greece [2] involves a virtual reconstruction of the Temple of Zeus. Our work to date on combining virtual reality and PDAs has confined the PDA to a secondary role, acting as a dynamic remote control for the environment [5]. Since this initial research was conducted, the power of PDAs has reached the point where we believe it is possible to use them to render virtual environments. There has been significant research into augmenting physical spaces with mobile computers. The most famous of these is perhaps Lancaster’s tourist guide system [1]. In terms of systems that project historical information onto physical spaces, there have been two major projects. The first of these is based in a castle in the UK, where a large “Augurscope” [4], similar in appearance to a television camera, is used to provide a window onto the castle as it would have appeared in Medieval times. The device is large, expensive and entirely unsuited to the requirements of the museum. Another approach is that taken by a team in Italy to recreate Pompeii [3]. They use dedicated Tablet PCs that use a tilt interface to move the view camera through the virtual environment. We wish to extend this work and investigate whether a PDA can give the same levels of interaction. Furthermore, we wish to use the PDAs as a Peephole [5] onto the virtual environment – imagine that the PDA acts as a letterbox through which one can peer into the virtual world.  Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. MobileHCI'05, September 19!22, 2005, Salzburg, Austria. Copyright 2005 ACM 1-59593-089-2/05/0009"$5.00. The rest of this paper reports our attempts at providing an intuitive navigation scheme for viewing virtual worlds using a PDA  3. SYSTEM DESIGN At the time this system was created (late 2004) the graphical processing abilities of PDAs was highly limited. We therefore designed a client-server system, wherein the user view of the three-dimensional environment was rendered on a server. The client PDA would request a particular view of the environment, which would be sent over a wireless network as a two-dimensional image. Sadly, network performance hampered the interaction to the point that the only way to provide seamless interaction was to render the environment in wire-frame when the user was moving, and fill in the textures and surfaces when the user was stationary within the environment. Some brief user testing showed that the wire-frame rendering was found to be an acceptable compromise. The focus of this paper, however, is on the design of an input system for the PDA which would allow users to fully explore a virtual environment. 3.1 Input Design True to the traditions of Jeff Hawkins, we started out by giving a non-functioning PDA, with a picture of a virtual environment stuck on the screen, to a large, diverse group of users. We asked these users how they thought they should navigate around the environment using the PDA. Anyone familiar with first-person games or virtual environments will be aware of this problem; controlling view (usually with a mouse) and character movement (usually with the keyboard) are two distinct tasks that requires some skill to master. As a PDA has neither a mouse or a keyboard, we were intrigued as to how users would imagine the control working.  As it turned out, all users expected the view on the screen to map directly to the device’s physical position – e.g. as the device is moved to the right, the image on the screen would show what was to the right of the original image. From Yee’s work [6], we knew that this was, in principle, a workable option. As for walking around in the environment, two main alternatives came to light. The first of these is simply to use the ‘D’ pad on the device to control walking. The second alternative was to control movement by tilting the PDA. As the PDA was tilted forward, the user position in the environment begins to move forward, at a rate proportional to the angle of tilt. Once the PDA is returned to the vertical, the movement stops.  4. IMPLEMENTATION 4.1 Looking Around The most expedient way to track the motion of a PDA in three-dimension space is to fit it with a three-dimensional tracker. Although trackers were available to us, this does not represent a viable solution for the scenario we are describing (our trackers are tethered and are only accurate to within a few meters of the base station). Instead we chose to test a system based on a relatively cheap two-dimensional tilt meter. For the actual experiments, we used a Gyration Technologies surface-less mouse, taped to the back of the PDA (see Figure 1). This mouse transmits its coordinates wirelessly to a base station which, in this case, was plugged in to the rendering server. Initial attempts at implementing the system this way had to be refined to ensure the mouse was perpendicular to the plane of movement. This was helped by the LCD screens on the devices, as users held them at a similar angle to ensure that the screen could be read. Once the ‘angle of attachment’ had been established, the results from the mouse were fairly accurate. By turning one’s body (and thus the PDA) to the left or the right, an x-coordinate change will be picked up by the mouse, and a relative change will occur to the bearing that the user is facing in the virtual world (see Figure 2a). A similar method is employed for looking up and down, with the mouse picking up a y- change as the user looks up and down (see Figure 2b). Figure 1: PDA with Gyration Mouse attached.    Figure 2: (a) shows movement in the horizontal direction;  (b) shows vertical movement 4.2 Moving Around If you recall, there were two suggestions for moving within the environment. The first of these, using the ‘D’ pad, is simple to implement. Being based on device tilting, the second is a little more complex. Using a two-dimensional input device, it is not possible to differentiate between PDA movement to change the viewpoint and PDA movement to change environment position. Therefore, we had to introduce a ‘walk’ mode and a ‘look’ mode to the interface, as in Figure 3. Once we had this scheme working, it became necessary to add a Reset button to the interface. This was necessitated by a lack of accuracy in the mouse, wherein small PDA movements would not be detected and the user’s position would become unsynchronized with the virtual view. (Reset button is also shown in Figure 3). (a) (b)   Figure 3: Mode buttons and Reset button added to the interface 5. EVALUATION PROCEDURE To evaluate the relative merits of both techniques, we used 10 volunteers to navigate a small virtual environment, five using each technique. Each participant was given an introduction to the research project, instructions for using the system and was then left to ‘play’ with the system for as long as they wished. There was no separate ‘training’ for the system as, given the context of final use, it was important to see how the system would be used from first contact. During usage, the users were observed and, after completion, were interviewed and given a satisfaction questionnaire to complete, (the questionnaire, we developed ourselves). The main components of the observation were to firstly determine how long it took before the users were interacting naturally with the system (defined as not hesitating before using a function) and secondly to record the number of mistakes they made in trying to navigate around objects (e.g. getting stuck against a wall). 6. RESULTS 6.1 Satisfaction Questionnaire Beginning with the questionnaires, the most marked result was the users’ perceived ease-of-use with the navigation techniques. The upper portion of Figure 4 shows the distribution of answers for the tilt system, and the lower portion, the results for the ‘D’ pad. On the scale used, a high number indicates dissatisfaction, so we can see that there is greater dissatisfaction with the tilt method. This would indicate that future development should focus on using the ‘D’ pad method.  Figure 4: Box and whisper plots for user satisfaction of navigation techniques. The upper box is for titling and the lower for the ‘D’ pad.  Figure 5: Comparison of time taken for users to familiarize themselves with the each navigation method. 6.2 Becoming Familiar During the experiment, we were watching for two event types. The first of these was the point at which navigation became intuitive; i.e. the user no longer paused or was tentative when using the interface. When this occurred, we recorded the time taken for this to happen. As can be seen from Figure 5, this happened a lot earlier for the ‘D’ pad system. 6.3 Mistakes The other event type recorded during the observation were the ‘mistakes’ made by the participants. A mistake in this context was defined as an unintended or erroneous movement; e.g. turning to the right then immediately left (as the user had intended to go left in the first place) or becoming stuck in a corner and unable to move away from it. Again, the tilt method suffered greatly, with users making, on average, 3.5 times as many mistakes as those using the ‘D’ pad: see Figure 6.  Figure 6: Average number of mistakes made by users of each system  6.4 Interviews In the interviews conducted after using the system, all users reported that the viewing system was very intuitive and no one reported problems with using it. Terms such as “natural” and “more real” were used, reporting that it gave a more natural way of viewing virtual space than was possible with a mouse and keyboard. One user, however, pointed out that viewing this way may prove too tiring for users over a prolonged period – the ‘gorilla arm’ effect. We would need to conduct much longer studies to determine if this is, indeed, a factor.  The interviews also served to confirm the users’ frustration with the tilting navigation. Our implementation of tilting had a slight lag (approximately 300 ms) which we thought might cause the frustration. However, from the users perspective, the most frustrating aspect was the ambiguity of tilting: using the ‘D’ pad would move position an exact amount, but it was hard to predict how much movement a ‘tilt’ would create. Although they did not express it in these terms, users were also frustrated that the tilt interface was modal – a gesture would have different results depending on the interface being in ‘walk’ or ‘view’ mode. The ‘D’ pad system was obviously modeless, with the pad controlling movement and PDA position controlling view. 7. DISCUSSION & CONCLUSIONS The goal of this project is to create a terminal for viewing an historically important virtual environment as part of a museum exhibition. As such, it needs to be able to intuitive in use so we asked non-experts to describe how such a system might work. Of the two ideas proposed, our initially supposition was that an interface based on gesture alone would prove more intuitive than a combination of  keypad and gesture. From the observations and the feedback from the users, it is abundantly clear that the ‘D’ pad is the design we should use in the final implementation. There are  many reasons why this might be the case. It could be as simple as one interface begin modal and the other not. Another confounding reason may be the update lag caused by rendering the image on a server. It is possible to eliminate both of these problems by using a device with a three-dimensional input device and a PDA with sufficiently powerful graphics processor. (At present we are considering switching from PDAs to the Sony PSP1 as a development platform to provide better graphics power). However, for the time being, we will remain with the combination of ‘D’ pad and two-dimensional tilt sensor and we would recommend that other researchers should follow suit. 8. FUTURE WORK At present we are putting together the final virtual environment and working with the museum to determine the best way to deploy the PDA viewing terminals. Initially, due to funding restrictions, their use will be limited to inside the museum building. We are currently applying for funding to set up a WiFi network to flood the former District Six site, so that the navigation will become a one-to-one mapping between physical and virtual location.                                                                     1 http://www.us.playstation.com/psp.aspx 9. ACKNOWLEDGMENTS This work was supported by the South Africa’s National Research Federation. Prototypes were developed using PDAs donated by bridges.org and HP. The authors would also like to thank Zaher Hamza and Gary Pnematicatos, who created the infrastructure that made these experiments possible. 6. REFERENCES [1] Cheverst, K., Davies, N., Mitchell, K., Friday, A., and  Efstratiou, C. Developing a Context-aware Electronic Tourist Guide: Some Issues and  Experiences. Proceedings of the SIGCHI conference on Human factors in computing systems  (CHI ’00), (The Hague, The Netherlands, April 1-6, 2000). ACM Press, New York, NY, 2000, 17-24. [2] Gaitatzes, A., Christopoulos, D., and Roussou, M. Reviving the past: cultural heritage meets virtual reality, Proceedings of the 2001 Conference on Virtual reality, archeology, and cultural heritage, (Glyfada, Greece 2001) ACM Press, New York, NY, 2001, 103-110.  [3] Scagliarini, D., Coralini, A., Vecchietti, E., Salmon Cinotti, T., Roffia, L., Galasso, S., Malavasi, M., Pigozzi, M., Romagnoli, E., and Sforza, F. Exciting understanding in Pompeii through on-site parallel interaction with dual time virtual models Proceedings of the 2001 Conference on Virtual reality, archeology, and cultural heritage, (Glyfada, Greece 2001) ACM Press, New York, NY, 2001, 83-90. [4] Schnädelbach, H., Koleva, B., Flintham, M., Fraser, M., Izadi, S., Chandler, P., Foster, M., Benford, S., Greenhalgh, C., and Rodden, T. The Augurscope: A Mixed Reality Interface for Outdoors Proceedings of the SIGCHI conference on Human factors in computing systems (CHI ’02),  (Minneapolis, Minnesota, USA, 2002). ACM Press, New York, NY, 2002, 9-16.  [5] Schroder, R., and Marsden, G. RIVERN: Controlling Virtual Environments from PDAs Proceedings of the 2004 annual research conference of the South African institute of computer scientists and information technologists on IT research in developing countries (SAICSIT 2004),  (Stellenbosch, South Africa) SAICSIT press, Pretoria, RSA, 2004, 135-141. [6] Yee, K-P. Interaction techniques for handheld devices: Peephole displays: pen interaction on spatially aware handheld computers Proceedings of the SIGCHI conference on Human factors in computing systems (CHI ’03),  (Fort Lauderdale, Florida, USA, 2003). ACM Press, New York, NY, 2003, 1-8 

Interfaces to Mobile Virtual Reality

http://pubs.cs.uct.ac.za/id/eprint/216/1/s16-marsden.pdf

Interfaces to Mobile Virtual Reality

Abstract

Similar works

Full text

Available Versions

UCT Computer Science Research Document Archive