Controlled Interaction: Strategies For Using Virtual Reality To Study Perception

Immersive virtual reality systems employing head-mounted displays offer great promise for the investigation of perception and action, but there are well-documented limitations to most virtual reality systems. In the present article, we suggest strategies for studying perception/action interactions that try to depend on both scale-invariant metrics (such as power function exponents) and careful consideration of the requirements of the interactions under investigation. New data concerning the effect of pincushion distortion on the perception of surface orientation are presented, as well as data documenting the perception of dynamic distortions associated with head movements with uncorrected optics. A review of several successful uses of virtual reality to study the interaction of perception and action emphasizes scale-free analysis strategies that can achieve theoretical goals while minimizing assumptions about the accuracy of virtual simulations

Multimodal Interaction in a Haptic Environment

In this paper we investigate the introduction of haptics in a multimodal tutoring environment. In this environment a haptic device is used to control a virtual piece of sterile cotton and a virtual injection needle. Speech input and output is provided to interact with a virtual tutor, available as a talking head, and a virtual patient. We introduce the haptic tasks and how different agents in the multi-agent system are made responsible for them. Notes are provided about the way we introduce an affective model in the tutor agent

Virtual audio reproduced in a headrest

When virtual audio reproduction is simultaneously required in many seats, such as in aircraft or cinemas, it may be convenient to use loudspeakers mounted inside each seat's headrest. In this preliminary study, the feasibility of virtual audio reproduction in the headrest of a single seat is explored using an inversion technique to compensate for crosstalk and the synthesis of head related transfer functions. Although large changes in the magnitude of the signals reproduced at the listener's ears are observed as the listener moves their head within the headrest, informal listening tests indicate that the reproduced acoustic images are surprisingly stable in about an eighth of an arc either side of the loudspeaker positions. Not surprisingly, frontal images are more difficult to reproduce with headrest loudspeakers

The eyes prefer real images

For better or worse, virtual imaging displays are with us in the form of narrow-angle combining-glass presentations, head-up displays (HUD), and head-mounted projections of wide-angle sensor-generated or computer-animated imagery (HMD). All military and civil aviation services and a large number of aerospace companies are involved in one way or another in a frantic competition to develop the best virtual imaging display system. The success or failure of major weapon systems hangs in the balance, and billions of dollars in potential business are at stake. Because of the degree to which national defense is committed to the perfection of virtual imaging displays, a brief consideration of their status, an investigation and analysis of their problems, and a search for realistic alternatives are long overdue

Viewing a Graph in a Virtual Reality Display is Three Times as Good as a 2D Diagram

An experiment is reported which tests whether network information is more effectively displayed in a three dimensional space than in a two dimensional space. The experimental task is to trace a path in a network and the experiment is carried out in 2D, in a 3D stereo view, in a 2D view with head coupled perspective, and in a 3D stereo view with head coupled perspective; this last condition creates a localized virtual reality display. The results show that the motion parallax obtained from the head coupling of perspective is more important than stereopsis in revealing structural information. Overall the results show that three times as much information can be perceived in the head coupled stereo view as in the 2D view

Real-Time Head Gesture Recognition on Head-Mounted Displays using Cascaded Hidden Markov Models

Head gesture is a natural means of face-to-face communication between people but the recognition of head gestures in the context of virtual reality and use of head gesture as an interface for interacting with virtual avatars and virtual environments have been rarely investigated. In the current study, we present an approach for real-time head gesture recognition on head-mounted displays using Cascaded Hidden Markov Models. We conducted two experiments to evaluate our proposed approach. In experiment 1, we trained the Cascaded Hidden Markov Models and assessed the offline classification performance using collected head motion data. In experiment 2, we characterized the real-time performance of the approach by estimating the latency to recognize a head gesture with recorded real-time classification data. Our results show that the proposed approach is effective in recognizing head gestures. The method can be integrated into a virtual reality system as a head gesture interface for interacting with virtual worlds