The effect of modality on social presence, presence and performance in collaborative virtual environments

Humans rely on all their senses when interacting with others in order to communicate and collaborate efficiently. In mediated interaction the communication channel is more or less constrained, and humans have to cope with the fact that they cannot get all the information that they get in face-to-face interaction. The particular concern in this thesis is how humans are affected by different multimodal interfaces when they are collaborating with another person in a shared virtual environment. One aspect considered is how different modalities affect social presence, i.e. people’s ability to perceive the other person’s intentions and emotions. Another aspect investigated is how different modalities affect people’s notion of being present in a virtual environment that feels realistic and meaningful. Finally, this thesis attempts to understand how human behavior and efficiency in task performance are affected when using different modalities for collaboration. In the experiment presented in articles A and B, a shared virtual environment that provided touch feedback was used, making it possible to feel the shape, weight and softness of objects as well as collisions between objects and forces produced by another person. The effects of touch feedback on people’s task performance, perceived social presence, perceived presence and perceived task performance were investigated in tasks where people manipulated objects together. Voice communication was possible during the collaboration. Touch feedback improved task performance significantly, making it both faster and more precise. People reported significantly higher levels of presence and perceived performance, but no difference was found in the perceived social presence between the visual only condition and the condition with touch feedback. In article C an experiment is presented, where people performed a decision making task in a collaborative virtual environment (CVE) using avatar representations. They communicated either by text-chat, a telephone connection or a video conference system when collaborating in the CVE. Both perceived social presence and perceived presence were significantly lower in the CVE text-chat condition than in the CVE telephone and CVE video conference conditions. The number of words and the tempo in the dialogue as well as the task completion time differed significantly for persons that collaborated using CVE text-chat compared to those that used a telephone or a video conference in the CVE. The tempo in the dialogue was also found to be significantly higher when people communicated using a telephone compared to a video conference system in CVEs. In a follow-up experiment people performed the same task using a website instead, with no avatar but with the same information content as before. Subjects communicated either by telephone or a video conference iv system. Results from the follow-up experiment showed that people that used a telephone completed tasks significantly faster than those that used a video conference system, and that the tempo in the dialogue was significantly higher in the web environments than in the CVEs. Handing over objects is a common event during collaboration in face-to face interaction. In the experiment presented in article D and E, the effects of providing touch feedback was investigated in a shared virtual environment in which subjects passed a series of cubic objects to each other and tapped them at target areas. Subjects could not communicate verbally during the experiment. The framework of Fitts’ law was applied and it was hypothesized that object hand off constituted a collaboratively performed Fitts’ law task, with target distance to target size ratio as a fundamental performance determinant. Results showed that task completion time indeed linearly increased with Fitts’ index of difficulty, both with and without touch feedback. The error rate was significantly lower in the condition with touch feedback than in the condition with only visual feedback. It was also found that touch feedback significantly increased people’s perceived presence, social presence and perceived performance in the virtual environment. The results presented in article A and E analyzed together, suggest that when voice communication is provided the effect of touch feedback on social presence might be overshadowed. However, when verbal communication is not possible, touch proves to be important for social presence.  QC 2010063

Presence in multimodal interfaces

• Theory on presence from the perspective of telecommunications and virtual environments. • Presentation of subjective and objective measures of presence. • How interfaces that support the modality touch affect perceived presence in distributed environments. • Description of research that focus on how haptic force-feedback affect perceived presence measured by subjective and objective measures and task performance

Exploring communicative modes for a virtual environment: a social psychological analysis

This paper reports from an experimental study of cooperative work in a collaborative virtual environment (CVE) with sixty test subjects. A method from social psychology, interaction process analysis (IPA), was used for analyzing group dynamics within the CVE in relation to three communicative modes: text-chat, voice, and video conferencing. The study investigated how the communicative mode influenced the quality of the social interaction and how cooperative work in a CVE could be analyzed by using categories of social interaction. It was found that the frequencies of utterances in the voice and video conferencing conditions were significantly larger than in the text-chat condition. The dialogue of the text-chat condition was found to be more task oriented than the dialogues of the voice and video conferencing condition. The social psychological method used was not developed for use within CVE:s, but it was found to work well since the scope of the IPA method is broad and applicable to virtually any small group

WoodenHaptics : A Starting Kit for Crafting Force-Reflecting Spatial Haptic Devices

Spatial haptic interfaces have been around for 20 years. Yet, few affordable devices have been produced, and the design space in terms of physical workspace and haptic fidelity of devices that have been produced are limited and discrete. In this paper, an open-source, open-hardware module-based kit is presented that allows an interaction designer with little electro-mechanical experience to manufacture and assemble a fully working spatial haptic interface. It also allows for modification in shape and size as well as tuning of parameters to fit a particular task or application. Results from an evaluation showed that the haptic quality of the WoodenHaptics device was on par with a Phantom Desktop and that a novice could assemble it with guidance in a normal office space. This open source starting kit, uploaded free-to-download online, affords sketching in hardware; it “unsticks” the hardware from being a highly-specialized and esoteric craft to being an accessible and user-friendly technology, while maintaining the feel of high-fidelity haptics.QC 20150420</p

Supporting presence in collaborative environments by haptic force feedback

An experimental study of interaction in a collaborative desktop virtual environment is described. The aim of the experiment was to investigate if added haptic force feedback in such an environment affects perceived virtual presence, perceived social presence, perceived task performance, and task performance. A between-group design was employed, where seven pairs of subjects used an interface with graphic representation of the environment, audio connection, and haptic force feedback. Seven other pairs of subjects used an interface without haptic force feedback, but with identical features otherwise. The PHANToM, a one-point haptic device, was used for the haptic force feedback, and a program especially developed for the purpose provided the virtual environment. The program enables for two individuals placed in different locations to simultaneously feel and manipulate dynamic objects in a shared desktop virtual environment. Results show that haptic force feedback significantly improves task performance, perceived task performance, and perceived virtual presence in the collaborative distributed environment. The results suggest that haptic force feedback increases perceived social presence, but the difference is not significant

Sonification of haptic interaction in a virtual scene

This paper presents a brief overview of work-in-progress for a study on correlations between visual and haptic spatial attention in a multimodal single-user application comparing different modalities. The aim is to gain insight into how auditory and haptic versus visual representations of temporal events may affect task performance and spatial attention. For this purpose, a 3D application involving one haptic model and two different sound models for interactive sonification are developed.Qc 20150323</p