Correlations between vocal input and visual response apparently enhance presence in a virtual environment

This work investigates novel alternative means of interaction in a virtual environment (VE).We analyze whether humans can remap established body functions to learn to interact with digital information in an environment that is cross-sensory by nature and uses vocal utterances in order to influence (abstract) virtual objects. We thus establish a correlation among learning, control of the interface, and the perceived sense of presence in the VE. The application enables intuitive interaction by mapping actions (the prosodic aspects of the human voice) to a certain response (i.e., visualization). A series of single-user and multiuser studies shows that users can gain control of the intuitive interface and learn to adapt to new and previously unseen tasks in VEs. Despite the abstract nature of the presented environment, presence scores were generally very high

Respiratory, postural and spatio-kinetic motor stabilization, internal models, top-down timed motor coordination and expanded cerebello-cerebral circuitry: a review

Human dexterity, bipedality, and song/speech vocalization in Homo are reviewed within a motor evolution perspective in regard to 

(i) brain expansion in cerebello-cerebral circuitry, 
(ii) enhanced predictive internal modeling of body kinematics, body kinetics and action organization, 
(iii) motor mastery due to prolonged practice, 
(iv) task-determined top-down, and accurately timed feedforward motor adjustment of multiple-body/artifact elements, and 
(v) reduction in automatic preflex/spinal reflex mechanisms that would otherwise restrict such top-down processes. 

Dual-task interference and developmental neuroimaging research argues that such internal modeling based motor capabilities are concomitant with the evolution of 
(vi) enhanced attentional, executive function and other high-level cognitive processes, and that 
(vii) these provide dexterity, bipedality and vocalization with effector nonspecific neural resources. 

The possibility is also raised that such neural resources could 
(viii) underlie human internal model based nonmotor cognitions. 
&#xa

Paralinguistic vocal control of interactive media: how untapped elements of voice might enhance the role of non-speech voice input in the user's experience of multimedia.

Much interactive media development, especially commercial development, implies the dominance of the visual modality, with sound as a limited supporting channel. The development of multimedia technologies such as augmented reality and virtual reality has further revealed a distinct partiality to visual media. Sound, however, and particularly voice, have many aspects which have yet to be adequately investigated. Exploration of these aspects may show that sound can, in some respects, be superior to graphics in creating immersive and expressive interactive experiences. With this in mind, this thesis investigates the use of non-speech voice characteristics as a complementary input mechanism in controlling multimedia applications. It presents a number of projects that employ the paralinguistic elements of voice as input to interactive media including both screen-based and physical systems. These projects are used as a means of exploring the factors that seem likely to affect users’ preferences and interaction patterns during non-speech voice control. This exploration forms the basis for an examination of potential roles for paralinguistic voice input. The research includes the conceptual and practical development of the projects and a set of evaluative studies. The work submitted for Ph.D. comprises practical projects (50 percent) and a written dissertation (50 percent). The thesis aims to advance understanding of how voice can be used both on its own and in combination with other input mechanisms in controlling multimedia applications. It offers a step forward in the attempts to integrate the paralinguistic components of voice as a complementary input mode to speech input applications in order to create a synergistic combination that might let the strengths of each mode overcome the weaknesses of the other

Understanding player experience in social digital games : the role of social presence

Including others in a playful activity fundamentally changes the concept of play into social play. This thesis focuses on social play in digital gaming. Previous studies have revealed the crucial part of social interaction in play, as the intrinsic need to belong seems to be the core motivation for engaging in such activities. Besides interacting with the game and focusing on the game content, gamers are confronted with emotions, behaviours, opinions and performances of others that can easily be perceived when playing side by side (co-located co-play). However, the widespread penetration of the Internet also allows for social play without the restriction of co-players having to be in the same room (mediated co-play). In online co-play settings a smaller amount of social information can be exchanged compared to co-located co-play. These settings therefore differ in the way gamers can interact with each other, which has an influence on how digital gaming is experienced. Focus groups, contextual inquiries, and four experimental studies were employed to uncover which aspects of social play in digital gaming make mediated and co-located co-play feel different. The first study was conducted to explore the motivations of gamers to physically meet others to play digital games, compared to meeting them online. Based on interpersonal communications, focus groups and contextual inquiries, findings revealed that the appeal of co-located above online co-play results from the sociable, warm, sensitive, personal and intimate interactions that are possible when playing side-byside. Furthermore, co-located co-play offers the possibility to experience social fun before and after game play, which often is not afforded in online co-play. The first lab experiment was conducted to empirically demonstrate to what extent player experience is influenced by the way co-players are present; e.g. as a virtual (human controlled), mediated or co-located co-player. Results on self-reports indicated that playing sidebyside significantly adds to the enjoyment and involvement in games compared to playing against a virtual (i.e. computer controlled) or distant co-player. These results could be explained by introducing the concept of social presence, which is defined as the feeling of being together with another individual. Results demonstrated that social presence mediated the enjoyment in social play. In the second and third lab experiment the possibilities for verbal and non-verbal interaction were manipulated between coplayers in co-located co-play. One experiment investigated this for competitive play; a the other for collaborative play. Self-reports showed that interactions through auditory cues positively influenced player experience. Interestingly, the presence of visual cues of one’s co-player had no significant influence on play. Observation data of players confirmed that audio cues (e.g., talking, laughing) were far more often used than visual cues (e.g., eye contact, making gestures); visual cues were only used before and after play. Furthermore, we demonstrated that social presence – due to interaction by audio cues – mediates the enjoyment in social play. A final lab experiment was conducted to investigate the importance of the connection between players through the game for our results in the previous chapters. In this experiment players were colocated, but not always played together and/or were not always able to see each other’s scores. Results showed that a subjective shared experience increases the strength of the social connection between players, and positively affects feelings of social presence, enjoyment and involvement. Similar to the previous findings, social presence mediated the increase in feelings of enjoyment and involvement. In sum, our set of studies offered empirical support for when, why and how social interaction influences players’ experience in co-play settings. Furthermore, the results indicate the importance of social presence as a mediating factor of enjoyment and involvement in social play. This provides new theoretical insights for communication experiences in other media, and social presence in general. Furthermore, findings may be useful to game designers who may want to enhance players’ experiences in during digital play

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion

Examining Emotional Responses to Effective Versus Ineffective Virtual Buddies

The purpose of this research study was to explore the impact of virtual character design on user emotional experience and user behavior in a simulated environment. With simulation training increasing in popularity as a tool for teaching social skills, it is essential that social interactions in virtual environments provide authentic opportunities for practice (Swartout et al., 2006). This study used Interactive Performance Theory (Wirth, 2012) to examine the effect of designing a virtual buddy character with ineffective traits instead of effective or expert traits. The sample population for this study (n = 145) consisted of first year university students enrolled in courses in the fall of 2013 at the University of Central Florida. Data on participant emotional experience and behavior were collected through questionnaires, researcher observations, and physiological signal recording that included participant heart rate and galvanic skin response. Data were analyzed using multivariate analysis of variances (MANOVA), Kruskal-Wallis one-way analysis of variance, and qualitative thematic coding of participant verbal behavior and written responses. Results of the analysis revealed that participants who interacted with an ineffective virtual buddy character had statistically significant higher averages of verbal statements to the antagonist in the simulated environment and statistically significant lower perceptions of antagonist amiability than participants who interacted with an effective virtual buddy. Additionally, participants who interacted with a virtual buddy of the opposite gender gave statistically significant higher ecological validity scores to the simulated environment than participants who interacted with a virtual buddy of the same gender. Qualitative analysis also revealed that participants tended to describe the female buddy character with more ineffective traits than the male buddy character even though effective and ineffective design conditions were equally divided for both groups. Further research should be conducted on the effect of virtual buddy character design in different types of simulation environments and with different target audiences