I am here - are you there? Sense of presence and implications for virtual world design

We use the language of presence and place when we interact online: in our instant text messaging windows we often post: Are you there? Research indicates the importance of the sense of presence for computer-supported collaborative virtual learning. To realize the potential of virtual worlds such as Second Life, which may have advantages over conventional text-based environments, we need an understanding of design and the emergence of the sense of presence. A construct was created for the sense of presence, as a collaborative, action-based process (Spagnolli, Varotto, & Mantovani, 2003) with four dimensions (sense of place, social presence, individual agency, and mediated collaborative actions). Nine design principles were mapped against the four dimensions. The guiding question for the study\u27s exploration of the sense of presence was: In the virtual world Second Life, what is the effect on the sense of presence in collaborative learning spaces designed according to the sense of presence construct proposed, using two of the nine design principles, wayfinding and annotation? Another question of interest was: What are the relationships, if any, among the four dimensions of presence? The research utilized both quantitative and qualitative measures. Twenty learners recruited from the Graduate School of Education and Psychology at Pepperdine University carried out three assigned collaborative activities in Second Life under design conditions foregrounding each of the two design conditions, and a combination of the two. Analyses from surveys, Second Life interactions, interviews and a focus group were conducted to investigate how various designed learning environments based in the virtual world contributed to the sense of presence, and to learners\u27 ability to carry out collaborative learning. The major research findings were: (a) the construct appears robust, and future research in its application to other virtual worlds may be fruitful; (b) the experience of wayfinding (finding a path through a virtual space) resulted overall in an observed pattern of a slightly stronger sense of place; (c) the experience of annotation (building) resulted overall in an observed pattern of a slightly stronger sense of agency; and (d) there is a positive association between sense of place and sense of agency

Social Interactions in Immersive Virtual Environments: People, Agents, and Avatars

Immersive virtual environments (IVEs) have received increased popularity with applications in many fields. IVEs aim to approximate real environments, and to make users react similarly to how they would in everyday life. An important use case is the users-virtual characters (VCs) interaction. We interact with other people every day, hence we expect others to appropriately act and behave, verbally and non-verbally (i.e., pitch, proximity, gaze, turn-taking). These expectations also apply to interactions with VCs in IVEs, and this thesis tackles some of these aspects. We present three projects that inform the area of social interactions with a VC in IVEs, focusing on non-verbal behaviours. In our first study on interactions between people, we collaborated with the Social Neuroscience group at the Institute of Cognitive Neuroscience from UCL on a dyad multi-modal interaction. This aims to understand the conversation dynamics, focusing on gaze and turn-taking. The results show that people have a higher frequency of gaze change (from averted to direct and vice versa) when they are being looked at compared to when they are not. When they are not being looked at, they are also directing their gaze to their partners more compared to when they are being looked at. Another contribution of this work is the automated method of annotating speech and gaze data. Next, we consider agents’ higher-level non-verbal behaviours, covering social attitudes. We present a pipeline to collect data and train a machine learning (ML) model that detects social attitudes in a user-VC interaction. Here we collaborated with two game studios: Dream Reality Interaction and Maze Theory. We present a case study for the ML pipeline on social engagement recognition for the Peaky Blinders narrative VR game from Maze Theory studio. We use a reinforcement learning algorithm with imitation learning rewards and a temporal memory element. The results show that the model trained with raw data does not generalise and performs worse (60% accuracy) than the one trained with socially meaningful data (83% accuracy). In IVEs, people embody avatars and their appearance can impact social interactions. In collaboration with Microsoft Research, we report a longitudinal study in mixed-reality on avatar appearance in real-work meetings between co-workers comparing personalised full-body realistic and cartoon avatars. The results imply that when participants use realistic avatars first, they may have higher expectations and they perceive their colleagues’ emotional states with less accuracy. Participants may also become more accustomed to cartoon avatars as time passes and the overall use of avatars may lead to less accurately perceiving negative emotions. The work presented here contributes towards the field of detecting and generating nonverbal cues for VCs in IVEs. These are also important building blocks for creating autonomous agents for IVEs. Additionally, this work contributes to the games and work industry fields through an immersive ML pipeline for detecting social attitudes and through insights into using different avatar styles over time in real-world meetings

Experimental studies of the interaction between people and virtual humans with a focus on social anxiety

Psychotherapy has been one of the major applications of Virtual Reality technology; examples include fear of flying, heights, spiders, and post‐traumatic stress disorder. Virtual reality has been shown to be useful, in the context of exposure therapy for the treatment of social anxiety, such as fear of public speaking, where the clients learn how to conquer their anxiety through interactions with Virtual Characters (avatars). This thesis is concerned with the interaction between human participants and avatars in a Virtual Environment (VE), with the main focus being on Social Anxiety. It is our hypothesis that interactions between people and avatars can evoke in people behaviours that correspond to their degree of social anxiety or confidence. Moreover the responses of people to avatars will also depend on their degree of exhibited social anxiety – they will react differently to a shy avatar compared to a confident avatar. The research started with an experimental study on the reaction of shy and confident male volunteers to an approach by an attractive and friendly virtual woman in a VE. The results show that the participants responded according to expectations towards the avatar at an emotional, physiological, and behavioural level. The research then studied a particular cue which represents shyness – “blushing”. Experiments were carried out on how participant responds towards a blushing avatar. The results suggested that, even without consciously noticing the avatar’s blushing, the participants had an improved relationship with her when she was blushing. Finally, the research further investigated how people respond towards a shy avatar as opposed to a confident one. The results show that participants gave more positive comments to the personality of the avatar displaying signs of shyness

The social AR continuum : wearable AR for sharing social experiences.

The primary goal of the work reported in this thesis is to explore, develop and evaluate novel interaction techniques for Augmented Reality (AR) wearable head- sets, by focusing on how they can be used to share social experiences with family and friends. AR has the potential to provide an intuitive and natural approach to sharing our social experiences in life with others while being co-present. In order to better visualise and interact with social networks on wearable AR devices, we intro- duce the concept of the "Social AR Continuum", which describes the space of sharing experiences in AR across various axes. We discuss the advantages and limitations of various implementations and techniques of shared social experiences on wearable AR. Based on Human-Computer Interaction methodologies, we conducted user studies to evaluate user presences and system usability of our implementations for visualising, sharing and interacting with social experiences on AR headsets. The work focused on the essential Social AR Continuum dimensions of representing contacts, sharing data and interactions, developed user interfaces on these dimensions and evaluated them using user studies. Our results show that using AR to share social experiences can increase users’ social presence. The results are summarised as design recommendations to help interface designers better design shared social experiences on wearable AR systems

Exploring Collaborative Music Making Experience in Shared Virtual Environments.

PhD ThesisVirtual Environments (VEs), as media providing high-level immersion, o er people an opportunity to mimic natural interpersonal interactions digitally. As a multi-player version of VEs, Shared Virtual Environments (SVEs) inherit VEs' advantages in enabling natural interactions and generating a high level of immersion, and will possibly play an increasingly important role in supporting digitally-mediated collaboration. Though SVEs have been extensively explored for education, entertainment, work, and training, as yet, few SVEs exist in the eld of supporting creative collaboration and as a result, research on the creative aspect of collaboration in SVEs remains very poor. This raises questions about how to design the user experience to support creative collaboration in SVEs. This thesis starts with an introduction and related work. An SVE called Let's Move (LeMo) will then be briefed. LeMo allows two people to interact with each other and create music collaboratively in its virtual environment. Three studies based on LeMo will then be presented: Study I explores how free-form visual 3D annotations and work identity in uence the collaboration, Study II and Study III explore how working space con gurations a ect the collaboration. Results indicate that: (1) 3D annotations can support people's collaborative music making (CMM) in SVEs through ve classes of use; (2) group territory, personal territory, and territorial behaviour emerge during collaborative music making in SVEs; (3) manipulating characteristics of personal space a ected collaborative behaviour, formation of territory, work e ciency, sense of contribution, preference, and so on. Then an overall discussion between studies is made and further implications for SVEs supporting collaborative music making (and other types of collaboration) in SVEs are given. The ndings of this thesis contribute towards the design of Human-Computer Interaction of Shared Virtual Environments focusing on supporting collaborative music making

Identifying national security fundamentals in the big data of digital asymmetrical environment communication

This conceptualization article discusses the impact of technology to society from variousperspectives which relates on humanities aspect of social convention, hegemony, generation warfare and intelligence constructs.The narratives provide understandings of the current transition, emphasizing the changing need of intelligence protocols to understand the society in current technological advancement era. Intelligence output is detrimental as information is no longer secluded in this borderless communication spheres. The article narrates generation warfare intelligence in complex information system of digital asymmetrical environment in current generation warfare to digest the social implications and of technology to society and provides appropriate recommendations.Keywords: communication; generation warfare; hegemony; intelligence; society; socialconvention

Sonic interactions in virtual environments

This book tackles the design of 3D spatial interactions in an audio-centered and audio-first perspective, providing the fundamental notions related to the creation and evaluation of immersive sonic experiences. The key elements that enhance the sensation of place in a virtual environment (VE) are: Immersive audio: the computational aspects of the acoustical-space properties of Virutal Reality (VR) technologies Sonic interaction: the human-computer interplay through auditory feedback in VE VR systems: naturally support multimodal integration, impacting different application domains Sonic Interactions in Virtual Environments will feature state-of-the-art research on real-time auralization, sonic interaction design in VR, quality of the experience in multimodal scenarios, and applications. Contributors and editors include interdisciplinary experts from the fields of computer science, engineering, acoustics, psychology, design, humanities, and beyond. Their mission is to shape an emerging new field of study at the intersection of sonic interaction design and immersive media, embracing an archipelago of existing research spread in different audio communities and to increase among the VR communities, researchers, and practitioners, the awareness of the importance of sonic elements when designing immersive environments