From presence to consciousness through virtual reality

Immersive virtual environments can break the deep, everyday connection between where our senses tell us we are and where we are actually located and whom we are with. The concept of 'presence' refers to the phenomenon of behaving and feeling as if we are in the virtual world created by computer displays. In this article, we argue that presence is worthy of study by neuroscientists, and that it might aid the study of perception and consciousness

The responses of people to virtual humans in an immersive virtual environment

This paper presents an experiment investigating the impact of behavior and responsiveness on social responses to virtual humans in an immersive virtual environment (IVE). A number of responses are investigated, including presence, copresence, and two physiological responses—heart rate and electrodermal activity (EDA). Our findings suggest that increasing agents’ responsiveness even on a simple level can have a significant impact on certain aspects of people’s social responses to humanoid agents. Despite being aware that the agents were computer-generated, participants with higher levels of social anxiety were significantly more likely to avoid “disturbing” them. This suggests that on some level people can respond to virtual humans as social actors even in the absence of complex interaction. Responses appear to be shaped both by the agents’ behaviors and by people’s expectations of the technology. Participants experienced a significantly higher sense of personal contact when the agents were visually responsive to them, as opposed to static or simply moving. However, this effect diminished with experienced computer users. Our preliminary analysis of objective heart-rate data reveals an identical pattern of responses

Development and evaluation of an interactive virtual audience for a public speaking training application

Einleitung: Eine der häufigsten sozialen Ängste ist die Angst vor öffentlichem Sprechen. Virtual-Reality- (VR-) Trainingsanwendungen sind ein vielversprechendes Instrument, um die Sprechangst zu reduzieren und die individuellen Sprachfähigkeiten zu verbessern. Grundvoraussetzung hierfür ist die Implementierung eines realistischen und interaktiven Sprecher-Publikum-Verhaltens. Ziel: Die Studie zielte darauf ab, ein realistisches und interaktives Publikum für eine VR-Anwendung zu entwickeln und zu bewerten, welches für die Trainingsanwendung von öffentlichem Sprechen angewendet wird. Zunächst wurde eine Beobachtungsstudie zu den Verhaltensmustern von Sprecher und Publikum durchgeführt. Anschließend wurden die identifizierten Muster in eine VR-Anwendung implementiert. Die Wahrnehmung der implementierten Interaktionsmuster wurde in einer weiteren Studie aus Sicht der Nutzer evaluiert. Beobachtungsstudie (1): Aufgrund der nicht ausreichenden Datengrundlage zum realen interaktiven Verhalten zwischen Sprecher und Publikum lautet die erste Forschungsfrage "Welche Sprecher-Publikums-Interaktionsmuster können im realen Umfeld identifiziert werden?". Es wurde eine strukturierte, nicht teilnehmende, offene Beobachtungsstudie durchgeführt. Ein reales Publikum wurde auf Video aufgezeichnet und die Inhalte analysiert. Die Stichprobe ergab N = 6484 beobachtete Interaktionsmuster. Es wurde festgestellt, dass Sprecher mehr Dialoge als das Publikum initiieren und wie die Zuschauer auf Gesichtsausdrücke und Gesten der Sprecher reagieren. Implementierungsstudie (2): Um effiziente Wege zur Implementierung der Ergebnisse der Beobachtungsstudie in die Trainingsanwendung zu finden, wurde die Forschungsfrage wie folgt formuliert: "Wie können Interaktionsmuster zwischen Sprecher und Publikum in eine virtuelle Anwendung implementiert werden?". Das Hardware-Setup bestand aus einer CAVE, Infitec-Brille und einem ART Head-Tracking. Die Software wurde mit 3D-Excite RTT DeltaGen 12.2 realisiert. Zur Beantwortung der zweiten Forschungsfrage wurden mehrere mögliche technische Lösungen systematisch untersucht, bis effiziente Lösungen gefunden wurden. Infolgedessen wurden die selbst erstellte Audioerkennung, die Kinect-Bewegungserkennung, die Affectiva-Gesichtserkennung und die selbst erstellten Fragen implementiert, um das interaktive Verhalten des Publikums in der Trainingsanwendung für öffentliches Sprechen zu realisieren. Evaluationsstudie (3): Um herauszufinden, ob die Implementierung interaktiver Verhaltensmuster den Erwartungen der Benutzer entsprach, wurde die dritte Forschungsfrage folgendermaßen formuliert: “Wie beeinflusst die Interaktivität einer virtuellen Anwendung für öffentliches Reden die Benutzererfahrung?”. Eine experimentelle Benutzer-Querschnittsstudie wurde mit N = 57 Teilnehmerinnen (65% Männer, 35% Frauen; Durchschnittsalter = 25.98, SD = 4.68) durchgeführt, die entweder der interaktiven oder nicht-interaktiven VR-Anwendung zugewiesen wurden. Die Ergebnisse zeigten, dass, es einen signifikanten Unterschied in der Wahrnehmung zwischen den beiden Anwendungen gab. Allgemeine Schlussfolgerungen: Interaktionsmuster zwischen Sprecher und Publikum, die im wirklichen Leben beobachtet werden können, wurden in eine VR-Anwendung integriert, die Menschen dabei hilft, Angst vor dem öffentlichen Sprechen zu überwinden und ihre öffentlichen Sprechfähigkeiten zu trainieren. Die Ergebnisse zeigten eine hohe Relevanz der VR-Anwendungen für die Simulation öffentlichen Sprechens. Obwohl die Fragen des Publikums manuell gesteuert wurden, konnte das neu gestaltete Publikum mit den Versuchspersonen interagieren. Die vorgestellte VR-Anwendung zeigt daher einen hohen potenziellen Nutzen, Menschen beim Trainieren von Sprechfähigkeiten zu unterstützen. Die Fragen des Publikums wurden immer noch manuell von einem Bediener reguliert und die Studie wurde mit Teilnehmern durchgeführt, die nicht unter einem hohen Grad an Angst vor öffentlichem Sprechen leiden. Bei zukünftigen Studien sollten fortschrittlichere Technologien eingesetzt werden, beispielsweise Spracherkennung, 3D-Aufzeichnungen oder 3D-Livestreams einer realen Person und auch Teilnehmer mit einem hohen Grad an Angst vor öffentlichen Ansprachen beziehungsweise Sprechen in der Öffentlichkeit.Introduction: Fear of public speaking is the most common social fear. Virtual reality (VR) training applications are a promising tool to improve public speaking skills. To be successful, applications should feature a high scenario fidelity. One way to improve it is to implement realistic speaker-audience interactive behavior. Objective: The study aimed to develop and evaluate a realistic and interactive audience for a VR public speaking training application. First, an observation study on real speaker-audience interactive behavior patterns was conducted. Second, identified patterns were implemented in the VR application. Finally, an evaluation study identified users’ perceptions of the training application. Observation Study (1): Because of the lack of data on real speaker-audience interactive behavior, the first research question to be answered was “What speaker-audience interaction patterns can be identified in real life?”. A structured, non-participant, overt observation study was conducted. A real audience was video recorded, and content analyzed. The sample resulted in N = 6,484 observed interaction patterns. It was found that speakers, more often than audience members, initiate dialogues and how audience members react to speakers’ facial expressions and gestures. Implementation Study (2): To find efficient ways of implementing the results of the observation study in the training application, the second research question was formulated as: “How can speaker-audience interaction patterns be implemented into the virtual public speaking application?”. The hardware setup comprised a CAVE, Infitec glasses, and ART head tracking. The software was realized with 3D-Excite RTT DeltaGen 12.2. To answer the second research question, several possible technical solutions were explored systematically, until efficient solutions were found. As a result, self-created audio recognition, Kinect motion recognition, Affectiva facial recognition, and manual question generation were implemented to provide interactive audience behavior in the public speaking training application. Evaluation Study (3): To find out if implementing interactive behavior patterns met users’ expectations, the third research question was formulated as “How does interactivity of a virtual public speaking application affect user experience?”. An experimental, cross-sectional user study was conducted with (N = 57) participants (65% men, 35% women; Mage = 25.98, SD = 4.68) who used either an interactive or a non-interactive VR application condition. Results revealed that there was a significant difference in users’ perception of the two conditions. General Conclusions: Speaker-audience interaction patterns that can be observed in real life were incorporated into a VR application that helps people to overcome the fear of public speaking and train their public speaking skills. The findings showed a high relevance of interactivity for VR public speaking applications. Although questions from the audience were still regulated manually, the newly designed audience could interact with the speakers. Thus, the presented VR application is of potential value in helping people to train their public speaking skills. The questions from the audience were still regulated manually by an operator and we conducted the study with participants not suffering from high degrees of public speaking fear. Future work may use more advanced technology, such as speech recognition, 3D-records, or live 3D-streams of an actual person and include participants with high degrees of public speaking fear

A Review into eHealth Services and Therapies: Potential for Virtual Therapeutic Communities - Supporting People with Severe Personality Disorder

eHealth has expanded hugely over the last fifteen years and continues to evolve, providing greater benefits for patients, health care professionals and providers alike. The technologies that support these systems have become increasingly more sophisticated and have progressed significantly from standard databases, used for patient records, to highly advanced Virtual Reality (VR) systems for the treatment of complex mental health illnesses. The scope of this paper is to initially explore e-Health, particularly in relation to technologies supporting the treatment and management of wellbeing in mental health. It then provides a case study of how technology in e-Health can lend itself to an application that could support and maintain the wellbeing of people with a severe mental illness. The case study uses Borderline Personality Disorder as an example, but could be applicable in many other areas, including depression, anxiety, addiction and PTSD. This type of application demonstrates how e-Health can empower the individuals using it but also potentially reducing the impact upon health care providers and services.Comment: Book chapte

Beyond the screen – The potential of smartphone apps and immersive technologies in exposure-based interventions for phobias

Specific phobias are extremely common among adults. They are characterized by strong emotional reactions and avoidance behavior when exposed to the feared stimuli. Specifically fears concerning heights or animals such as spiders are highly prevalent, followed by fear of social situations such as fear of public speaking. The gold standard in treating specific phobias is exposure-based therapy. However, exposure-based therapy is limited in its practicability in clinical routine and poses a high hurdle for affected individuals. Virtual and augmented reality (VR/AR) smartphone apps offer attractive platforms to simulate exposure situations and by that increase the accessibility of mental health services in general. Thus, novel smartphone-based treatments hold the potential to facilitate the dissemination of exposure-based treatments for specific phobias. The studies presented as part of this thesis aimed at investigating three newly developed interventions for fear of heights, fear of public speaking and fear of spiders, using the currently available advanced technologies. In the first study (Bentz et al., 2021), a stand-alone, automated and gamified VR exposure app Easyheights was developed using 360° images. The app’s effectiveness to reduce fear of heights and avoidance behavior was investigated in a randomized controlled trial in an adult population with clinical and subclinical fear of heights. The repeated use of the app led to reduced fear and avoidance behavior in a real-life situation on a tower. For the second study (Müller, Fehlmann et al., 2022), the developed stand-alone, automated and gamified VR exposure app Fearless Speech aimed at reducing public speaking anxiety (PSA) and avoidance of eye contact. A virtual audience with 360° videos was used for the exposure and gaze control for the eye contact training. The app was investigated in a randomized controlled trial in healthy adults with subclinical PSA. After the repeated use of the app, participants showed reduced fear and improved eye contact in a real-life speech situation. The third study (Zimmer et al., 2021) examined the developed stand-alone, automated and gamified AR exposure app Phobys. In comparison to VR, AR has only recently been introduced to clinical research. The app was designed to reduce fear, disgust and avoidance behavior in adults with clinical and subclinical fear of spiders. The results of the randomized controlled trial showed that repeatedly using the app led to reduced fear, disgust and avoidance behavior in a real-life situation with a real spider. The results of these studies support the potential of stand-alone, automated VR and AR interventions delivered through smartphone apps. The developed apps allow for a high-quality user experience with a highly realistic environment, gaze control for an easy navigation as well as the possibility of interaction. In addition, gamification elements foster engagement with the apps. All three investigated apps offer low-threshold and low-cost treatment for individuals affected by specific phobias. Testing the effectiveness of these newly developed apps in real-life settings sets them apart from previous studies. Hence, this thesis highlights the potential of using smartphone apps with immersive technologies to advance and disseminate exposure-based treatments for specific phobias

Art museums and the incorporation of virtual reality: Examining the impact of VR on spatial and social norms

Art museums implicate established spatial and social norms. The norms that shape these behaviours are not fixed, but rather subject to change as the sociality and physicality of these spaces continues to develop. In recent years, the re-emergence of virtual reality (VR) has led to this technology being incorporated into art museums in the form of VR-based exhibits. While a growing body of research now explores the various applications, uses and effects of VR, there is a notable dearth of studies examining the impact VR might be having on the spatial and social experience of art museums. This article, therefore, reports on an original research project designed to address these concerns. The project was conducted at Anise Gallery in London, United Kingdom, between June and July 2018 and focused on the multisensory, and VR-based, exhibition, Scents of Shad Thames. The research involved 19 semi-structured interviews with participants who had just experienced this exhibition. Drawing on scholarly literature that surrounds the spatial and social norms pertaining to art museums, this study advances along three lines. First, the research explores whether the inclusion of VR might alter the practice of people watching, which is endemic of this setting. Second, the research explores whether established ways of navigating the physical setting of art museums might influence how users approach the digital space of VR. Third, the research examines whether the incorporation of VR might produce a qualitatively different experience of the art museum as a shared social space

Virtual reality public speaking training: effectiveness and user technology acceptance

Public speaking is a fundamental task in many professional or personal situations. At the same time, there is widespread fear of it, and it takes practice to present well. Previous studies suggest that Virtual Reality Public Speaking Training (VRPST) offers a promising opportunity for this. However, studies evaluating objective and subjective indicators are lacking so far, and valid control conditions are missing in previous studies. We aimed to overcome these drawbacks. In our experiment, participants (N = 42) had the task of presenting a card game to a four-person audience using five provided PowerPoint slides within a time limit of 5 minutes. They prepared either using VRPST or using common self-directed preparation (control condition), being randomly assigned to a condition. Both groups were instructed to prepare for the task at home and given 30 min to learn the rules of the game and present them using the slides. The control group was given an additional 30 min to prepare individually for the presentation task at home. The experimental group received an additional 30-min VRPST session. This training session was done without specific feedback and the presentation was repeated three times. The quality of the rule explanation, the audience-assessed presentation quality, and the subjects� self-assessed presentation quality were measured. Our results indicate that the VRPST is effective. Subjects who completed the VRPST did a better job of explaining the rules and were better rated by the audience. In addition, the experimental subjects also tended to rate their presentation better in the VRPST condition. Further analyses of those participants who completed the VRPST show high technology acceptance. Our results show the VR training had a significant performance-enhancing effect and that participants would use the VRPST if it were available to them. It seems that practicing a presentation in VR is useful and even better than a conventional preparation

Attention and Social Cognition in Virtual Reality:The effect of engagement mode and character eye-gaze

Technical developments in virtual humans are manifest in modern character design. Specifically, eye gaze offers a significant aspect of such design. There is need to consider the contribution of participant control of engagement. In the current study, we manipulated participants’ engagement with an interactive virtual reality narrative called Coffee without Words. Participants sat over coffee opposite a character in a virtual café, where they waited for their bus to be repaired. We manipulated character eye-contact with the participant. For half the participants in each condition, the character made no eye-contact for the duration of the story. For the other half, the character responded to participant eye-gaze by making and holding eye contact in return. To explore how participant engagement interacted with this manipulation, half the participants in each condition were instructed to appraise their experience as an artefact (i.e., drawing attention to technical features), while the other half were introduced to the fictional character, the narrative, and the setting as though they were real. This study allowed us to explore the contributions of character features (interactivity through eye-gaze) and cognition (attention/engagement) to the participants’ perception of realism, feelings of presence, time duration, and the extent to which they engaged with the character and represented their mental states (Theory of Mind). Importantly it does so using a highly controlled yet ecologically valid virtual experience

Virtual reality public speaking training: effectiveness and user technology acceptance

Public speaking is a fundamental task in many professional or personal situations. At the same time, there is widespread fear of it, and it takes practice to present well. Previous studies suggest that Virtual Reality Public Speaking Training (VRPST) offers a promising opportunity for this. However, studies evaluating objective and subjective indicators are lacking so far, and valid control conditions are missing in previous studies. We aimed to overcome these drawbacks. In our experiment, participants (N = 42) had the task of presenting a card game to a four-person audience using five provided PowerPoint slides within a time limit of 5 minutes. They prepared either using VRPST or using common self-directed preparation (control condition), being randomly assigned to a condition. Both groups were instructed to prepare for the task at home and given 30 min to learn the rules of the game and present them using the slides. The control group was given an additional 30 min to prepare individually for the presentation task at home. The experimental group received an additional 30-min VRPST session. This training session was done without specific feedback and the presentation was repeated three times. The quality of the rule explanation, the audience-assessed presentation quality, and the subjects’ self-assessed presentation quality were measured. Our results indicate that the VRPST is effective. Subjects who completed the VRPST did a better job of explaining the rules and were better rated by the audience. In addition, the experimental subjects also tended to rate their presentation better in the VRPST condition. Further analyses of those participants who completed the VRPST show high technology acceptance. Our results show the VR training had a significant performance-enhancing effect and that participants would use the VRPST if it were available to them. It seems that practicing a presentation in VR is useful and even better than a conventional preparation

Incorporating Virtual Reality Training in an Introductory Public Speaking Course

This study presents the results of two studies using a virtual reality (VR) public-speaking training simulation as an instructional aid in a basic communication course. Results from the first study suggest that VR practice was associated with higher subsequent speech delivery grades in the course compared to no practice. However, VR practice did not reduce public speaking anxiety (PSA). In a follow-up study, VR practice was compared with other forms of lab-based practice including in front of a mirror and a recorded video session. All forms of lab practice (VR, mirror, or video) were associated with higher speech grades than no practice, but there were no differences between lab-practice conditions in terms of outcomes. Results are discussed in terms of adopting and using virtual public-speaking simulations in large undergraduate public-speaking courses