Time to Smile: How Onset Asynchronies Between Reciprocal Facial Expressions Influence the Experience of Responsiveness of a Virtual Agent

Human social interactions are highly coordinated in time and involve the reciprocal exchange of facial emotional expressions. While timing has been identified as an important factor in social communication via gaze or speech, so far there has been no systematic investigation on how the temporal delays in the exchange of facial emotional expressions between interactive partners influence the experience of responsiveness. In the present study, 40 participants were cued to direct a facial emotional expression (angry or happy) towards a virtual agent in front of them and the virtual agent would then react with a congruent facial expression. The temporal delay between the cue and the reaction was manipulated as an independent variable. Exact delays between facial expressions were determined based on the onset latencies of participants’ facial expressions measured via facial EMG (M. Zygomaticus and M. Corrugator). Ratings of experienced responsiveness of the virtual agent were collected as a dependent measure. Using a linear mixed effect model in a growth curve analysis revealed that experienced responsiveness peaked at delays around 700 ms. Moreover, experienced responsiveness at shorter temporal delays was higher when happy versus angry facial expressions were exchanged, while the reversed pattern was found at longer delays. Our results demonstrate a crucial role of timing in non-verbal communication, suggesting that temporal relations between facial expressions are processed as social information. Furthermore, our results can inform the implementation of virtual social interactions

Face masks reduce interpersonal distance in virtual reality

During the COVID-19 pandemic several behavioral measures have been implemented to reduce viral transmission. While these measures reduce the risk of infections, they may also increase risk behavior. Here, we experimentally investigate the influence of face masks on physical distancing. Eighty-four participants with or without face masks passed virtual agents in a supermarket environment to reach a target while interpersonal distance was recorded. Agents differed in wearing face masks and age (young, elderly). In addition, situational constraints varied in whether keeping a distance of 1.5 m required an effortful detour or not. Wearing face masks (both self and other) reduced physical distancing. This reduction was most prominent when keeping the recommended distance was effortful, suggesting an influence of situational constraints. Similarly, increased distances to elderly were only observed when keeping a recommended distance was effortless. These findings highlight contextual constraints in compensation behavior and have important implications for safety policies

Cigarette craving in virtual reality cue exposure in abstainers and relapsed smokers

Cue exposure therapy (CET) in substance-use disorders aims to reduce craving and ultimately relapse rates. Applying CET in virtual reality (VR) was proposed to increase its efficacy, as VR enables the presentation of social and environmental cues along with substance-related stimuli. However, limited success has been reported so far when applying VR-CET for smoking cessation. Understanding if effects of VR-CET differ between future abstainers and relapsing smokers may help to improve VR-CET. Data from 102 participants allocated to the intervention arm (VR-CET) of a recent RCT comparing VR-CET to relaxation in the context of smoking cessation was analyzed with respect to tolerability, presence, and craving during VR-CET. Cue exposure was conducted in four VR contexts (Loneliness/Rumination, Party, Stress, Café), each presented twice. Relapsed smokers compared to abstainers experienced higher craving during VR-CET and stronger craving responses especially during the Stress scenario. Furthermore, lower mean craving during VR-CET positively predicted abstinence at 6-month follow-up. Attempts to improve smoking cessation outcomes of VR-CET should aim to identify smokers who are more at risk of relapse based on high craving levels during VR-CET. Specifically measuring craving responses during social stress seems to be well suited to mark relapse. We propose to investigate individualized treatment approaches accordingly

Interpersonal Distance During Real-Time Social Interaction: Insights From Subjective Experience, Behavior, and Physiology

Physical distance is a prominent feature in face-to-face social interactions and allows regulating social encounters. Close interpersonal distance (IPD) increases emotional responses during interaction and has been related to avoidance behavior in social anxiety. However, a systematic investigation of the effects of IPD on subjective experience combined with measures of physiological arousal and behavioral responses during real-time social interaction has been missing. Virtual Reality allows for a controlled manipulation of IPD while maintaining naturalistic social encounters. The present study investigates IPD in social interaction using a novel paradigm in Virtual Reality. Thirty-six participants approached virtual agents and engaged in short interactions. IPD was varied between 3.5 and 1 m by manipulating the distance at which agents reacted to the participant's approach. Closer distances were rated as more arousing, less pleasant, and less natural than longer distances and this effect was significantly modulated by social anxiety scores. Skin conductance responses were also increased at short distances compared to longer distances. Finally, an interaction of IPD and social anxiety was observed for avoidance behavior, measured as participants' backward motion during interaction, with stronger avoidance related to close distances and high values of social anxiety. These results highlight the influence of IPD on experience, physiological response, and behavior during social interaction. The interaction of social anxiety and IPD suggests including the manipulation of IPD in behavioral tests in Virtual Reality as a promising tool for the treatment of social anxiety disorder

Social Presence as a Moderator of the Effect of Agent Behavior on Emotional Experience in Social Interactions in Virtual Reality

Background: Exposure therapy involves exposure to feared stimuli and is considered to be the gold-standard treatment for anxiety disorders. While its application in Virtual Reality (VR) has been very successful for phobic disorders, the effects of exposure to virtual social stimuli in Social Anxiety Disorder are heterogeneous. This difference has been linked to demands on realism and presence, particularly social presence, as a pre-requisite in evoking emotional experiences in virtual social interactions. So far, however, the influence of social presence on emotional experience in social interactions with virtual agents remains unknown. Objective: We investigated the relationship between realism and social presence and the moderating effect of social presence on the relationship between agent behavior and experienced emotions in virtual social interaction. Methods: Healthy participants (N = 51) faced virtual agents showing supportive and dismissive behaviors in two virtual environments (short interactions and oral presentations). At first, participants performed five blocks of short one-on-one interactions with virtual agents (two male and two female agents per block). Secondly, participants gave five presentations in front of an audience of 16 agents. In each scenario, agent behavior was a within subjects factor, resulting in one block of neutral, two blocks of negative, and two blocks of positive agent behavior. Ratings of agent behavior (valence and realism), experience (valence and arousal), and presence (physical and social) were collected after every block. Moderator effects were investigated using mixed linear models with random intercepts. Correlations were analyzed via repeated measures correlations. Results: Ratings of valence of agent behaviors showed reliable relationships with experienced valence and less reliable relationships with experienced arousal. These relationships were moderated by social presence in the presentation scenario. Results for the interaction scenario were weaker but potentially promising for experimental studies. Variations in social presence and realism over time were correlated but social presence proved a more reliable moderator. Conclusion: Our findings emphasize the role of social presence for emotional experience in response to specific agent behaviors in virtual social interactions. While these findings should be replicated with experimental designs and in clinical samples, variability in social presence might account for heterogeneity in efficacy of virtual exposure to treat social anxiety disorder

Angry facial expressions bias towards aversive actions

Social interaction requires fast and efficient processing of another person’s intentions. In face-to-face interactions, aversive or appetitive actions typically co-occur with emotional expressions, allowing an observer to anticipate action intentions. In the present study, we investigated the influence of facial emotions on the processing of action intentions. Thirty-two participants were presented with video clips showing virtual agents displaying a facial emotion (angry vs. happy) while performing an action (punch vs. fist-bump) directed towards the observer. During each trial, video clips stopped at varying durations of the unfolding action, and participants had to recognize the presented action. Naturally, participants’ recognition accuracy improved with increasing duration of the unfolding actions. Interestingly, while facial emotions did not influence accuracy, there was a significant influence on participants’ action judgements. Participants were more likely to judge a presented action as a punch when agents showed an angry compared to a happy facial emotion. This effect was more pronounced in short video clips, showing only the beginning of an unfolding action, than in long video clips, showing near-complete actions. These results suggest that facial emotions influence anticipatory processing of action intentions allowing for fast and adaptive responses in social interactions

Public speaking training in front of a supportive audience in Virtual Reality improves performance in real-life

Public speaking is a challenging task that requires practice. Virtual Reality allows to present realistic public speaking scenarios in this regard, however, the role of the virtual audience during practice remains unknown. In the present study, 73 participants completed a Virtual Reality practice session while audience was manipulated to be supportive or unsupportive or presentations were practiced without audience. Importantly, following the virtual practice, participants held the presentation during a real university course via Zoom. We measured emotional experience, self-efficacy, and the subjective evaluation of performance at baseline, after VR practice, and after the real presentation. Additionally, participants’ performance in the real presentation was evaluated by instructors (blinded to condition). Supportive in contrast to unsupportive audiences led to more positive believes about one’s own performance, while there were no changes in beliefs in the group without audience. Importantly, practice in front of a supportive compared to unsupportive audience resulted in a more positive evaluation of speaker confidence in real-life public speaking as rated by the instructors. These results demonstrate an impact of virtual social feedback during public speaking on subsequent subjective performance evaluation. This may increase self-confidence resulting in actual improved public speaking performance in real-life

Public speaking training in front of a supportive audience in Virtual Reality improves performance in real-life

Abstract Public speaking is a challenging task that requires practice. Virtual Reality allows to present realistic public speaking scenarios in this regard, however, the role of the virtual audience during practice remains unknown. In the present study, 73 participants completed a Virtual Reality practice session while audience was manipulated to be supportive or unsupportive or presentations were practiced without audience. Importantly, following the virtual practice, participants held the presentation during a real university course via Zoom. We measured emotional experience, self-efficacy, and the subjective evaluation of performance at baseline, after VR practice, and after the real presentation. Additionally, participants’ performance in the real presentation was evaluated by instructors (blinded to condition). Supportive in contrast to unsupportive audiences led to more positive believes about one’s own performance, while there were no changes in beliefs in the group without audience. Importantly, practice in front of a supportive compared to unsupportive audience resulted in a more positive evaluation of speaker confidence in real-life public speaking as rated by the instructors. These results demonstrate an impact of virtual social feedback during public speaking on subsequent subjective performance evaluation. This may increase self-confidence resulting in actual improved public speaking performance in real-life