A second chance to make a first impression? How appearance and nonverbal behavior affect perceived warmth and competence of virtual agents over time

Bergmann K, Eyssel FA, Kopp S. A second chance to make a first impression? How appearance and nonverbal behavior affect perceived warmth and competence of virtual agents over time. In: Walker M, Neff M, Paiva A, Nakano Y, eds. Proceedings of the 12th International Conference on Intelligent Virtual Agents. Berlin/Heidelberg: Springer; 2012: 126-138.First impressions of others are fundamental for the further develop- ment of a relationship and are thus of major importance for the design of vir- tual agents, too. We addressed the question whether there is a second chance for first impressions with regard to the major dimensions of social cognition–warmth and competence. We employed a novel experimental set-up that combined agent appearance (robot-like vs. human-like) and agent behavior (gestures present vs. absent) of virtual agents as between-subject factors with a repeated measures de- sign. Results indicate that ratings of warmth depend on interaction effects of time and agent appearance, while evaluations of competence seem to depend on the interaction of time and nonverbal behavior. Implications of these results for basic and applied research on intelligent virtual agents will be discussed

Trust in interdependent and task-oriented human-computer cooperation

Kulms P. Trust in interdependent and task-oriented human-computer cooperation. Bielefeld: Universität Bielefeld; 2018.This thesis presents a new paradigm for the modeling of cooperative human–computer interaction in order to evaluate the antecedents, formation, and regulation of human–computer trust. Human–computer trust is the degree to which human users trust computers to help them achieve their goals, and functions as powerful psychological variable that governs user behavior. The modeling framework presented in this thesis aims to extend predominant methods for the study of trust and cooperation by building on competent problemsolving and equal goal contributions by users and computers. Specifically, the framework permits users to participate in interactive and interdependent decision-making games with autonomous computer agents. The main task is to solve a two-dimensional puzzle, similar to the popular game Tetris. The games derived from this framework include cooperative interaction factors known from interpersonal cooperation: the duality of competence and selfishness, anthropomorphism, task advice, and social blame. One validation study (68 participants) and four experiments (318 participants) investigate how these cooperative interaction factors influence human–computer trust. In particular, the results show how trust in computers is mediated by warmth as universal dimension of social cognition, how anthropomorphism of computers influences trust formation over time, and how expressive anthropomorphic cues can be used to regulate trust. We explain how these findings can be applied to design trustworthy computer agents for successful cooperation

The devil is in the details: The effect of nonverbal cues on crowdfunding success

Many reward-based crowdfunding platforms encourage entrepreneurs to introduce their projects and make a personal appeal with a video clip. In this study, we investigate the impact of such a pitch video on financing outcomes. Grounded in social perception literature, we propose that effective use of nonverbal cues in a pitch video increases funding success. We coded and analyzed videos of crowdfunding campaigns and found that an entrepreneur could improve the funding outcomes by gazing less, appearing early, and reducing speech hesitations in a pitch video. We also found that smiling has no impact on funding success

Three essays on likability factors, crowdfunding, and entrepreneurial performance

In this dissertation, I conduct three empirical studies exploring the relation between likability factors, crowdfunding characteristics and entrepreneurial performance. Together these studies integrate aspects of major entrepreneurial likability factors including liking of the entrepreneur (source attractiveness, credibility, personal traits) and liking of the message (verbal content and expression), and components of nonverbal and verbal cues. I apply computer-mediated communication (CMC) and persuasion theories, political and marketing literature to provide a more fine-grained understanding of likability on crowdfunding success. In the first essay, I study how the non-verbal cues of a crowdfunding video influence the crowdfunding success. By employing social presence theory, I argue, hypothesize and test that effective use of non-verbal cues in a pitch video increases funding success. In the second essay, I explore how verbal cues (readability and complexity) and non-verbal cues (smiling and professional attire) interact to influence crowdfunding outcome. Findings of this essay indicate that powerful persuasion results from both expression (verbal cues) and impression (non-verbal cues). The third essay examines the mediating effect of likability between nonverbal, verbal cues and crowdfunding success. According to the likability factors extracted from political and advertising campaign literature, I conclude five main dimensions of likability in crowdfunding context. The results show that message factors are more influential than source factors in affecting crowdfunding outcome. Findings of three essays show that entrepreneurs should be careful to deliver a message which is immediate, simple, informative, humorous, storytelling and less complimentary to their funders. The more their messages are liked, the more likely funders will back their projects, and then the more success their crowdfunding campaign will be

by

th

Influence of warmth and competence on the promotion of safe in-group selection. Stereotype content model and social categorization of faces

Categorizing an individual as a friend or foe plays a pivotal role in navigating the social world. According to the Stereotype Content Model, social perception relies on two fundamental dimensions, Warmth and Competence, which allow us to process the intentions of others and their ability to enact those intentions, respectively. Social cognition research indicates that, in categorization tasks, people tend to classify other individuals as more likely to belong to the out-group than the in-group (In-group Overexclusion Effect, IOE) when lacking diagnostic information, probably with the aim of protecting in-group integrity. Here, we explored the role of Warmth and Competence in group-membership decisions by testing 62 participants in a social-categorization task consisting of 150 neutral faces. We assessed whether (i) Warmth and Competence ratings could predict the in-group/out-group categorization, and (ii) the reliance on these two dimensions differed in low-IOE vs. high-IOE participants. Data showed that high ratings of Warmth and Competence were necessary to categorize a face as in-group. Moreover, while low-IOE participants relied on Warmth, high-IOE participants relied on Competence. This finding suggests that the proneness to include/exclude unknown identities in/from one's own in-group is related to individual differences in the reliance on SCM social dimensions. Furthermore, the primacy of Warmth effect seems not to represent a universal phenomenon adopted in the context of social evaluatio

Managing an agent's self-presentational strategies during an interaction

In this paper we present a computational model for managing the impressions of warmth and competence (the two fundamental dimensions of social cognition) of an Embodied Conversational Agent (ECA) while interacting with a human. The ECA can choose among four different self-presentational strategies eliciting different impressions of warmth and/or competence in the user, through its verbal and non-verbal behavior. The choice of the non-verbal behaviors displayed by the ECA relies on our previous studies. In our first study, we annotated videos of human-human natural interactions of an expert on a given topic talking to a novice, in order to find associations between the warmth and competence elicited by the expert's non-verbal behaviors (such as type of gestures, arms rest poses, smiling). In a second study, we investigated whether the most relevant non-verbal cues found in the previous study were perceived in the same way when displayed by an ECA. The computational learning model presented in this paper aims to learn in real-time the best strategy (i.e., the degree of warmth and/or competence to display) for the ECA, that is, the one which maximizes user's engagement during the interaction. We also present an evaluation study, aiming to investigate our model in a real context. In the experimental scenario, the ECA plays the role of a museum guide introducing an exposition about video games. We collected data from 75 visitors of a science museum. The ECA was displayed in human dimension on a big screen in front of the participant, with a Kinect on the top. During the interaction, the ECA could adopt one of 4 self-presentational strategies during the whole interaction, or it could select one strategy randomly for each speaking turn, or it could use a reinforcement learning algorithm to choose the strategy having the highest reward (i.e., user's engagement) after each speaking turn

Revealing the latent structure of multidimensional facial features that drive social trait perceptions

Humans readily attribute social traits to others based on their facial appearance, influencing behaviors, social interactions, and decision making. According to influential models of social perception, these judgments are based on two central dimensions – trustworthiness/warmth and dominance/competence. Because of the wide-reaching impact of such social attributions, a long-standing focus has been to identify the facial features that elicit these social judgments. However, the face is complex, comprising 3D shape, 2D complexion, and dynamic facial expressions, making such investigations empirically challenging. As a result, central questions regarding fundamental facial features of social traits, and how these relate to other social judgments such as social class and emotion, remain unanswered. In this thesis, I use data-driven psychophysical methods to mathematically model the 3D shape, 2D complexion and dynamic facial expressions that elicit judgments of four key social trait dimensions – dominance, competence, trustworthiness, and warmth. I then identify the latent face feature space underlying these judgments using a data-reduction technique. Results reveal two latent 3D shape and three latent 2D complexion feature spaces on the basis of which social traits cluster into four distinct subgroups. Moreover, I show that these social trait feature spaces correlate positively with facial expression features (shape) and age-cues (complexion). I then examine whether these features also drive perceptions of another important social judgment – social class. To do so, I model the 3D shape and 2D complexion of social class, using the same approach as for social traits. I compare these models using a data reduction technique and a supervised machine learning approach. Results show that in line with conceptual overlaps arising from social class stereotypes (e.g., poor = incompetent), social class and social trait dimensions share facial features. However, no single trait’s features fully account for social class features. Finally, I compare the social trait facial expression models to emotional expressions and social trait face shapes to reveal the features shared between each. Results showed that longstanding associations between perception of emotion and social traits (e.g., happy = trustworthy), only partially account for social trait perception. The current work informs central theories of social perception, highlights drivers of socially relevant stereotype associations, and can aid the development of psychologically grounded digital agents

Social Perception of Pedestrians and Virtual Agents Using Movement Features

In many tasks such as navigation in a shared space, humans explicitly or implicitly estimate social information related to the emotions, dominance, and friendliness of other humans around them. This social perception is critical in predicting others’ motions or actions and deciding how to interact with them. Therefore, modeling social perception is an important problem for robotics, autonomous vehicle navigation, and VR and AR applications. In this thesis, we present novel, data-driven models for the social perception of pedestrians and virtual agents based on their movement cues, including gaits, gestures, gazing, and trajectories. We use deep learning techniques (e.g., LSTMs) along with biomechanics to compute the gait features and combine them with local motion models to compute the trajectory features. Furthermore, we compute the gesture and gaze representations using psychological characteristics. We describe novel mappings between these computed gaits, gestures, gazing, and trajectory features and the various components (emotions, dominance, friendliness, approachability, and deception) of social perception. Our resulting data-driven models can identify the dominance, deception, and emotion of pedestrians from videos with an accuracy of more than 80%. We also release new datasets to evaluate these methods. We apply our data-driven models to socially-aware robot navigation and the navigation of autonomous vehicles among pedestrians. Our method generates robot movement based on pedestrians’ dominance levels, resulting in higher rapport and comfort. We also apply our data-driven models to simulate virtual agents with desired emotions, dominance, and friendliness. We perform user studies and show that our data-driven models significantly increase the user’s sense of social presence in VR and AR environments compared to the baseline methods.Doctor of Philosoph