Perceiving Sociable Technology: Exploring the Role of Anthropomorphism and Agency Perception on Human-Computer Interaction (HCI)

With the arrival of personal assistants and other AI-enabled autonomous technologies, social interactions with smart devices have become a part of our daily lives. Therefore, it becomes increasingly important to understand how these social interactions emerge, and why users appear to be influenced by them. For this reason, I explore questions on what the antecedents and consequences of this phenomenon, known as anthropomorphism, are as described in the extant literature from fields ranging from information systems to social neuroscience. I critically analyze those empirical studies directly measuring anthropomorphism and those referring to it without a corresponding measurement. Through a grounded theory approach, I identify common themes and use them to develop models for the antecedents and consequences of anthropomorphism. The results suggest anthropomorphism possesses both conscious and non-conscious components with varying implications. While conscious attributions are shown to vary based on individual differences, non-conscious attributions emerge whenever a technology exhibits apparent reasoning such as through non-verbal behavior like peer-to-peer mirroring or verbal paralinguistic and backchanneling cues. Anthropomorphism has been shown to affect users’ self-perceptions, perceptions of the technology, how users interact with the technology, and the users’ performance. Examples include changes in a users’ trust on the technology, conformity effects, bonding, and displays of empathy. I argue these effects emerge from changes in users’ perceived agency, and their self- and social- identity similarly to interactions between humans. Afterwards, I critically examine current theories on anthropomorphism and present propositions about its nature based on the results of the empirical literature. Subsequently, I introduce a two-factor model of anthropomorphism that proposes how an individual anthropomorphizes a technology is dependent on how the technology was initially perceived (top-down and rational or bottom-up and automatic), and whether it exhibits a capacity for agency or experience. I propose that where a technology lays along this spectrum determines how individuals relates to it, creating shared agency effects, or changing the users’ social identity. For this reason, anthropomorphism is a powerful tool that can be leveraged to support future interactions with smart technologies

Connecting people through physiosocial technology

Social connectedness is one of the most important predictors of health and well-being. The goal of this dissertation is to investigate technologies that can support social connectedness. Such technologies can build upon the notion that disclosing emotional information has a strong positive influence on social connectedness. As physiological signals are strongly related to emotions, they might provide a solid base for emotion communication technologies. Moreover, physiological signals are largely lacking in unmediated communication, have been used successfully by machines to recognize emotions, and can be measured relatively unobtrusively with wearable sensors. Therefore, this doctoral dissertation examines the following research question: How can we use physiological signals in affective technology to improve social connectedness? First, a series of experiments was conducted to investigate if computer interpretations of physiological signals can be used to automatically communicate emotions and improve social connectedness (Chapters 2 and 3). The results of these experiments showed that computers can be more accurate at recognizing emotions than humans are. Physiological signals turned out to be the most effective information source for machine emotion recognition. One advantage of machine based emotion recognition for communication technology may be the increase in the rate at which emotions can be communicated. As expected, experiments showed that increases in the number of communicated emotions increased feelings of closeness between interacting people. Nonetheless, these effects on feelings of closeness are limited if users attribute the cause of the increases in communicated emotions to the technology and not to their interaction partner. Therefore, I discuss several possibilities to incorporate emotion recognition technologies in applications in such a way that users attribute the communication to their interaction partner. Instead of using machines to interpret physiological signals, the signals can also be represented to a user directly. This way, the interpretation of the signal is left to be done by the user. To explore this, I conducted several studies that employed heartbeat representations as a direct physiological communication signal. These studies showed that people can interpret such signals in terms of emotions (Chapter 4) and that perceiving someone's heartbeat increases feelings of closeness between the perceiver and sender of the signal (Chapter 5). Finally, we used a field study (Chapter 6) to investigate the potential of heartbeat communication mechanisms in practice. This again confirmed that heartbeat can provide an intimate connection to another person, showing the potential for communicating physiological signals directly to improve connectedness. The last part of the dissertation builds upon the notion that empathy has positive influences on social connectedness. Therefore, I developed a framework for empathic computing that employed automated empathy measurement based on physiological signals (Chapter 7). This framework was applied in a system that can train empathy (Chapter 8). The results showed that providing users frequent feedback about their physiological synchronization with others can help them to improve empathy as measured through self-report and physiological synchronization. In turn, this improves understanding of the other and helps people to signal validation and caring, which are types of communication that improve social connectedness. Taking the results presented in this dissertation together, I argue that physiological signals form a promising modality to apply in communication technology (Chapter 9). This dissertation provides a basis for future communication applications that aim to improve social connectedness

Measuring, analysing and artificially generating head nodding signals in dyadic social interaction

Social interaction involves rich and complex behaviours where verbal and non-verbal signals are exchanged in dynamic patterns. The aim of this thesis is to explore new ways of measuring and analysing interpersonal coordination as it naturally occurs in social interactions. Specifically, we want to understand what different types of head nods mean in different social contexts, how they are used during face-to-face dyadic conversation, and if they relate to memory and learning. Many current methods are limited by time-consuming and low-resolution data, which cannot capture the full richness of a dyadic social interaction. This thesis explores ways to demonstrate how high-resolution data in this area can give new insights into the study of social interaction. Furthermore, we also want to demonstrate the benefit of using virtual reality to artificially generate interpersonal coordination to test our hypotheses about the meaning of head nodding as a communicative signal. The first study aims to capture two patterns of head nodding signals – fast nods and slow nods – and determine what they mean and how they are used across different conversational contexts. We find that fast nodding signals receiving new information and has a different meaning than slow nods. The second study aims to investigate a link between memory and head nodding behaviour. This exploratory study provided initial hints that there might be a relationship, though further analyses were less clear. In the third study, we aim to test if interactive head nodding in virtual agents can be used to measure how much we like the virtual agent, and whether we learn better from virtual agents that we like. We find no causal link between memory performance and interactivity. In the fourth study, we perform a cross-experimental analysis of how the level of interactivity in different contexts (i.e., real, virtual, and video), impacts on memory and find clear differences between them

Three Essays on How Marketplace Interpersonal Relationships Affect Persuasion

The purpose of this dissertation is to investigate how marketplace interpersonal relationships affect the persuasiveness of marketing messages, specifically how consumer process and respond to marketing messages. I examine interpersonal relationship in the marketplace from three perspectives: consumer-marketer relationship (essay I), consumer-consumer relationship (essay II), and consumer-humanized product relationship (essay III). In the first essay, I examine how marketers can strategically use appreciation instead of apology after service delay to optimize the effectiveness of symbolic recovery. As an initial recovery effort after service delay, marketers need to decide “what to say” to consumers to restore their satisfaction. Prior work on service recovery suggests that marketers should employ an apology strategy (e.g., saying “Sorry about the delay”). In this article, I propose that an appreciation strategy (e.g., saying “Thank you for your patience”) is often more effective in restoring satisfaction. Drawing from research on linguistic framing and self-concept, I reason that such a subtle shift of focus in the marketer-consumer interaction, from emphasizing marketers’ mistake and accountability to spotlighting consumers’ merits and contribution, can increase consumers’ self-esteem and hence recovery satisfaction. Using various service delay contexts, including two real-world delay situations, I show that appreciation is more effective than apology in promoting recovery satisfaction (Studies 1-2). I further provide convergent evidence that the superiority of appreciation to apology is caused by consumers’ elevated self-esteem as a result of being thanked (Studies 3-5). I also identify two boundary conditions, severity of delay and obviousness of marketers’ fault, for the superior effect of appreciation, such that the superiority of appreciation disappears when the service delay is perceived to minor (Study 6) and that superiority of appreciation is reversed when marketers’ fault is obvious (Study 7). In the second essay, I examine the diverse effects of friend and family reminders on consumers’ regulatory focus and the persuasiveness of product appeals. Prior research suggests that close friends and family members exert similar effects on consumer behavior because both represent strong social ties and are subject to communal norms. However, on the basis of the auto-motive model and regulatory fit theory, I postulate that exposure to relationship reminders of close friends and family can actually have different impacts on consumers’ subsequent purchase decisions. Across four experiments, I demonstrate that exposure to relationship reminders of close friends increases purchase intentions toward products with promotion-focused appeals while exposure to relationship reminders of family members increases purchase intentions toward products with prevention-focused appeals. In the third essay, I examine how consumers view anthropomorphism in general. Specifically drawing from recent research on anthropomorphism and gender identity, I propose and attest to the identity-signaling function of anthropomorphism by examining the anthropomorphism–femininity association and its marketing implications. Eight studies provide convergent evidence for such an association. The pilot study shows that engaging in anthropomorphic activities and purchasing anthropomorphic products are positively associated with femininity. Studies 1 and 2 provide evidence for both causal directions of the anthropomorphism–femininity association by demonstrating that people perceive a feminine (vs. masculine) person as more likely to purchase anthropomorphic products and judge a person who owns anthropomorphic (vs. nonanthropomorphic) products as more likely to be a woman. Study 3 further examines the association by examining how recalling one’s own anthropomorphic activities influences self-perceived femininity. Study 4 provides direct evidence using an Implicit Association Test. Finally, studies 5ab and 6 demonstrate the implications of the anthropomorphism–femininity association from the perspective of masculinity maintenance and gift-giving, respectively

The Social Life of Metaphor

The five experiments in this dissertation examine the social effects of metaphor context production and comprehension. In Studies 1 and 2, participants wrote a meaningful discourse context for metaphorical or literal sentences. Participants providing context for metaphor used more idiomatic emotional expressions, cognitive mechanism words (e.g., “think”) and adverbs. Those responding to the literal prompts used physical descriptions. These results are interpreted in light of research that shows idiomatic expressions and cognitive mechanism words are used to express emotion and signal friendship. In Study 2, use of affective content in the metaphor condition was positively correlated with scores on the Reading the Mind in the Eyes task (Baron-Cohen et al., 2001). Participants in the metaphor group also scored higher on this task compared to the literal group. The Eyes findings show writers in the metaphor condition framed their context to engage an ostensive audience. Studies 3 and 4 consisted of reading short scenarios that ended with metaphorical or literal statements, followed by questions assessing social and emotional inferences of the participants. Participants also completed the Eyes task. Use of metaphor by characters in a story was perceived as more emotionally intense and suggestive of interpersonal closeness. Scores on the Eyes task positively and uniquely correlated with social variables (closeness and emotional intensity) when scenarios ended with metaphor, but not when they ended with literal statements. These correlations show those who perceived metaphor as socially informative were more accurate at identifying emotions in others. Study 5 tested the premise that even out of context, metaphor comprehension proceeds through inferences of an implicit intention (e.g., Katz, 2005; Ritchie, 2006). After reading metaphorical or literal sentences, the participants completed the Eyes task and a non-social, creativity task (wherein participants provided nouns in response to verb prompts). Participants who read metaphor did better on the Eyes task than those who read literal counterparts, supporting the claim that, even out of context, metaphor conveys an interpersonal intention. Additionally, compared to the literal group, participants in the metaphor group provided more “social” words in response to verb prompts. Results are discussed in light of embodied cognition

Affective Motivational Collaboration Theory

Existing computational theories of collaboration explain some of the important concepts underlying collaboration, e.g., the collaborators\u27 commitments and communication. However, the underlying processes required to dynamically maintain the elements of the collaboration structure are largely unexplained. Our main insight is that in many collaborative situations acknowledging or ignoring a collaborator\u27s affective state can facilitate or impede the progress of the collaboration. This implies that collaborative agents need to employ affect-related processes that (1) use the collaboration structure to evaluate the status of the collaboration, and (2) influence the collaboration structure when required. This thesis develops a new affect-driven computational framework to achieve these objectives and thus empower agents to be better collaborators. Contributions of this thesis are: (1) Affective Motivational Collaboration (AMC) theory, which incorporates appraisal processes into SharedPlans theory. (2) New computational appraisal algorithms based on collaboration structure. (3) Algorithms such as goal management, that use the output of appraisal to maintain collaboration structures. (4) Implementation of a computational system based on AMC theory. (5) Evaluation of AMC theory via two user studies to a) validate our appraisal algorithms, and b) investigate the overall functionality of our framework within an end-to-end system with a human and a robot