Multimodal Dyadic Impression Recognition via Listener Adaptive Cross-Domain Fusion

As a sub-branch of affective computing, impression recognition, e.g., perception of speaker characteristics such as warmth or competence, is potentially a critical part of both human-human conversations and spoken dialogue systems. Most research has studied impressions only from the behaviors expressed by the speaker or the response from the listener, yet ignored their latent connection. In this paper, we perform impression recognition using a proposed listener adaptive cross-domain architecture, which consists of a listener adaptation function to model the causality between speaker and listener behaviors and a cross-domain fusion function to strengthen their connection. The experimental evaluation on the dyadic IMPRESSION dataset verified the efficacy of our method, producing concordance correlation coefficients of 78.8% and 77.5% in the competence and warmth dimensions, outperforming previous studies. The proposed method is expected to be generalized to similar dyadic interaction scenarios.Comment: Accepted to ICASSP2023. arXiv admin note: substantial text overlap with arXiv:2203.1393

Managing an agent's self-presentational strategies during an interaction

In this paper we present a computational model for managing the impressions of warmth and competence (the two fundamental dimensions of social cognition) of an Embodied Conversational Agent (ECA) while interacting with a human. The ECA can choose among four different self-presentational strategies eliciting different impressions of warmth and/or competence in the user, through its verbal and non-verbal behavior. The choice of the non-verbal behaviors displayed by the ECA relies on our previous studies. In our first study, we annotated videos of human-human natural interactions of an expert on a given topic talking to a novice, in order to find associations between the warmth and competence elicited by the expert's non-verbal behaviors (such as type of gestures, arms rest poses, smiling). In a second study, we investigated whether the most relevant non-verbal cues found in the previous study were perceived in the same way when displayed by an ECA. The computational learning model presented in this paper aims to learn in real-time the best strategy (i.e., the degree of warmth and/or competence to display) for the ECA, that is, the one which maximizes user's engagement during the interaction. We also present an evaluation study, aiming to investigate our model in a real context. In the experimental scenario, the ECA plays the role of a museum guide introducing an exposition about video games. We collected data from 75 visitors of a science museum. The ECA was displayed in human dimension on a big screen in front of the participant, with a Kinect on the top. During the interaction, the ECA could adopt one of 4 self-presentational strategies during the whole interaction, or it could select one strategy randomly for each speaking turn, or it could use a reinforcement learning algorithm to choose the strategy having the highest reward (i.e., user's engagement) after each speaking turn

The Perceived Impact of the Use of African American Vernacular English in a Sales Situation

According to William Raspberry “Good English, well-spoken and written, will open more doors than a college degree. Bad English will slam doors you didn’t even know existed.” This dissertation examines the use of Standard American English (SAE) vis-à-vis African American Vernacular English (AAVE) in a sales situation. This dissertation is grounded in the Stereotype Content Model (SCM) and its two dimensions, warmth and competence. This research reveals the perceived influence that a particular dialect has on stereotype activation, through warmth and competence in a personal selling situation. This study extends marketing and sales literature by specifically focusing on AAVE vis-à-vis SAE in a sales situation to better understand how language activates the perceived warmth and competence of consumers and managers and in turn impacts cognitive, affective, and behavioral outcomes. This dissertation analyzes SCM and its effect on cognitive, affective, and behavioral outcomes through two studies representing the consumer perspective and managerial perspective. Communication plays a significant role in the exchange process that occurs in a sales context and in a hiring situation involving a sales context. Language can provide cues about the speaker’s ethnicity, geographic location, and educational background. Based on the salesperson’s presentation, the consumer and manager can determine whether he/she would like to initiate a business relationship. Unfamiliarity with the use of AAVE vis-à-vis SAE as well as difficulty with processing the information being conveyed can prompt consumers and managers to base their perceptions on biases and stereotypes. The consumer perspective examined the impact of AAVE vis-à-vis SAE on the activation of warmth and competence. The activation of warmth and competence was examined in regard to its influence on the consumer’s attitude toward the brand, evaluations of the salesperson, and purchase intention. The manager perspective examines how the use of AAVE vis-à-vis SAE activates the dimensions of warmth and competence. The activation of warmth and competence was examined for its effect on the manager’s attitude toward the salesperson, evaluations of the salesperson, and hiring recommendation. The results of the experiment indicate the need for a salesperson who wants to increase the likelihood of career success to speak SAE in order to obtain a more favorable rating from both customers and managers. Overall, this study advances the marketing literature by revealing practical and theoretical implications surrounding salespersons’ linguistic ability and their overall success in the sales arena. The results of this study demonstrate that SAE speakers are perceived as being both warmer and more competent than speakers of AAVE. The findings of this dissertation extend the knowledge regarding how language, specifically AAVE vis-à-vis SAE, can influence consumer and manager perceptions of salespersons in a sales situation

Analytics and Intuition in the Process of Selecting Talent

In management, decisions are expected to be based on rational analytics rather than intuition. But intuition, as a human evolutionary achievement, offers wisdom that, despite all the advances in rational analytics and AI, should be used constructively when recruiting and winning personnel. Integrating these inner experiential competencies with rational-analytical procedures leads to smart recruiting decisions

THE ARCHITECTURE AND DEVELOPMENT OF MINDREADING: BELIEFS, PERSPECTIVES, AND CHARACTER

This dissertation puts forward a series of arguments and theoretical proposals about the architecture and development of the human capacity to reason about the internal, psychological causes of behavior, known as “theory of mind” or “mindreading.” Chapter 1, “Foundations and motivations,” begins by articulating the philosophical underpinnings of contemporary theory-of-mind debates, especially the dispute between empiricists and nativists. I then argue for a nativist approach to theory-of-mind development, and then go on to outline how the subsequent chapters each address specific challenges for this nativist perspective. Chapter 2, “Pragmatic development and the false-belief task,” addresses the central puzzle of the theory-of-mind development literature: why is it that children below the age of five fail standard false-belief tasks, and yet are able to pass implicit versions of the false-belief task at a far younger age? According to my novel, nativist account, while they possess the concept of BELIEF very early in development, children’s early experiences with the pragmatics of belief discourse initially distort the way they interpret standard false-belief tasks; as children gain the relevant experience from their social and linguistic environment, this distortion eventually dissipates. In the Appendix (co-authored with Peter Carruthers), I expand upon this proposal to show how it can also account for another set of phenomena typically cited as evidence against nativism: the Theory-of-Mind Scale. Chapter 3, “Spontaneous mindreading: A problem for the two-systems account,” challenges the “two-systems” account of mindreading, which provides a different explanation for the implicit/explicit false-belief task gap, and has implications for the architecture of mature, adult mindreading. Using evidence from adults’ perspective-taking abilities I argue that this account is theoretically and empirically unsound. Chapter 4, “Character and theory of mind: An integrative approach,” begins by noting that contemporary accounts of mindreading neglect to account for the role of character or personality-trait representations in action-prediction and interpretation. Employing a hierarchical, predictive coding approach, I propose that character-trait representations are rapidly inferred in order to inform and constrain our mental-state attributions. Because this is a “covering concept” dissertation, each of these chapters (including the Appendix) is written so that it is independent of all of the others; they can be read in any order, and do not presuppose one another

Analytics and Intuition in the Process of Selecting Talent

In management, decisions are expected to be based on rational analytics rather than intuition. But intuition, as a human evolutionary achievement, offers wisdom that, despite all the advances in rational analytics and AI, should be used constructively when recruiting and winning personnel. Integrating these inner experiential competencies with rational-analytical procedures leads to smart recruiting decisions

The Influence of Entrepreneurs’ Physical and Cognitive Traits on Entrepreneurial Outcomes

The dissertation examines the effect of two separate yet significant individual level factors—physical and cognitive traits—on entrepreneurial outcomes. The physical factor on which the dissertation focuses is the entrepreneur’s vocal cues and the cognitive traits that the dissertation investigates are the entrepreneur’s goal orientation and his/her ability to take perspectives. Using theories of information processing and expectancy violations, the dissertation examines whether and how vocal attractiveness impacts individual investors’ information retention, attraction towards the venture, and willingness to invest. The results from controlled lab experiments across two samples suggest that vocal attractiveness increases information retention and venture attraction and the effect of vocal attractiveness on entrepreneurial outcomes is mediated through processing fluency and positive affect. Also, the effects of vocal attractiveness on processing fluency and positive affect is conditional on whether the investors’ expectations regarding the entrepreneurs’ vocal attractiveness is positively or negatively violated. Thus, entrepreneurs’ vocal attractiveness facilitates information retention and leads to higher levels of venture attraction through cognitive and affective routes and investors’ expectations play an important role in their decision-making. The dissertation also examines the impact of entrepreneurs’ sex and gendered voice on entrepreneurial outcomes. Entrepreneurs often portray their masculinity or femininity through gendered voice. Using the gender role stereotyping theory, the dissertation identifies sex-typed services or products offered by the entrepreneurs as another source of information that investors can readily access and incorporate in their decision-making. Contrary to the extant literature, the dissertation finds that wholesale adoption of masculinity during pitching entrepreneurial ventures is detrimental, especially for female entrepreneurs. The results from a controlled lab experiment suggest that, investors are more likely to invest when entrepreneurs present a masculine (feminine) service in a masculine (feminine) voice, irrespective of their biological sex. Thus, investors don’t always prefer masculinity over femininity in entrepreneurial pitching, rather, investors’ decisions are impacted by the interactions of entrepreneurs’ sex, gendered voice, and sex-typed services or products offered by the entrepreneurs. Finally, the dissertation examines the effect of cognitive traits on entrepreneurial creativity. Creativity lies at the heart of entrepreneurship as entrepreneurs often start with a creative idea that they turn into a viable venture. Using the goal orientation theory, the dissertation suggests that goal orientation is an antecedent of entrepreneurial creativity and the effect of goal orientation on entrepreneurial creativity is partially mediated through entrepreneurs’ ability to take users’ perspectives. In addition, the indirect effects of perspective taking on entrepreneurial creativity is contingent on individual-level moderators such as entrepreneurial exhaustion and experience. The results from two cross-sectional surveys of entrepreneurship students and a two-wave, lagged survey of entrepreneurs suggest that perspective taking mediates the relationship between goal orientation and entrepreneurial creativity and the indirect effects of perspective taking is conditional on the personal-level moderators. Thus, the dissertation disentangles the role of cognitive processes in facilitating entrepreneurial creativity

Supervision Satisfaction: What Impacts Masters Level Supervision When Reviewing Supervisor Category and Student Concentration?

Supervision Satisfaction: What Impacts Masters Level Supervision When Reviewing Supervisor Category and Student Concentration

Creating and Using the Persona in Teaching: Challenges of Connection and Control

Because teaching is inherently interpersonal and relational work, teachers use expressive behaviors such as voice, body language, and facial expression as they interact in the classroom. Yet the effects of the expressive dimensions of teachers’ practice on their relationships with children and their instruction are not well-understood in education. This dissertation investigates and conceptualizes this expressive domain in teaching, which it refers to as “creating and using a persona,” or “persona work.” This study draws on classroom observations, teacher interviews, and student focus groups and surveys to explore patterns in teachers’ persona work and unearth its purposes. Specifically, it employs multiple case study analysis to describe the expressive practice of six White, female, experienced and “expert” teachers. It also shows how the 220 students across these teachers’ diverse middle school English language arts and social studies classrooms responded to their persona work. Taken together, findings from this dissertation show that the teachers’ persona work was central to their instruction and relationships with children, and that it had the power to create as well as limit opportunities for children. The teachers in this study used their expressive behaviors to control and shape interactional conditions in the classroom. Their persona work helped teachers engage children and maintain their attention, lent clarity to teachers’ explanations, communicated teachers’ expectations, and otherwise augmented teachers’ instructional and relational goals. However, teachers’ persona work was not always successful, and did not always benefit every child. In particular, especially among children of color, teachers’ persona work could also be inequitable and could communicate a lack of care or intellectual regard. As this study shows, although teachers’ persona work might help some children learn and engage, it can also limit other children’s opportunities in classrooms, especially among students from historically marginalized backgrounds. This study has important ramifications for teaching and teacher education, especially in relation to cross-cultural teaching contexts. Without growing teachers’ abilities to create and use personas in the classroom in ways that are just, equitable, and responsive to all children, the field continues to relegate to chance teachers’ mastery over this ubiquitous, influential, but until now underdeveloped domain of teaching practice. This, in turn, will continue to put young people—and especially children of color—dangerously at risk.PHDEducational StudiesUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/144032/1/marthacp_1.pd