Predicting video-conferencing conversation outcomes based on modeling facial expression synchronization

Effective video-conferencing conversations are heavily influenced by each speaker's facial expression. In this study, we propose a novel probabilistic model to represent interactional synchrony of conversation partners' facial expressions in video-conferencing communication. In particular, we use a hidden Markov model (HMM) to capture temporal properties of each speaker's facial expression sequence. Based on the assumption of mutual influence between conversation partners, we couple their HMMs as two interacting processes. Furthermore, we summarize the multiple coupled HMMs with a stochastic process prior to discover a set of facial synchronization templates shared among the multiple conversation pairs. We validate the model, by utilizing the exhibition of these facial synchronization templates to predict the outcomes of video-conferencing conversations. The dataset includes 75 video-conferencing conversations from 150 Amazon Mechanical Turkers in the context of a new recruit negotiation. The results show that our proposed model achieves higher accuracy in predicting negotiation winners than support vector machine and canonical HMMs. Further analysis indicates that some synchronized nonverbal templates contribute more in predicting the negotiation outcomes

Capturing Upper Body Motion in Conversation: an Appearance Quasi-Invariant Approach

We address the problem of body communication retrieval and measuring in seated conversations by means of marker-less motion capture. In psychological studies, the use of au-tomatic methods is key to reduce the subjectivity present in manual behavioral coding used to extract these cues. These studies usually involve hundreds of subjects with different clothing, non-acted poses, or different distances to the cam-era in uncalibrated, RGB-only video. However, range cam-eras are not yet common in psychology research, especially in existing recordings. Therefore, it becomes highly relevant to develop a fast method that is able to work in these con-ditions. Given the known relationship between depth and motion estimates, we propose to robustly integrate highly appearance-invariant image motion features in a machine learning approach, complemented with an effective tracking scheme. We evaluate the method’s performance with exist-ing databases and a database of upper body poses displayed in job interviews that we make public, showing that in our scenario it is comparable to that of Kinect without using a range camera, and state-of-the-art w.r.t. the HumanEva and ChaLearn 2011 evaluation datasets

Automated analysis of interactional synchrony using robust facial tracking and expression recognition

In this paper, we propose an automated, data-driven and unobtrusive framework to analyze interactional synchrony. We use this information to determine whether interpersonal synchrony can be an indicator of deceit. Our framework includes a robust facial tracking module, an effective expression recognition method, synchrony feature extraction and feature selection methods. These synchrony features are used to learn classification models for the deception recognition. To evaluate our proposed framework, we have conducted extensive experiments on a database of 242 video samples. We validate the performance of each technical module in our framework, and also show that these synchrony features are very effective at detecting deception.This item from the UA Faculty Publications collection is made available by the University of Arizona with support from the University of Arizona Libraries. If you have questions, please contact us at [email protected]

Temporal interaction patterns in negotiations

This dissertation focuses on temporal interaction patterns in negotiations that have previously been neglected and examines their impact on the subsequent interaction and on the negotiated outcome. Although negotiations are defined as social interactions, there is still relatively little understanding of the observable interaction patterns that actually develop in negotiations. It requires time-consuming coding efforts and interaction patterns are challenging to analyze. However, studying negotiation behavior from an interaction-based perspective is crucial, as behavioral antecedents can be significantly more important in the prediction of subsequent behaviors in an interaction process than interindividual difference and contextual variables. Therefore, the studies presented in this dissertation contribute to a more comprehensive understanding of temporal interaction patterns in negotiation. Specifically, we study the occurrence of active listening (patterns) and their effect on negotiation outcomes, behavioral antecedents and consequences of (dis-)honest behavior, and effects of behavior announcement patterns on negotiation outcomes. The results of these studies contribute to negotiation theory but are also of high practical value. We provide concrete and readily applicable advice on the use of active listening, on the use and promotion of honest behavior and the inhibition of dishonest behavior that should improve practitioners’ negotiation interactions and outcomes

Temporal interaction patterns in negotiations

This dissertation focuses on temporal interaction patterns in negotiations that have previously been neglected and examines their impact on the subsequent interaction and on the negotiated outcome. Although negotiations are defined as social interactions, there is still relatively little understanding of the observable interaction patterns that actually develop in negotiations. It requires time-consuming coding efforts and interaction patterns are challenging to analyze. However, studying negotiation behavior from an interaction-based perspective is crucial, as behavioral antecedents can be significantly more important in the prediction of subsequent behaviors in an interaction process than interindividual difference and contextual variables. Therefore, the studies presented in this dissertation contribute to a more comprehensive understanding of temporal interaction patterns in negotiation. Specifically, we study the occurrence of active listening (patterns) and their effect on negotiation outcomes, behavioral antecedents and consequences of (dis-)honest behavior, and effects of behavior announcement patterns on negotiation outcomes. The results of these studies contribute to negotiation theory but are also of high practical value. We provide concrete and readily applicable advice on the use of active listening, on the use and promotion of honest behavior and the inhibition of dishonest behavior that should improve practitioners’ negotiation interactions and outcomes