Detecting Low Rapport During Natural Interactions in Small Groups from Non-Verbal Behaviour

Rapport, the close and harmonious relationship in which interaction partners are "in sync" with each other, was shown to result in smoother social interactions, improved collaboration, and improved interpersonal outcomes. In this work, we are first to investigate automatic prediction of low rapport during natural interactions within small groups. This task is challenging given that rapport only manifests in subtle non-verbal signals that are, in addition, subject to influences of group dynamics as well as inter-personal idiosyncrasies. We record videos of unscripted discussions of three to four people using a multi-view camera system and microphones. We analyse a rich set of non-verbal signals for rapport detection, namely facial expressions, hand motion, gaze, speaker turns, and speech prosody. Using facial features, we can detect low rapport with an average precision of 0.7 (chance level at 0.25), while incorporating prior knowledge of participants' personalities can even achieve early prediction without a drop in performance. We further provide a detailed analysis of different feature sets and the amount of information contained in different temporal segments of the interactions.Comment: 12 pages, 6 figure

Social transmission of leadership preference:knowledge of group membership and partisan media reporting moderates perceptions of leadership ability from facial cues to competence and dominance

While first impressions of dominance and competence can influence leadership preference, social transmission of leadership preference has received little attention. The capacity to transmit, store and compute information has increased greatly over recent history, and the new media environment may encourage partisanship (i.e. ‘echo chambers’), misinformation and rumour spreading to support political and social causes and be conducive both to emotive writing and emotional contagion, which may shape voting behaviour. In our pre-registered experiment, we examined whether implicit associations between facial cues to dominance and competence (intelligence) and leadership ability are strengthened by partisan media and knowledge that leaders support or oppose us on a socio-political issue of personal importance. Social information, in general, reduced well-established implicit associations between facial cues and leadership ability. However, as predicted, social knowledge of group membership reduced preferences for facial cues to high dominance and intelligence in out-group leaders. In the opposite-direction to our original prediction, this ‘in-group bias’ was greater under less partisan versus partisan media, with partisan writing eliciting greater state anxiety across the sample. Partisanship also altered the salience of women’s facial appearance (i.e., cues to high dominance and intelligence) in out-group versus in-group leaders. Independent of the media environment, men and women displayed an in-group bias toward facial cues of dominance in same-sex leaders. Our findings reveal effects of minimal social information (facial appearance, group membership, media reporting) on leadership judgements, which may have implications for patterns of voting or socio-political behaviour at the local or national level

Modeling Language Characteristics of Leaders in Authoritarian Regimes over Decades

The present research investigated the linguistic patterns in the discourse of three prominent autocratic political leaders whose reigns lasted for multiple decades. The texts of Fidel Castro, Zedong Mao, and Hosni Mubarak were analyzed using computational linguistic methodologies and nonlinear modeling techniques to explore the temporal trajectory of formality over time. Specifically, this metric of formality increases with abstractness of words, syntactic complexity, cohesion (referential and deep), and the informational genre (as opposed to narrative). At the other end of the continuum, informal discourse tends to have concrete words, simple syntax, low cohesion and high narrativity. The findings are aligned with theoretically grounded hypotheses of aging and persuasion in hopes of identifying which most appropriately explains the formality of leaders’ political texts

Language as a marker of CEO transition and company performance

textAn increasing number of researchers are beginning to explore leadership effectiveness in the context of language. To gain a better understanding of what constitutes an effective leader, particularly in the context of transition (exiting or entering leadership role), the current project examined Chief Executive Officer (CEO) language use in quarterly conference calls and its' association with company performance. Three research questions were asked: 1) What language patterns are associated with an outgoing CEO versus an incoming CEO? 2) To what degree does CEO language change depending on whether company performance increases or decreases in the year prior to exiting tenure or subsequent to their entering tenure 3) To what degree does CEO language predict company performance and company performance predict language use? In order to answer these questions, language use in the question and answer portion of quarterly conference calls was examined for 215 companies in the year prior to old CEO departure and in the first year for new CEO. Computerized text analysis was used to examine language associated with self-focus, other-focus, and positive and negative affect. Results suggest that old and new CEOs use distinctive language patterns when they are entering and exiting their leadership positions. Language was found to predict company performance and company performance was found to predict language. The current project points to the power of language as a tool to explore leadership effectiveness in the context of transition. Specifically, language analysis can help identify degree of old CEO detachment and new CEO assimilation within their company. In addition, language can be used as a marker of company performance.Psycholog

Computational Sociolinguistics: A Survey

Language is a social phenomenon and variation is inherent to its social nature. Recently, there has been a surge of interest within the computational linguistics (CL) community in the social dimension of language. In this article we present a survey of the emerging field of "Computational Sociolinguistics" that reflects this increased interest. We aim to provide a comprehensive overview of CL research on sociolinguistic themes, featuring topics such as the relation between language and social identity, language use in social interaction and multilingual communication. Moreover, we demonstrate the potential for synergy between the research communities involved, by showing how the large-scale data-driven methods that are widely used in CL can complement existing sociolinguistic studies, and how sociolinguistics can inform and challenge the methods and assumptions employed in CL studies. We hope to convey the possible benefits of a closer collaboration between the two communities and conclude with a discussion of open challenges.Comment: To appear in Computational Linguistics. Accepted for publication: 18th February, 201

Sensing, interpreting, and anticipating human social behaviour in the real world

Low-level nonverbal social signals like glances, utterances, facial expressions and body language are central to human communicative situations and have been shown to be connected to important high-level constructs, such as emotions, turn-taking, rapport, or leadership. A prerequisite for the creation of social machines that are able to support humans in e.g. education, psychotherapy, or human resources is the ability to automatically sense, interpret, and anticipate human nonverbal behaviour. While promising results have been shown in controlled settings, automatically analysing unconstrained situations, e.g. in daily-life settings, remains challenging. Furthermore, anticipation of nonverbal behaviour in social situations is still largely unexplored. The goal of this thesis is to move closer to the vision of social machines in the real world. It makes fundamental contributions along the three dimensions of sensing, interpreting and anticipating nonverbal behaviour in social interactions. First, robust recognition of low-level nonverbal behaviour lays the groundwork for all further analysis steps. Advancing human visual behaviour sensing is especially relevant as the current state of the art is still not satisfactory in many daily-life situations. While many social interactions take place in groups, current methods for unsupervised eye contact detection can only handle dyadic interactions. We propose a novel unsupervised method for multi-person eye contact detection by exploiting the connection between gaze and speaking turns. Furthermore, we make use of mobile device engagement to address the problem of calibration drift that occurs in daily-life usage of mobile eye trackers. Second, we improve the interpretation of social signals in terms of higher level social behaviours. In particular, we propose the first dataset and method for emotion recognition from bodily expressions of freely moving, unaugmented dyads. Furthermore, we are the first to study low rapport detection in group interactions, as well as investigating a cross-dataset evaluation setting for the emergent leadership detection task. Third, human visual behaviour is special because it functions as a social signal and also determines what a person is seeing at a given moment in time. Being able to anticipate human gaze opens up the possibility for machines to more seamlessly share attention with humans, or to intervene in a timely manner if humans are about to overlook important aspects of the environment. We are the first to propose methods for the anticipation of eye contact in dyadic conversations, as well as in the context of mobile device interactions during daily life, thereby paving the way for interfaces that are able to proactively intervene and support interacting humans.Blick, Gesichtsausdrücke, Körpersprache, oder Prosodie spielen als nonverbale Signale eine zentrale Rolle in menschlicher Kommunikation. Sie wurden durch vielzählige Studien mit wichtigen Konzepten wie Emotionen, Sprecherwechsel, Führung, oder der Qualität des Verhältnisses zwischen zwei Personen in Verbindung gebracht. Damit Menschen effektiv während ihres täglichen sozialen Lebens von Maschinen unterstützt werden können, sind automatische Methoden zur Erkennung, Interpretation, und Antizipation von nonverbalem Verhalten notwendig. Obwohl die bisherige Forschung in kontrollierten Studien zu ermutigenden Ergebnissen gekommen ist, bleibt die automatische Analyse nonverbalen Verhaltens in weniger kontrollierten Situationen eine Herausforderung. Darüber hinaus existieren kaum Untersuchungen zur Antizipation von nonverbalem Verhalten in sozialen Situationen. Das Ziel dieser Arbeit ist, die Vision vom automatischen Verstehen sozialer Situationen ein Stück weit mehr Realität werden zu lassen. Diese Arbeit liefert wichtige Beiträge zur autmatischen Erkennung menschlichen Blickverhaltens in alltäglichen Situationen. Obwohl viele soziale Interaktionen in Gruppen stattfinden, existieren unüberwachte Methoden zur Augenkontakterkennung bisher lediglich für dyadische Interaktionen. Wir stellen einen neuen Ansatz zur Augenkontakterkennung in Gruppen vor, welcher ohne manuelle Annotationen auskommt, indem er sich den statistischen Zusammenhang zwischen Blick- und Sprechverhalten zu Nutze macht. Tägliche Aktivitäten sind eine Herausforderung für Geräte zur mobile Augenbewegungsmessung, da Verschiebungen dieser Geräte zur Verschlechterung ihrer Kalibrierung führen können. In dieser Arbeit verwenden wir Nutzerverhalten an mobilen Endgeräten, um den Effekt solcher Verschiebungen zu korrigieren. Neben der Erkennung verbessert diese Arbeit auch die Interpretation sozialer Signale. Wir veröffentlichen den ersten Datensatz sowie die erste Methode zur Emotionserkennung in dyadischen Interaktionen ohne den Einsatz spezialisierter Ausrüstung. Außerdem stellen wir die erste Studie zur automatischen Erkennung mangelnder Verbundenheit in Gruppeninteraktionen vor, und führen die erste datensatzübergreifende Evaluierung zur Detektion von sich entwickelndem Führungsverhalten durch. Zum Abschluss der Arbeit präsentieren wir die ersten Ansätze zur Antizipation von Blickverhalten in sozialen Interaktionen. Blickverhalten hat die besondere Eigenschaft, dass es sowohl als soziales Signal als auch der Ausrichtung der visuellen Wahrnehmung dient. Somit eröffnet die Fähigkeit zur Antizipation von Blickverhalten Maschinen die Möglichkeit, sich sowohl nahtloser in soziale Interaktionen einzufügen, als auch Menschen zu warnen, wenn diese Gefahr laufen wichtige Aspekte der Umgebung zu übersehen. Wir präsentieren Methoden zur Antizipation von Blickverhalten im Kontext der Interaktion mit mobilen Endgeräten während täglicher Aktivitäten, als auch während dyadischer Interaktionen mittels Videotelefonie