2,291 research outputs found
Emergent Leadership Detection Across Datasets
Automatic detection of emergent leaders in small groups from nonverbal
behaviour is a growing research topic in social signal processing but existing
methods were evaluated on single datasets -- an unrealistic assumption for
real-world applications in which systems are required to also work in settings
unseen at training time. It therefore remains unclear whether current methods
for emergent leadership detection generalise to similar but new settings and to
which extent. To overcome this limitation, we are the first to study a
cross-dataset evaluation setting for the emergent leadership detection task. We
provide evaluations for within- and cross-dataset prediction using two current
datasets (PAVIS and MPIIGroupInteraction), as well as an investigation on the
robustness of commonly used feature channels (visual focus of attention, body
pose, facial action units, speaking activity) and online prediction in the
cross-dataset setting. Our evaluations show that using pose and eye contact
based features, cross-dataset prediction is possible with an accuracy of 0.68,
as such providing another important piece of the puzzle towards emergent
leadership detection in the real world.Comment: 5 pages, 3 figure
Investigating Social Interactions Using Multi-Modal Nonverbal Features
Every day, humans are involved in social situations and interplays, with the goal of
sharing emotions and thoughts, establishing relationships with or acting on other
human beings. These interactions are possible thanks to what is called social intelligence,
which is the ability to express and recognize social signals produced during
the interactions. These signals aid the information exchange and are expressed
through verbal and non-verbal behavioral cues, such as facial expressions, gestures,
body pose or prosody. Recently, many works have demonstrated that social signals
can be captured and analyzed by automatic systems, giving birth to a relatively
new research area called social signal processing, which aims at replicating human
social intelligence with machines. In this thesis, we explore the use of behavioral
cues and computational methods for modeling and understanding social interactions.
Concretely, we focus on several behavioral cues in three specic contexts:
rst, we analyze the relationship between gaze and leadership in small group interactions.
Second, we expand our analysis to face and head gestures in the context of
deception detection in dyadic interactions. Finally, we analyze the whole body for
group detection in mingling scenarios
Detecting Low Rapport During Natural Interactions in Small Groups from Non-Verbal Behaviour
Rapport, the close and harmonious relationship in which interaction partners
are "in sync" with each other, was shown to result in smoother social
interactions, improved collaboration, and improved interpersonal outcomes. In
this work, we are first to investigate automatic prediction of low rapport
during natural interactions within small groups. This task is challenging given
that rapport only manifests in subtle non-verbal signals that are, in addition,
subject to influences of group dynamics as well as inter-personal
idiosyncrasies. We record videos of unscripted discussions of three to four
people using a multi-view camera system and microphones. We analyse a rich set
of non-verbal signals for rapport detection, namely facial expressions, hand
motion, gaze, speaker turns, and speech prosody. Using facial features, we can
detect low rapport with an average precision of 0.7 (chance level at 0.25),
while incorporating prior knowledge of participants' personalities can even
achieve early prediction without a drop in performance. We further provide a
detailed analysis of different feature sets and the amount of information
contained in different temporal segments of the interactions.Comment: 12 pages, 6 figure
Sensing, interpreting, and anticipating human social behaviour in the real world
Low-level nonverbal social signals like glances, utterances, facial expressions and body language are central to human communicative situations and have been shown to be connected to important high-level constructs, such as emotions, turn-taking, rapport, or leadership. A prerequisite for the creation of social machines that are able to support humans in e.g. education, psychotherapy, or human resources is the ability to automatically sense, interpret, and anticipate human nonverbal behaviour. While promising results have been shown in controlled settings, automatically analysing unconstrained situations, e.g. in daily-life settings, remains challenging. Furthermore, anticipation of nonverbal behaviour in social situations is still largely unexplored. The goal of this thesis is to move closer to the vision of social machines in the real world. It makes fundamental contributions along the three dimensions of sensing, interpreting and anticipating nonverbal behaviour in social interactions. First, robust recognition of low-level nonverbal behaviour lays the groundwork for all further analysis steps. Advancing human visual behaviour sensing is especially relevant as the current state of the art is still not satisfactory in many daily-life situations. While many social interactions take place in groups, current methods for unsupervised eye contact detection can only handle dyadic interactions. We propose a novel unsupervised method for multi-person eye contact detection by exploiting the connection between gaze and speaking turns. Furthermore, we make use of mobile device engagement to address the problem of calibration drift that occurs in daily-life usage of mobile eye trackers. Second, we improve the interpretation of social signals in terms of higher level social behaviours. In particular, we propose the first dataset and method for emotion recognition from bodily expressions of freely moving, unaugmented dyads. Furthermore, we are the first to study low rapport detection in group interactions, as well as investigating a cross-dataset evaluation setting for the emergent leadership detection task. Third, human visual behaviour is special because it functions as a social signal and also determines what a person is seeing at a given moment in time. Being able to anticipate human gaze opens up the possibility for machines to more seamlessly share attention with humans, or to intervene in a timely manner if humans are about to overlook important aspects of the environment. We are the first to propose methods for the anticipation of eye contact in dyadic conversations, as well as in the context of mobile device interactions during daily life, thereby paving the way for interfaces that are able to proactively intervene and support interacting humans.Blick, Gesichtsausdrücke, Körpersprache, oder Prosodie spielen als nonverbale Signale eine zentrale Rolle in menschlicher Kommunikation. Sie wurden durch vielzählige Studien mit wichtigen Konzepten wie Emotionen, Sprecherwechsel, Führung, oder der Qualität des Verhältnisses zwischen zwei Personen in Verbindung gebracht. Damit Menschen effektiv während ihres täglichen sozialen Lebens von Maschinen unterstützt werden können, sind automatische Methoden zur Erkennung, Interpretation, und Antizipation von nonverbalem Verhalten notwendig. Obwohl die bisherige Forschung in kontrollierten Studien zu ermutigenden Ergebnissen gekommen ist, bleibt die automatische Analyse nonverbalen Verhaltens in weniger kontrollierten Situationen eine Herausforderung. Darüber hinaus existieren kaum Untersuchungen zur Antizipation von nonverbalem Verhalten in sozialen Situationen. Das Ziel dieser Arbeit ist, die Vision vom automatischen Verstehen sozialer Situationen ein Stück weit mehr Realität werden zu lassen. Diese Arbeit liefert wichtige Beiträge zur autmatischen Erkennung menschlichen Blickverhaltens in alltäglichen Situationen. Obwohl viele soziale Interaktionen in Gruppen stattfinden, existieren unüberwachte Methoden zur Augenkontakterkennung bisher lediglich für dyadische Interaktionen. Wir stellen einen neuen Ansatz zur Augenkontakterkennung in Gruppen vor, welcher ohne manuelle Annotationen auskommt, indem er sich den statistischen Zusammenhang zwischen Blick- und Sprechverhalten zu Nutze macht. Tägliche Aktivitäten sind eine Herausforderung für Geräte zur mobile Augenbewegungsmessung, da Verschiebungen dieser Geräte zur Verschlechterung ihrer Kalibrierung führen können. In dieser Arbeit verwenden wir Nutzerverhalten an mobilen Endgeräten, um den Effekt solcher Verschiebungen zu korrigieren. Neben der Erkennung verbessert diese Arbeit auch die Interpretation sozialer Signale. Wir veröffentlichen den ersten Datensatz sowie die erste Methode zur Emotionserkennung in dyadischen Interaktionen ohne den Einsatz spezialisierter Ausrüstung. Außerdem stellen wir die erste Studie zur automatischen Erkennung mangelnder Verbundenheit in Gruppeninteraktionen vor, und führen die erste datensatzübergreifende Evaluierung zur Detektion von sich entwickelndem Führungsverhalten durch. Zum Abschluss der Arbeit präsentieren wir die ersten Ansätze zur Antizipation von Blickverhalten in sozialen Interaktionen. Blickverhalten hat die besondere Eigenschaft, dass es sowohl als soziales Signal als auch der Ausrichtung der visuellen Wahrnehmung dient. Somit eröffnet die Fähigkeit zur Antizipation von Blickverhalten Maschinen die Möglichkeit, sich sowohl nahtloser in soziale Interaktionen einzufügen, als auch Menschen zu warnen, wenn diese Gefahr laufen wichtige Aspekte der Umgebung zu übersehen. Wir präsentieren Methoden zur Antizipation von Blickverhalten im Kontext der Interaktion mit mobilen Endgeräten während täglicher Aktivitäten, als auch während dyadischer Interaktionen mittels Videotelefonie
Tracking The Leader: Gaze Behaviour In Group Interactions
Can social gaze behaviour reveal the leader during real-world group interactions? To answer this question, we developed a novel tripartite approach combining i) computer vision methods for remote gaze estimation, ii) a detailed taxonomy to encode the implicit semantics of multi-party gaze features, and iii) machine learning methods to establish dependencies between leadership and visual behaviours. We found that social gaze behaviour distinctively identified group leaders. Crucially, the relationship between leadership and gaze behaviour generalized across democratic and autocratic leadership styles under conditions of low and high time-pressure, suggesting that gaze can serve as a general marker of leadership. These findings provide the first direct evidence that group visual patterns can reveal leadership across different social behaviours and validate a new promising method for monitoring natural group interactions
Multimodality in Group Communication Research
Team interactions are often multisensory, requiring members to pick up on
verbal, visual, spatial and body language cues. Multimodal research, research
that captures multiple modes of communication such as audio and visual signals,
is therefore integral to understanding these multisensory group communication
processes. This type of research has gained traction in biomedical engineering
and neuroscience, but it is unclear the extent to which communication and
management researchers conduct multimodal research. Our study finds that
despite its' utility, multimodal research is underutilized in the communication
and management literature's. This paper then covers introductory guidelines for
creating new multimodal research including considerations for sensors, data
integration and ethical considerations.Comment: 27 pages, 3 figure
Análise computacional de aspetos de comunicação não verbal em contextos de grupo
Human communication is a major field of study in psychology and social sciences.
Topics such as emergent leadership and group dynamics are commonly studied
cases when referring to group settings. Experiments regarding group settings are
usually analyzed in conversational and collaborative tasks environments in order to
study the communication process in small groups.
Former studies’ methods involve human analysis and manual annotation of others’
behaviors in communication settings. Later studies try to replace time consuming
and failure prone annotations by resorting to computational methods.
Having a custom, newly-gathered audiovisual dataset, from an experiment conducted
by the Department of Education and Psychology of the University of Aveiro,
a multidisciplinary group from the same institution with members from psychology
and engineering backgrounds, took the initiative to create computational methods
in order to facilitate the analysis of the collected data.
For that purpose, this work presents a multimodal computational framework using
state-of-the-art methods in computer vision, capable of enriching image data with
annotations of a broad range of nonverbal communication aspects, both at an
individual and group levels, thus facilitating the study of nonverbal communication
and group dynamics.
This works contributes to the community by presenting methods to directly increase
human knowledge about the human communication process, involving data
transformation processes in order to transform raw feature data into humanly understandable
meanings and a visualization tool capable of visualizing such methods
applied to the input data.A comunicação humana é uma grande área de estudo na psicologia e ciências
sociais. Temas como liderança emergente e dinâmicas de grupo são temas frequentemente
estudados quando se estudam contextos de grupo. Dentro da área
de estudos sobre contextos de grupo analisam-se situações de conversação e realização
de tarefas colaborativas em grupos de pequena dimensão.
Estudos primordiais envolviam análise e anotação humana para a anotação dos
comportamentos revelados nas experiências realizadas, equanto que estudos mais
recentes tendem a adotar métodos computacionais de forma a susbtituir os métodos
anteriormente usados por serem dispendiosos em termos de tempo e propícios a
erros.
Tendo como caso de estudo um conjunto de dados audiovisuais de uma experiência
realizada pelo Departamento de Educação e Psicologia da Universidade de Aveiro,
um grupo de investigação multidisciplinar das áreas da psicologia e engenharia
tomou a iniciativa de desenvolver métodos computacionais capazes de facilitar o
processo de análise dos dados recolhidos.
Como tal, este trabalho apresenta uma abordagem computacional multimodal,
utilizando métodos "Estado da arte", capaz de enriquecer os dados visuais com
anotações de uma larga extensão de aspetos de comunicação não-verbal, tanto
a nível individual como de grupo, facilitando assim o estudo da comunicação em
geral e das dinâmicas de grupo.
Este trabalho contribui para a comunidade, fornecendo métodos para aumentar
o conhecimento existente sobre o processo de comunicação humana, incluindo
processos de transformação de dados, desde dados numéricos de baixa interpretação
para informação interpretável e compreensível, assim como uma ferramenta de
visualização capaz de apresentar tais métodos aplicados aos dados de entrada.Mestrado em Engenharia Informátic
Stepping toward Collective Mindsets: An Investigation of Group- and Leader-based Synchrony in Work Teams
The keys to creating effective team performance have long been under investigation by researchers. Past research identifies social cohesion as an important precursor, but how to achieve social cohesion is lesser understood. This dissertation proposes that at the core of an effective team is synchrony--the act of moving together as one--which has been shown to predict a variety of psychological and social outcomes. The question of whether--and if so, how--synchrony's benefits extend to the domain of team performance, however, remains untested. This multilevel study consists of two studies examining real undergraduate student teams working together over an academic semester. First, Study 1 tests for construct validity of a synchrony-based relational leadership skill, called synchrony detection, hypothesized to be related to unlocking greater team synchrony. Synchrony detection is proposed to be comprised of two latent factors: pattern recognition style and emotional competency, each with three and four measures, respectively. In addition, I developed a novel measure for this dissertation called AccuSync, which aims to gauge an individual's synchrony detection ability. Results of a confirmatory factor analysis in Study 1 indicate that the battery of measures used here do not support synchrony detection as a construct. AccuSync also demonstrates low scale reliability. Taken together, results of Study 1 warrant more construct validity studies, including development of more refined synchrony detection measures. Future considerations, promising exploratory correlations, and significance of synchrony detection are discussed in light of the null results. Second, Study 2 tests a series of predictive links between synchrony, entitativity, and cohesion as team-level characteristics and their relationship to team performance. Results of structural equation models in Study 2 reveal that synchrony unlocks team performance, as measured by instructor-assigned project grades. Specifically, synchrony enables a social process of greater team entitativity and cohesion to emerge within teams, in turn predicting better team performance. In light of significant Study 2 results, analytical alternatives for considering team-level emergent processes are provided, along with implications for leaders, managers, and educators wishing to extract the benefits of synchrony to build cohesive, yet effective, teams.Doctor of Philosoph
- …