Emergent leaders through looking and speaking: from audio-visual data to multimodal recognition

In this paper we present a multimodal analysis of emergent leadership in small groups using audio-visual features and discuss our experience in designing and collecting a data corpus for this purpose. The ELEA Audio-Visual Synchronized corpus (ELEA AVS) was collected using a light portable setup and contains recordings of small group meetings. The participants in each group performed the winter survival task and filled in questionnaires related to personality and several social concepts such as leadership and dominance. In addition, the corpus includes annotations on participants' performance in the survival task, and also annotations of social concepts from external viewers. Based on this corpus, we present the feasibility of predicting the emergent leader in small groups using automatically extracted audio and visual features, based on speaking turns and visual attention, and we focus specifically on multimodal features that make use of the looking at participants while speaking and looking at while not speaking measures. Our findings indicate that emergent leadership is related, but not equivalent, to dominance, and while multimodal features bring a moderate degree of effectiveness in inferring the leader, much simpler features extracted from the audio channel are found to give better performanc

Investigating Social Interactions Using Multi-Modal Nonverbal Features

Every day, humans are involved in social situations and interplays, with the goal of sharing emotions and thoughts, establishing relationships with or acting on other human beings. These interactions are possible thanks to what is called social intelligence, which is the ability to express and recognize social signals produced during the interactions. These signals aid the information exchange and are expressed through verbal and non-verbal behavioral cues, such as facial expressions, gestures, body pose or prosody. Recently, many works have demonstrated that social signals can be captured and analyzed by automatic systems, giving birth to a relatively new research area called social signal processing, which aims at replicating human social intelligence with machines. In this thesis, we explore the use of behavioral cues and computational methods for modeling and understanding social interactions. Concretely, we focus on several behavioral cues in three specic contexts: rst, we analyze the relationship between gaze and leadership in small group interactions. Second, we expand our analysis to face and head gestures in the context of deception detection in dyadic interactions. Finally, we analyze the whole body for group detection in mingling scenarios

Sensing, interpreting, and anticipating human social behaviour in the real world

Low-level nonverbal social signals like glances, utterances, facial expressions and body language are central to human communicative situations and have been shown to be connected to important high-level constructs, such as emotions, turn-taking, rapport, or leadership. A prerequisite for the creation of social machines that are able to support humans in e.g. education, psychotherapy, or human resources is the ability to automatically sense, interpret, and anticipate human nonverbal behaviour. While promising results have been shown in controlled settings, automatically analysing unconstrained situations, e.g. in daily-life settings, remains challenging. Furthermore, anticipation of nonverbal behaviour in social situations is still largely unexplored. The goal of this thesis is to move closer to the vision of social machines in the real world. It makes fundamental contributions along the three dimensions of sensing, interpreting and anticipating nonverbal behaviour in social interactions. First, robust recognition of low-level nonverbal behaviour lays the groundwork for all further analysis steps. Advancing human visual behaviour sensing is especially relevant as the current state of the art is still not satisfactory in many daily-life situations. While many social interactions take place in groups, current methods for unsupervised eye contact detection can only handle dyadic interactions. We propose a novel unsupervised method for multi-person eye contact detection by exploiting the connection between gaze and speaking turns. Furthermore, we make use of mobile device engagement to address the problem of calibration drift that occurs in daily-life usage of mobile eye trackers. Second, we improve the interpretation of social signals in terms of higher level social behaviours. In particular, we propose the first dataset and method for emotion recognition from bodily expressions of freely moving, unaugmented dyads. Furthermore, we are the first to study low rapport detection in group interactions, as well as investigating a cross-dataset evaluation setting for the emergent leadership detection task. Third, human visual behaviour is special because it functions as a social signal and also determines what a person is seeing at a given moment in time. Being able to anticipate human gaze opens up the possibility for machines to more seamlessly share attention with humans, or to intervene in a timely manner if humans are about to overlook important aspects of the environment. We are the first to propose methods for the anticipation of eye contact in dyadic conversations, as well as in the context of mobile device interactions during daily life, thereby paving the way for interfaces that are able to proactively intervene and support interacting humans.Blick, Gesichtsausdrücke, Körpersprache, oder Prosodie spielen als nonverbale Signale eine zentrale Rolle in menschlicher Kommunikation. Sie wurden durch vielzählige Studien mit wichtigen Konzepten wie Emotionen, Sprecherwechsel, Führung, oder der Qualität des Verhältnisses zwischen zwei Personen in Verbindung gebracht. Damit Menschen effektiv während ihres täglichen sozialen Lebens von Maschinen unterstützt werden können, sind automatische Methoden zur Erkennung, Interpretation, und Antizipation von nonverbalem Verhalten notwendig. Obwohl die bisherige Forschung in kontrollierten Studien zu ermutigenden Ergebnissen gekommen ist, bleibt die automatische Analyse nonverbalen Verhaltens in weniger kontrollierten Situationen eine Herausforderung. Darüber hinaus existieren kaum Untersuchungen zur Antizipation von nonverbalem Verhalten in sozialen Situationen. Das Ziel dieser Arbeit ist, die Vision vom automatischen Verstehen sozialer Situationen ein Stück weit mehr Realität werden zu lassen. Diese Arbeit liefert wichtige Beiträge zur autmatischen Erkennung menschlichen Blickverhaltens in alltäglichen Situationen. Obwohl viele soziale Interaktionen in Gruppen stattfinden, existieren unüberwachte Methoden zur Augenkontakterkennung bisher lediglich für dyadische Interaktionen. Wir stellen einen neuen Ansatz zur Augenkontakterkennung in Gruppen vor, welcher ohne manuelle Annotationen auskommt, indem er sich den statistischen Zusammenhang zwischen Blick- und Sprechverhalten zu Nutze macht. Tägliche Aktivitäten sind eine Herausforderung für Geräte zur mobile Augenbewegungsmessung, da Verschiebungen dieser Geräte zur Verschlechterung ihrer Kalibrierung führen können. In dieser Arbeit verwenden wir Nutzerverhalten an mobilen Endgeräten, um den Effekt solcher Verschiebungen zu korrigieren. Neben der Erkennung verbessert diese Arbeit auch die Interpretation sozialer Signale. Wir veröffentlichen den ersten Datensatz sowie die erste Methode zur Emotionserkennung in dyadischen Interaktionen ohne den Einsatz spezialisierter Ausrüstung. Außerdem stellen wir die erste Studie zur automatischen Erkennung mangelnder Verbundenheit in Gruppeninteraktionen vor, und führen die erste datensatzübergreifende Evaluierung zur Detektion von sich entwickelndem Führungsverhalten durch. Zum Abschluss der Arbeit präsentieren wir die ersten Ansätze zur Antizipation von Blickverhalten in sozialen Interaktionen. Blickverhalten hat die besondere Eigenschaft, dass es sowohl als soziales Signal als auch der Ausrichtung der visuellen Wahrnehmung dient. Somit eröffnet die Fähigkeit zur Antizipation von Blickverhalten Maschinen die Möglichkeit, sich sowohl nahtloser in soziale Interaktionen einzufügen, als auch Menschen zu warnen, wenn diese Gefahr laufen wichtige Aspekte der Umgebung zu übersehen. Wir präsentieren Methoden zur Antizipation von Blickverhalten im Kontext der Interaktion mit mobilen Endgeräten während täglicher Aktivitäten, als auch während dyadischer Interaktionen mittels Videotelefonie

Emergent leaders through looking and speaking: from audio-visual data to multimodal recognition

In this paper we present a multimodal analysis of emergent leadership in small groups using audio-visual features and discuss our experience in designing and collecting a data corpus for this purpose. The ELEA Audio-Visual Synchronized corpus (ELEA AVS) was collected using a light portable setup and contains recordings of small group meetings. The participants in each group performed the winter survival task and filled in questionnaires related to personality and several social concepts such as leadership and dominance. In addition, the corpus includes annotations on participants’ performance in the survival task, and also annotations of social concepts from external viewers. Based on this corpus, we present the feasibility of predicting the emergent leader in small groups using automatically extracted audio and visual features, based on speaking turns and visual attention, and we focus specifically on multimodal features that make use of the looking at participants while speaking and looking at while not speaking measures. Our findings indicate that emergent leadership is related, but not equivalent, to dominance, and while multimodal features bring a moderate degree of effectiveness in inferring the leader, much simpler features extracted from the audio channel are found to give better performance

Comparing Social Science and Computer Science Workflow Processes for Studying Group Interactions

In this article, a team of authors from the Geeks and Groupies workshop, in Leiden, the Netherlands, compare prototypical approaches to studying group interaction in social science and computer science disciplines, which we call workflows. To help social and computer science scholars understand and manage these differences, we organize workflow into three major stages: research design, data collection, and analysis. For each stage, we offer a brief overview on how scholars from each discipline work. We then compare those approaches and identify potential synergies and challenges. We conclude our article by discussing potential directions for more integrated and mutually beneficial collaboration that go beyond the producer–consumer model

Automized Assessment for Professional Skills – A Systematic Literature Review and Future Research Avenues

Globalization, technological progress, and demographic trends in-creasingly influence our labor markets. With changing labor markets and increas-ing digitalization, new competencies of workers are needed to meet demands. However, as a first step to developing these new skills, knowledge about the ex-isting skills and their status quo is necessary. Here, automated skill assessment offers a crucial added value, as it can create a reliable and objective database. Based on a systematic investigation, our analysis shows, in four different areas, how skills and competencies in the automated assessment are (1) defined, (2) included as an element of analysis, (3) methodically recorded and processed, (4) which data source is used. In doing so, we offer insights into existing approaches to automated assessment of professional skills. In doing so, we contribute to a better understanding of the design of automated skill assessment methods and provide perspectives on future research directions

A Meta-Analysis of Procedures to Change Implicit Measures

Using a novel technique known as network meta-analysis, we synthesized evidence from 492 studies (87,418 participants) to investigate the effectiveness of procedures in changing implicit measures, which we define as response biases on implicit tasks. We also evaluated these procedures’ effects on explicit and behavioral measures. We found that implicit measures can be changed, but effects are often relatively weak (|ds| \u3c .30). Most studies focused on producing short-term changes with brief, single-session manipulations. Procedures that associate sets of concepts, invoke goals or motivations, or tax mental resources changed implicit measures the most, whereas procedures that induced threat, affirmation, or specific moods/emotions changed implicit measures the least. Bias tests suggested that implicit effects could be inflated relative to their true population values. Procedures changed explicit measures less consistently and to a smaller degree than implicit measures and generally produced trivial changes in behavior. Finally, changes in implicit measures did not mediate changes in explicit measures or behavior. Our findings suggest that changes in implicit measures are possible, but those changes do not necessarily translate into changes in explicit measures or behavior

Strategies Used in Implementing the Multiple Eligibility Criteria Rule in Georgia Elementary Schools to increase Representation of Black American Students in Gifted Education

This study focused on the strategies used in implementing the multiple eligibility criteria rule in Georgia elementary schools to increase representation of Black American students in gifted education. The framework for this qualitative research project used a Critical Race Theory (CRT) lens while employing ethnographic study methods. The instruments for this study incorporated interviews, focus group discussion, and observations. The analysis of the research from this study found that multiple identification standards such as, motivation, creativity, class performance, love of learning, interest, as well as academics is beneficial when identifying Black American students. Data from this study suggested professional development in student identification and cultural awareness and differences of Black American students is helpful for identification. Enhanced parental support and teacher/parent communication would further improve efficiency when identifying gifted Black American students in the present identification system. The multiple eligibility criteria rule in Georgia is sufficient for promoting representation of Black American students, according to research however, schools must take advantage of the different testing assessments available. Having this flexibility in place widens the options for Black American students

Contemplating Mindfulness at Work: An Integrative Review

Mindfulness research activity is surging within organizational science. Emerging evidence across multiple fields suggests that mindfulness is fundamentally connected to many aspects of workplace functioning, but this knowledge base has not been systematically integrated to date. This review coalesces the burgeoning body of mindfulness scholarship into a framework to guide mainstream management research investigating a broad range of constructs. The framework identifies how mindfulness influences attention, with downstream effects on functional domains of cognition, emotion, behavior, and physiology. Ultimately, these domains impact key workplace outcomes, including performance, relationships, and well-being. Consideration of the evidence on mindfulness at work stimulates important questions and challenges key assumptions within management science, generating an agenda for future research

Promoting Privilege: Selecting Students for a Public Gifted School

Point of view: I am a cisgender, White male in my sixties. I retired recently after working with children in a professional capacity since the mid-1970s. During my education career, I was an elementary school teacher, gifted teacher, research specialist, and director of research and evaluation in a historically White school district that became majority African American during my tenure. Value of submission: Numerous educational policies and procedures in the United States benefit children from privileged families over their traditionally underserved counterparts, which include students of color and low-income students. This piece describes a public school district’s inequitable practices related to its program for gifted students, practices that are not uncommon in many American school districts. “Education is one of the best ways to address systemic inequities, but education systems in the US seem to be increasingly subject to criticism that they are unable to change and promote equity” (Cheville, 2018, p. 1). Despite their inherent resistance to change, educational agencies must be made aware of discriminatory policies and procedures. Stakeholders must then hold policy makers and educational leaders to account. As James Baldwin wrote nearly 60 years ago, “Not everything that is faced can be changed; but nothing can be changed until it is faced” (Baldwin, 1968, p. 38). Summary: Gifted education programs in public schools comprise mainly middle-class and upper-middle-class students of European and Asian descent. Students from low socioeconomic groups, African American students, Latinx students, and Indigenous American students continue to be underrepresented in gifted programs, despite the fact that this inequity was brought to light many years ago (Ford, 1998). Given our nation’s long history of overt and covert racism, it is not surprising that the manner by which students are identified for gifted services is systemically entrenched. Most states have mandates or provide guidance to local school districts regarding identification criteria; however, very few of the measurement instruments or methods used to evaluate children for gifted services are effective at facilitating equitable representation of all groups in gifted education programs. This piece examines one school district’s guidelines used to identify students for gifted services, including admittance to its prestigious school for gifted children. Because the guidelines are typical of practices employed by many school districts across the US, the information contained herein is generalizable to a larger audience