Detecting Low Rapport During Natural Interactions in Small Groups from Non-Verbal Behaviour

Rapport, the close and harmonious relationship in which interaction partners are "in sync" with each other, was shown to result in smoother social interactions, improved collaboration, and improved interpersonal outcomes. In this work, we are first to investigate automatic prediction of low rapport during natural interactions within small groups. This task is challenging given that rapport only manifests in subtle non-verbal signals that are, in addition, subject to influences of group dynamics as well as inter-personal idiosyncrasies. We record videos of unscripted discussions of three to four people using a multi-view camera system and microphones. We analyse a rich set of non-verbal signals for rapport detection, namely facial expressions, hand motion, gaze, speaker turns, and speech prosody. Using facial features, we can detect low rapport with an average precision of 0.7 (chance level at 0.25), while incorporating prior knowledge of participants' personalities can even achieve early prediction without a drop in performance. We further provide a detailed analysis of different feature sets and the amount of information contained in different temporal segments of the interactions.Comment: 12 pages, 6 figure

Analysis of Laughter in Cohesive Groups

Group cohesion describes the tendency of the group members’ shared commitment to group tasks and the interpersonal attraction among them. This paper presents a preliminary analysis of occurrence of laughter with respect to group cohesion using a corpus of multi-party interactions. Results indicate that the occurrence of laughter is higher in cohesive segments and a strong positive correlation exists between the perceived level of cohesion and laughter

Analysis of Laughter in Cohesive Groups

Group cohesion describes the tendency of the group members’ shared commitment to group tasks and the interpersonal attraction among them. This paper presents a preliminary analysis of occurrence of laughter with respect to group cohesion using a corpus of multi-party interactions. Results indicate that the occurrence of laughter is higher in cohesive segments and a strong positive correlation exists between the perceived level of cohesion and laughter

Fully Automatic Analysis of Engagement and Its Relationship to Personality in Human-Robot Interactions

Engagement is crucial to designing intelligent systems that can adapt to the characteristics of their users. This paper focuses on automatic analysis and classification of engagement based on humans’ and robot’s personality profiles in a triadic human-human-robot interaction setting. More explicitly, we present a study that involves two participants interacting with a humanoid robot, and investigate how participants’ personalities can be used together with the robot’s personality to predict the engagement state of each participant. The fully automatic system is firstly trained to predict the Big Five personality traits of each participant by extracting individual and interpersonal features from their nonverbal behavioural cues. Secondly, the output of the personality prediction system is used as an input to the engagement classification system. Thirdly, we focus on the concept of “group engagement”, which we define as the collective engagement of the participants with the robot, and analyse the impact of similar and dissimilar personalities on the engagement classification. Our experimental results show that (i) using the automatically predicted personality labels for engagement classification yields an F-measure on par with using the manually annotated personality labels, demonstrating the effectiveness of the automatic personality prediction module proposed; (ii) using the individual and interpersonal features without utilising personality information is not sufficient for engagement classification, instead incorporating the participants’ and robot’s personalities with individual/interpersonal features increases engagement classification performance; and (iii) the best classification performance is achieved when the participants and the robot are extroverted, while the worst results are obtained when all are introverted.This work was performed within the Labex SMART project (ANR-11-LABX-65) supported by French state funds managed by the ANR within the Investissements d’Avenir programme under reference ANR-11-IDEX-0004-02. The work of Oya Celiktutan and Hatice Gunes is also funded by the EPSRC under its IDEAS Factory Sandpits call on Digital Personhood (Grant Ref.: EP/L00416X/1).This is the author accepted manuscript. The final version is available from Institute of Electrical and Electronics Engineers via http://dx.doi.org/10.1109/ACCESS.2016.261452

Fragmented but coherent : Lexical cohesion on a YouTube channel

This paper investigates the use and functions of lexical cohesion within and across modes as well as levels of interaction on a YouTube channel, arguing that lexical cohesion contributes to coherence by establishing links between the video, the comments and sources elsewhere on the internet in a cross-modal and intertextual manner. The analysis focuses on one video and its comments section, investigating the use of cohesion in maintaining coherence and unity across a lengthy YouTube interaction initiated by the video and continued in the comments. Cohesive ties are examined with regard to their functions in the process of meaning-making in this mediated and multimodal context. The results reveal that the use of cohesive ties is extensive within and across modes and levels of interaction. Chains of cohesion are created within the comments, with cross-modal links to the video. Despite the complex multimodality of YouTube and the fragmentation of the comments as many participants post short comments one after another, the YouTube interaction is shown to be coherent. (c) 2021 The Author. Published by Elsevier Ltd. This is an open access article under the CC BY license (http:// creativecommons.org/licenses/by/4.0/).Peer reviewe

Emergent leaders through looking and speaking: from audio-visual data to multimodal recognition

In this paper we present a multimodal analysis of emergent leadership in small groups using audio-visual features and discuss our experience in designing and collecting a data corpus for this purpose. The ELEA Audio-Visual Synchronized corpus (ELEA AVS) was collected using a light portable setup and contains recordings of small group meetings. The participants in each group performed the winter survival task and filled in questionnaires related to personality and several social concepts such as leadership and dominance. In addition, the corpus includes annotations on participants' performance in the survival task, and also annotations of social concepts from external viewers. Based on this corpus, we present the feasibility of predicting the emergent leader in small groups using automatically extracted audio and visual features, based on speaking turns and visual attention, and we focus specifically on multimodal features that make use of the looking at participants while speaking and looking at while not speaking measures. Our findings indicate that emergent leadership is related, but not equivalent, to dominance, and while multimodal features bring a moderate degree of effectiveness in inferring the leader, much simpler features extracted from the audio channel are found to give better performanc

Critically Evaluating the Role of Intercultural Marketing Communications in Cultivating Relations in the Superdiverse Rainbow Nation

Contemporary societies can be classed as superdiverse. Existing research indicates that superdiversity translates into daily marketplace interactions between numerous cultural value systems. The experience of living within a superdiverse marketplace creates a range of challenges for consumers. Marketers can help shape society by mitigating these intercultural challenges through their marketing communication strategies. Problematically, the impact of superdiversity on marketing communication strategies and practices is understudied. Concurrently, little is known about consumers’ lived experiences of superdiversity. Thus, marketers are finding it increasingly difficult to communicate effectively with superdiverse consumer groups. As a result, a growing disconnect between consumers and marketers exists. Therefore, the research aim of this study was to critically evaluate the ways in which superdiversity impacts on marketing communications strategy development. To gain a holistic understanding, three research objectives were set. The first was to inspect the role of superdiversity ideology in marketing communications strategy evolution. The second was to examine the influence of consumers’ lived experiences of superdiversity on marketing communications strategy development. The third was to assess challenges prohibiting the social impact of marketing communications in superdiverse marketplaces. To improve understanding with novel insights, this project was set in a non-Western research setting - South Africa (SA). Post-apartheid, SA government sought to resolve tensions in a peaceful manner by establishing a Rainbow Nation (RN) ideology. This required widespread marketing communication. Mirroring the lack of knowledge identified above, SA consumers are increasingly discontent. A conceptual model informed the empirical study undertaken. Three theories underpin the conceptual model. They are embedded in an interculturalism approach, allowing for comparative study of marketing (Neo-Institutional theory) and how intercultural interactions and experiences shape meanings of living in superdiversity (Creolisation and Imagined intercultural contact theories). The conceptual model also supported the adoption of a multi-method qualitative research design. A scoping study reviewed SA campaigns using critical visual analysis approach. A case study examined National Heritage Day, including campaign, marketer and consumer data. A multimodal strategy was used for comparative analysis. The findings show the RN concept has lost its impact and relevancy. The marketplace calls for consumer participation and improved understanding of lived experiences to achieve the welfare stage. A more humanistic approach is needed to establish unity in diversity as a marketplace norm. Based on the findings a new diversity-sensitive marketing communications strategy – intercultural marketing - was proposed and developed. Intercultural marketing strategy is defined as a socially responsible approach to marketing communications strategy, concerned with facilitating intercultural interactions and improving societal welfare