Social resonance and embodied coordination in face-to-face conversation with artificial interlocutors

Kopp S. Social resonance and embodied coordination in face-to-face conversation with artificial interlocutors. Speech Communication. 2010;52(6):587-597.Human natural face-to-face communication is characterized by inter-personal coordination. In this paper, phenomena are analyzed that yield coordination of behaviors, beliefs, and attitudes between interaction partners, which can be tied to a concept of establishing social resonance. It is discussed whether these mechanisms can and should be transferred to conversation with artificial interlocutors like ECAs or humanoid robots. It is argued that one major step in this direction is embodied coordination, mutual adaptations that are mediated by flexible modules for the top-down production and bottom-up perception of expressive conversational behavior that ground in and, crucially, coalesce in the same sensorimotor structures. Work on modeling this for ECAs with a focus on coverbal gestures is presented. (C) 2010 Elsevier B.V. All rights reserved

Leading and following with a virtual trainer

This paper describes experiments with a virtual fitness trainer capable of mutually coordinated interaction. The virtual human co-exercises along with the user, leading as well as following in tempo, to motivate the user and to influence the speed with which the user performs the exercises. In a series of three experiments (20 participants in total) we attempted to influence the users' performance by manipulating the (timing of the) exercise behavior of the virtual trainer. The results show that it is possible to do this implicitly, using only micro adjustments to its bodily behavior. As such, the system is a rst step in the direction of mutually coordinated bodily interaction for virtual humans

Social behavior modeling based on Incremental Discrete Hidden Markov Models

12 pagesInternational audienceModeling multimodal face-to-face interaction is a crucial step in the process of building social robots or users-aware Embodied Conversational Agents (ECA). In this context, we present a novel approach for human behavior analysis and generation based on what we called "Incremental Discrete Hidden Markov Model" (IDHMM). Joint multimodal activities of interlocutors are first modeled by a set of DHMMs that are specific to supposed joint cognitive states of the interlocutors. Respecting a task-specific syntax, the IDHMM is then built from these DHMMs and split into i) a recognition model that will determine the most likely sequence of cognitive states given the multimodal activity of the in- terlocutor, and ii) a generative model that will compute the most likely activity of the speaker given this estimated sequence of cognitive states. Short-Term Viterbi (STV) decoding is used to incrementally recognize and generate behav- ior. The proposed model is applied to parallel speech and gaze data of interact- ing dyads

Peranan Penggunaan Media Sosial Dalam Meningkatkan Kinerja Penjualan UMKM yang Dimediasi Oleh Resonansi Kualitas Layanan Perdagangan Sosial

Usaha mikro, kecil, dan menengah (UMKM) saat ini sedang tren di Indonesia dan memiliki peran dalam ekonomi makro. UMKM membantu mengurangi kesenjangan ekonomi antara daerah perkotaan dan pedesaan dan meningkatkan kualitas hidup masyarakat di daerah terpencil. Peran UMKM dalam meningkatkan Produk Domestik Bruto (PDB) Indonesia juga tidak kalah penting. Meski begitu, ada beberapa faktor yang menghambat kinerja UMKM seperti kurangnya strategi yang jelas dalam memanfaatkan media sosial, serta ketidakmampuan mengikuti tren dan perubahan algoritma pada platform media sosial. Penelitian ini bertujuan untuk mengisi kesenjangan penelitian penggunaan media sosial terhadap kinerja penjualan, dan mengembangkan teori logika dominan layanan dengan menguji model resonansi kualitas layanan perdagangan sosial sebagai hasil sintesis beberapa variabel. Penelitian ini secara empiris mengkaji pengaruh penggunaan media sosial terhadap kinerja penjualan yang dimediasi oleh kualitas layanan elektronik, penjualan berorientasi pelanggan, dan resonansi kualitas layanan perdagangan sosial. Contoh penelitian ini adalah pelaku UMKM yang tergabung dalam Gereja Bethel Indonesia yang berdomisili di Jakarta, Tangerang, Bekasi, dan sekitarnya yang melakukan pemasaran dan penjualan produk menggunakan media sosial. Pengambilan sampel dilakukan dengan purposive sampling. Pengumpulan data dilakukan dengan menyebarkan kuesioner kepada responden sebanyak 215 kuesioner. Metode statistik yang digunakan adalah SEM PLS. Hasil penelitian ini menunjukkan bahwa penggunaan media sosial, kualitas layanan elektronik, penjualan yang berorientasi pada pelanggan, dan resonansi kualitas layanan social commerce berpengaruh signifikan terhadap kinerja penjualan. Jadi kualitas layanan elektronik, penjualan berorientasi pelanggan, dan resonansi kualitas layanan perdagangan sosial memediasi hubungan kinerja penjualan. Kata kunci: Penjualan berorientasi pelanggan, UMKM, resonansi, kualitas layanan, perdagangan sosial. Micro, small and medium enterprises (MSMEs) are currently trending in Indonesia and have a role in macroeconomics. MSMEs help reduce the economic gap between urban and rural areas and improve the quality of life of people in remote areas. Peran MSMEs in increasing Indonesia's Gross Domestic Product (GDP) are no less important. Even so, there are several factors that hinder the performance of MSMEs such as the lack of a clear strategy in utilizing social media, as well as the inability to follow trends and algorithm changes on social media platforms. This study aims to fill the research gap of social media use on sales performance, and develop a theory of service dominant logic by testing the resonance model of social commerce service quality as a result of synthesizing several variables. This study empirically examines the effect of social media use on sales performance mediated by electronic service quality, customer-oriented sales, and resonance of social commerce service quality. An example of this research is MSME actors who are members of the Indonesian Bethel Church domiciled in Jakarta, Tangerang, Bekasi, and surrounding areas who market and sell products using social media. Sampling is done by purposive sampling. Data collection was carried out by distributing questionnaires to respondents as many as 215 questionnaires. The statistical method used is SEM PLS. The results of this study show that the use of social media, the quality of electronic services, customer-oriented sales, and the resonance of social commerce service quality have a significant effect on sales performance. So e-service quality, customer-oriented sales, and resonant social commerce service quality mediate the relationship of social media use to sales performance. Keywords: Customer-oriented sales, MSMEs, resonance, service quality, social trade

A proposal to act on Theory of Mind by applying robotics and virtual worlds with children with ASD

The article proposes an intervention framed under a single-subject research design where robotics and a 3D virtual environment are used jointly to improve the development of Theory of Mind in children with ASD. The project aims at verifying if the use of a humanoid robot, with high interactive abilities and responses, along with a virtual robot in a social virtual world can enable an improved comprehension of emotions and perspective taking. Specifically the planned activities are designed to gradually support the subjects with ASD in interactional settings in order to make them acquire the needed self-confidence to finally interact with a classmate in the virtual environment

A Study of Prosodic Entrainment and Social Factors in Mandarin Conversations

In conversations, interlocutors usually adopt prosody to that of their partner, and they become similar in prosodic production for successful communication. This phenomenon of prosodic entrainment is related to complex factors. This study aims to explore the relationship between prosodic entrainment and social factors. Two analyses are accomplished: the analysis of prosodic entrainment and gender, and the analysis of prosodic entrainment and role. In terms of prosodic entrainment and gender, it is found that the most prosodic features are entrained in female-male conversations, and the least in male-male conversations. In terms of prosodic entrainment and roles, it is found that different roles have influence on the entrainment degree, and information givers entrain more to followers in conversation

Embodied cooperation using mobile devices: presenting and evaluating the Sync4All application

ABSTRACT Embodied cooperation "arises when two co-present, individuals in motion coordinate their goal-directed actions". The adoption of the embodied cooperation paradigm for the development of embodied and social multimedia systems opens new perspectives for future User Centric Media. Systems for embodied music listening, which enable users to influence music in real-time by movement and gesture, can greatly benefit from the embodied cooperation paradigm. This paper presents the design and the evaluation of an application, Sync4All, based on such a paradigm, allowing users to experience social embodied music listening. Each user rhythmically and freely moves a mobile phone trying to synchronise her movements with those of the other ones. The level of such a synchronisation influences the music experience. The evaluation of Sync4All was aimed at finding out which is the overall attitude of the users towards the application, and how the participants perceived embodied cooperation and music embodiment

Studying the Co-Construction of Interaction Protocols in Collaborative Tasks with Humans

International audienceIn interaction, humans align and effortlessly create common ground in communication, allowing efficient collabora- tion in widely diverse contexts. Robots are still far away from being able to adapt in such a flexible manner with non-expert humans to complete collaborative tasks. Challenges include the capability to understand unknown feedback or guidance signals, to make sense of what they refer to depending on their timing and context, and to agree on how to organize the interaction into roles and turns. As a first step in approaching this issue, we investigate here the processes used by humans to negotiate a protocol of interaction when they do not already share one. We introduce a new experimental setup, where two humans have to collaborate to solve a task. The channels of communication they can use are constrained and force them to invent and agree on a shared interaction protocol in order to solve the task. These constraints allow us to analyze how a communication protocol is progressively established through the interplay and history of individual actions. We report preliminary results obtained from a pilot study, and discuss how the understanding of strategies used by humans could be useful to achieve more flexible HRI