Classifying multimodal turn management in Danish dyadic first encounters

This paper deals with multimodal turn management in an annotated Danish corpus of video recorded dyadic conversations between young people who meet for the first time. Conversation participants indicate whether they wish to give, take or keep the turn through speech as well as body behaviours. In this study we present an analysis of turn management body behaviours as well as classification experiments run on the annotated data in order to investigate how far it is possible to distinguish between the different types of turn management expressed by body behaviours using their shape and the co-occurring speech expressions. Our study comprises body behaviours which have not been previously investigated with respect to turn management, so that it not only confirms preceding studies on turn management in English but also provides new insight on how speech and body behaviours are used together in communication. The classification experiments indicate that the shape annotations of all kinds of body behaviour together with information about the gesturer’s co-occurring speech are useful to classify turn management types, and that the various behaviours contribute to the expression of turn features in different ways. Thus, knowledge of the different cues used by speakers in face-to-face communication to signal different types of turn shift provides the basis for modelling turn management, which is in turn key to implement natural conversation flow in multimodal dialogue systemspeer-reviewe

Les processus inférentiels dans l’interaction multimodale en L2

Les interactions entre apprenants d’une langue seconde (L2) restant encore insuffisamment étudiées, les connaissances sur le type de ressources interactionnelles étant en jeu dans ces échanges et sur la manière dont elles sont mobilisées sont dès lors limitées. La communication étant multimodale par nature, bon nombre de ressources posturo-mimo-gestuelles (PMG) sont mobilisées par les locuteurs apprenants d’une L2 et participent ainsi de l’intention communicative de l’énonciateur. Il semble donc essentiel de mieux connaître le rôle de ces ressources PMG pour les énonciateurs apprenants d’une L2 dans le cadre d’interactions apprenants-apprenants, mais également la manière dont ces ressources sont décodées par leurs co-énonciateurs, par la mise en oeuvre de processus inférentiels permettant d’accéder au sens visé. L’étude empirique exploratoire que nous présentons dans cet article se propose d’apporter un éclairage sur ces questions. À partir d’un corpus d’interactions orales entre apprenants de l’anglais L2, nous mettons en évidence l’apport essentiel des processus inférentiels dans l’interprétation de ressources PMG au service de la tentative de mise en mots et de la gestion des tours de parole, ainsi que dans le décodage des ressources PMG se substituant au verbal. Il émerge également de notre analyse qu’une même ressource PMG peut avoir de multiples fonctions en contexte. Tous ces éléments nous poussent à plaider en faveur de l’intégration, dans l’apprentissage des langues, d’une formation à la production et à l’interprétation des ressources PMG comme autant d’indices « inférables » par les interlocuteurs dans une aire géographique et linguistique donnée.While published research on the role of the body in multimodal didactic discourse is growing at a fast pace, as shown by the numerous recent studies on the subject (Aden, 2017 ; Azaoui, 2015 ; Sime, 2001, 2008 ; Tellier, 2014, 2016), research on the multimodal nature of learner-learner interactions in a second language (L2) remains more confidential than that on interactions between teacher and learner(s) or between native speaker(s) and learner(s) (Kurhila, 2006). In addition, few studies have been published to date on the use of non-verbal resources in learner-learner exchanges (Gullberg, 2010). As a result, our knowledge of the types of interactional resources involved in these exchanges and of the way they are used is still limited. This is particularly true of interactional resources other than the words uttered by interactants. Interacting results from a complex cognitive process (McNeill, 2005 ; Kendon, 2004) which combines verbal resources as well as vocal and visual ones (postures, facial expressions and gestures, cf. “ressources posturo-mimo-gestuelles” for Cosnier, 1985). We investigate the impact of the visual resources in face-to-face L2 learner exchanges so as to better understand their role in learner-learner interactions, as well as the way they are interpreted by the co-speakers through the inferential processes in play. It is only thanks to these pragmatic inferential processes, which are instrumental in the interpretation of contextual elements (cf. Gumperz 1989), be they intra-textual or extra-textual, that co-speakers can truly access the speaker’s intended meaning. Gestures and more generally visual resources trigger inferential processes that make it possible to reconstruct the meaning of utterances, even more so when the latter are difficult to put into words. In the case of ambiguous, erroneous or incomplete utterances, coverbal gestures, i.e., gestures produced at the same time as speech acts (McNeill, 1992, 2005 ; Goldin-Meadow, 2003), can generate inferential work for the co-speakers and facilitate access to the speaker’s intended meaning (Tellier, 2014). The multimodal nature of interpersonal communication is claimed to be all the more perceptible in exchanges between L2 learners because of the deficits in their L2 system (David, 2017) : a number of studies (Gullberg, 1998 ; Hadar et al., 2001 ; Nobe, 2001 ; Yoshioka, 2005) show that the use of gestures is more prevalent during exchanges in L2 than in L1, in particular the use of deictic gestures (Sherman & Nicoladis, 2004). The exploratory empirical study presented in this paper aims to shed light on the role of visual resources for a given speaker in interactions between learners of L2 English, but also on the way these resources are interpreted by the co-speakers, who engage in inferential processes to gain access to the intended meaning. Our work is based on a corpus of face-to-face oral interactions between learners of L2 English in tertiary education. Our analysis highlights the essential role of inferential processes in the interpretation of visual resources in three different patterns : learners of L2 English tend to make inferences based on (1) visual resources used during word searches, (2) visual resources used in turn management, and (3) visual resources used as substitutes for verbal elements. Our results show that inferences based on visual resources are related to both the speaker’s communicative intent and the management of speech turns. Our study also sheds light on the multiple functions that a single visual resource can take on in a given context and leads us to advocate language classes with specific training in interpreting and resorting to visual resources as “inferrable” clues which are specific to a given geographical and linguistic area

Internet linguistics: a conversational analysis of online synchronous chat and face-to-face conversations of EFL undergraduate students in Jordan

This study aimed to shed light on Online Synchronous chat (OSC) on Facebook chatroom compared with Face-To-Face (FTF) conversations. The corpus was cumulated from the interaction of four groups consisted of (68) third-year English language and literature major students at Ajloun University College (AUC). The participants were selected purposively and distributed randomly into two OSCGs and two FTF groups. The interactions for FTF groups were video-recorded and the transcriptions were embedded line by line in each conversation. While the interaction on the two Facebook chatrooms were downloaded through a Facebook option called "download your information". Two instruments were used: a Speech Act Rubric Scale based on Grice's maxims, linguistics performance rubric checklist, and an open-end question had been just presented to the chatters. This study investigated whether interlocutors apply the four Gricean CPs and three linguistic aspects over seventeen turn-taking and repair acts. Thus, the comparison was a try to investigate the social and linguistic performance of OSC interlocutors. Results revealed the importance to improve chatrooms features regarding to speech acts theory and Grice's maxim. The analysis concluded that interaction on OSC still needs more investigation. More precisely, Facebook chatrooms neglects to some extent the two theories.Este estudio tuvo como objetivo arrojar luz sobre el Chatear Sincrónico en Línea (CSL) en el chat de Facebook en comparación con las conversaciones cara a cara (CAC). El corpus se recopiló a partir de la interacción de cuatro grupos, de 68 estudiantes de tercer curso del Grado de Lengua y Literatura inglesa en el Colegio Universitario de Ajloun (CUA). Los participantes fueron seleccionados deliberadamente y distribuidos aleatoriamente en dos grupos de chat sincrónico en línea y dos grupos cara a cara. Las interacciones de los grupos CAC se grabaron un video y sus transcripciones se incrustaron línea por línea en cada conversación. Mientras que la interacción en los dos chats de Facebook se descargó a través de una opción de Facebook llamada "descargar su información". Se utilizaron dos instrumentos: una lista de verificación de la rúbrica del acto de habla basada en las máximas de Grice, la lista de verificación de la rúbrica del rendimiento lingüístico y una pregunta abierta presentada a los interlocutores. Este estudio investigó si los interlocutores aplican los cuatro principios cooperativos de Grice y tres aspectos lingüísticos en diecisiete actos de reparación y toma de turnos. Por lo tanto, la comparación fue un intento de investigar el desempeño social y lingüístico de los interlocutores de CSL. Los resultados revelaron la importancia de mejorar las características de los chats según a la teoría de los actos de habla y la máxima de Grice. El análisis concluyó que la interacción en CSL aún necesita más investigación. Más precisamente, los chats de Facebook descuidan en cierta medida las dos teorías.Programa de Doctorado en Humanidades por la Universidad Carlos III de MadridPresidente: María Paloma Díaz Pérez.- Secretario: Linda Johanna Castañeda Quintero.- Vocal: Fernando Trujillo Sáe