Do (and say) as I say: Linguistic adaptation in human-computer dialogs

© Theodora Koulouri, Stanislao Lauria, and Robert D. Macredie. This article has been made available through the Brunel Open Access Publishing Fund.There is strong research evidence showing that people naturally align to each other’s vocabulary, sentence structure, and acoustic features in dialog, yet little is known about how the alignment mechanism operates in the interaction between users and computer systems let alone how it may be exploited to improve the efficiency of the interaction. This article provides an account of lexical alignment in human–computer dialogs, based on empirical data collected in a simulated human–computer interaction scenario. The results indicate that alignment is present, resulting in the gradual reduction and stabilization of the vocabulary-in-use, and that it is also reciprocal. Further, the results suggest that when system and user errors occur, the development of alignment is temporarily disrupted and users tend to introduce novel words to the dialog. The results also indicate that alignment in human–computer interaction may have a strong strategic component and is used as a resource to compensate for less optimal (visually impoverished) interaction conditions. Moreover, lower alignment is associated with less successful interaction, as measured by user perceptions. The article distills the results of the study into design recommendations for human–computer dialog systems and uses them to outline a model of dialog management that supports and exploits alignment through mechanisms for in-use adaptation of the system’s grammar and lexicon

A Study of Accomodation of Prosodic and Temporal Features in Spoken Dialogues in View of Speech Technology Applications

Inter-speaker accommodation is a well-known property of human speech and human interaction in general. Broadly it refers to the behavioural patterns of two (or more) interactants and the effect of the (verbal and non-verbal) behaviour of each to that of the other(s). Implementation of thisbehavior in spoken dialogue systems is desirable as an improvement on the naturalness of humanmachine interaction. However, traditional qualitative descriptions of accommodation phenomena do not provide sufficient information for such an implementation. Therefore, a quantitativedescription of inter-speaker accommodation is required. This thesis proposes a methodology of monitoring accommodation during a human or humancomputer dialogue, which utilizes a moving average filter over sequential frames for each speaker. These frames are time-aligned across the speakers, hence the name Time Aligned Moving Average (TAMA). Analysis of spontaneous human dialogue recordings by means of the TAMA methodology reveals ubiquitous accommodation of prosodic features (pitch, intensity and speech rate) across interlocutors, and allows for statistical (time series) modeling of the behaviour, in a way which is meaningful for implementation in spoken dialogue system (SDS) environments.In addition, a novel dialogue representation is proposed that provides an additional point of view to that of TAMA in monitoring accommodation of temporal features (inter-speaker pause length and overlap frequency). This representation is a percentage turn distribution of individual speakercontributions in a dialogue frame which circumvents strict attribution of speaker-turns, by considering both interlocutors as synchronously active. Both TAMA and turn distribution metrics indicate that correlation of average pause length and overlap frequency between speakers can be attributed to accommodation (a debated issue), and point to possible improvements in SDS “turntaking” behaviour. Although the findings of the prosodic and temporal analyses can directly inform SDS implementations, further work is required in order to describe inter-speaker accommodation sufficiently, as well as to develop an adequate testing platform for evaluating the magnitude ofperceived improvement in human-machine interaction. Therefore, this thesis constitutes a first step towards a convincingly useful implementation of accommodation in spoken dialogue systems

Acomodación fonética durante las interacciones conversacionales: una visión general

During conversational interactions such as tutoring, instruction-giving tasks, verbal negotiations, or just talking with friends, interlocutors’ behaviors experience a series of changes due to the characteristics of their counterpart and to the interaction itself. These changes are pervasively present in every social interaction, and most of them occur in the sounds and rhythms of our speech, which is known as acoustic-prosodic accommodation, or simply phonetic accommodation. The consequences, linguistic and social constraints, and underlying cognitive mechanisms of phonetic accommodation have been studied for at least 50 years, due to the importance of the phenomenon to several disciplines such as linguistics, psychology, and sociology. Based on the analysis and synthesis of the existing empirical research literature, in this paper we present a structured and comprehensive review of the qualities, functions, onto- and phylogenetic development, and modalities of phonetic accommodation.Durante las interacciones conversacionales como dar una tutoría, dar instrucciones, las negociaciones verbales, o simplemente hablar con amigos, los comportamientos de las personas experimentan una serie de cambios debido a las características de su interlocutor y a la interacción en sí. Estos cambios están presentes en cada interacción social, y la mayoría de ellos ocurre en los sonidos y ritmos del habla, lo cual se conoce como acomodación acústico-prosódica, o simplemente acomodación fonética. Las consecuencias, las limitaciones lingüísticas y sociales, y los mecanismos cognitivos subyacentes a la acomodación fonética se han estudiado durante al menos 50 años, debido a la importancia del fenómeno para varias disciplinas como la lingüística, la psicología, y la sociología. A partir del análisis y síntesis de la literatura de investigación empírica existente, en este artículo presentamos una revisión estructurada y exhaustiva de las cualidades, funciones, desarrollo onto- y filogenético, y modalidades de la acomodación fonética

Convergen rítmicamente los hablantes? Estudio sobre las propiedades segmentales temporales en el alemán de Zúrich y del cantón de Grisón antes y después de las interacciones dialógicas

This paper reports on the results of a research investigating whether rhythmic features, in terms of segmental timing properties, are object of speaker’s adjustments after the exposure to a conversational partner. In the context of dialects in contact, this is crucial to understand whether rhythmic attributes may bring about language variation and change. In the context of human-machine interactions, this can benefit the design of spoken dialogues systems to achieve human-likeness. To study rhythmic accommodation, we selected a corpus of pre- and post-dialogue recordings, performed by 18 speakers of Grison and Zurich German (henceforth GRG and ZHG), two Swiss German dialects characterised by noticeable segmental and supra-segmental differences. To quantify rhythmic convergence, we designed three measures based on the segmental timing differences between the two dialects. We compared the Euclidean distances in the three measures between GRG and ZHG speakers in a pair before and after two interactions. Results reveal that dyads members do not significantly shift the production of segmental timing features after the dialogues. Neither linguistic nor social factors can account for the observed accommodation pattern. Cross-dialectal segmental timing differences, captured by the three ratio measures, may be either robust against the influence of interlocutors’ acoustic behaviour or too subtle to be perceived or retained after interactions.Este artículo presenta los resultados de una investigación que examina si los rasgos rítmicos vinculados con las características temporales segmentales sufren adaptaciones por parte del locutor después de cierta exposición con un interlocutor. En un contexto de dialectos en contacto, es importante comprender la aportación del estudio de las propiedades rítmicas al campo de la variación y cambio de lengua. En un contexto de interacción persona-ordenador, esto puede beneficiar el diseño de sistemas de diálogos que logren un habla casi nativa. Para estudiar la acomodación rítmica, se seleccionó un corpus de grabaciones recopiladas antes y después de un diálogo entre 18 pares de locutores de alemán del cantón de Grisón y de alemán de Zúrich (de ahora en adelante, GRG y ZHG), dos dialectos suizo-alemanes que se caracterizan por sus diferencias segmentales y suprasegmentales. Para cuantificar la convergencia rítmica, se usaron tres medidas basadas en las diferencias temporales segmentales entre los dos dialectos. Se comparó la distancia euclidiana para las tres medidas entre locutores GRG y ZHG antes y después de dos interacciones. Los resultados revelan que los miembros del par no cambian significativamente la producción de sus características rítmicas temporales después del diálogo. Ningún factor lingüístico o social permite explicar el patrón de acomodación que se observa. Las diferencias rítmicas temporales inter-dialectos, que se capturan con las tres medidas, son o demasiado robustas contra la influencia del comportamiento rítmico de los interlocutores, o demasiado sutiles para ser percibidas o recordadas después de las interacciones

Judgment of the Humanness of an Interlocutor Is in the Eye of the Beholder

Despite tremendous advances in artificial language synthesis, no machine has so far succeeded in deceiving a human. Most research focused on analyzing the behavior of “good” machine. We here choose an opposite strategy, by analyzing the behavior of “bad” humans, i.e., humans perceived as machine. The Loebner Prize in Artificial Intelligence features humans and artificial agents trying to convince judges on their humanness via computer-mediated communication. Using this setting as a model, we investigated here whether the linguistic behavior of human subjects perceived as non-human would enable us to identify some of the core parameters involved in the judgment of an agents' humanness. We analyzed descriptive and semantic aspects of dialogues in which subjects succeeded or failed to convince judges of their humanness. Using cognitive and emotional dimensions in a global behavioral characterization, we demonstrate important differences in the patterns of behavioral expressiveness of the judges whether they perceived their interlocutor as being human or machine. Furthermore, the indicators of interest displayed by the judges were predictive of the final judgment of humanness. Thus, we show that the judgment of an interlocutor's humanness during a social interaction depends not only on his behavior, but also on the judge himself. Our results thus demonstrate that the judgment of humanness is in the eye of the beholder