Multi-session group scenarios for speech interface design

When developing adaptive speech-based multilingual interaction systems, we need representative data on the user's behaviour. In this paper we focus on a data collection method pertaining to adaptation in the user's interaction with the system. We describe a multi-session group scenario for Wizard of Oz studies with two novel features: firstly, instead of doing solo sessions with a static mailbox, our test users communicated with each other in a group of six, and secondly, the communication took place over several sessions in a period of five to eight days. The paper discusses our data collection studies using the method, concentrating on the usefulness of the method in terms of naturalness of the interaction and long-term developments

Estimating the number of segments for improving dialogue act labelling

In dialogue systems it is important to label the dialogue turns with dialogue-related meaning. Each turn is usually divided into segments and these segments are labelled with dialogue acts (DAs). A DA is a representation of the functional role of the segment. Each segment is labelled with one DA, representing its role in the ongoing discourse. The sequence of DAs given a dialogue turn is used by the dialogue manager to understand the turn. Probabilistic models that perform DA labelling can be used on segmented or unsegmented turns. The last option is more likely for a practical dialogue system, but it provides poorer results. In that case, a hypothesis for the number of segments can be provided to improve the results. We propose some methods to estimate the probability of the number of segments based on the transcription of the turn. The new labelling model includes the estimation of the probability of the number of segments in the turn. We tested this new approach with two different dialogue corpora: SwitchBoard and Dihana. The results show that this inclusion significantly improves the labelling accuracy. © Copyright Cambridge University Press 2011.Work supported by the EC (FEDER/FSE), the Spanish Government (MEC, MICINN, MITyC, MAEC, "Plan E", under grants MIPRCV "Consolider Ingenio 2010" CSD2007-00018, MITTRAL TIN2009-14633-C03-01, erudito.com TSI-020110-2009-439, FPI fellowship BES-2007-16834), and Generalitat Valenciana (grant Prometeo/2009/014 and grant ACOMP/2010/051).Tamarit Ballester, V.; Martínez-Hinarejos, C.; Benedí Ruiz, JM. (2012). Estimating the number of segments for improving dialogue act labelling. Natural Language Engineering. 18(1):1-19. doi:10.1017/S135132491000032XS119181Dybkjær, L., & Minker, W. (Eds.). (2008). Recent Trends in Discourse and Dialogue. Text, Speech and Language Technology. doi:10.1007/978-1-4020-6821-8Stolcke, A., Ries, K., Coccaro, N., Shriberg, E., Bates, R., Jurafsky, D., … Meteer, M. (2000). Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech. Computational Linguistics, 26(3), 339-373. doi:10.1162/089120100561737Fraser, N. M., & Gilbert, G. N. (1991). Simulating speech systems. Computer Speech & Language, 5(1), 81-99. doi:10.1016/0885-2308(91)90019-mSchatzmann J. , Thomson B. and Young S. 2007. Statistical user simulation with a hidden agenda. In Proceedings of the SIGdial Workshop on Discourse and Dialogue, pp. 273–82.Levin L. , Ries K. , Thymé-Gobbel A. , and Levie A. 1999. Tagging of speech acts and dialogue games in Spanish call home. In Workshop: Towards Standards and Tools for Discourse Tagging, pp. 42–7.Hinarejos, C. D. M., Granell, R., & Benedí, J. M. (2006). Segmented and unsegmented dialogue-act annotation with statistical dialogue models. Proceedings of the COLING/ACL on Main conference poster sessions -. doi:10.3115/1273073.1273146Benedí J. M. , Lleida E. , Varona A. , Castro M. J. , Galiano I. , Justo R. , López de Letona I. , and Miguel A. 2006. Design and acquisition of a telephone spontaneous speech dialogue corpus in Spanish: Dihana. In Fifth International Conference on Language Resources and Evaluation (LREC), pp. 1636–9.Young, S. J. (2000). Probabilistic methods in spoken–dialogue systems. Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences, 358(1769), 1389-1402. doi:10.1098/rsta.2000.0593Martínez-Hinarejos, C.-D., Benedí, J.-M., & Granell, R. (2008). Statistical framework for a Spanish spoken dialogue corpus. Speech Communication, 50(11-12), 992-1008. doi:10.1016/j.specom.2008.05.011Bisani M. and Ney H. 2004. Bootstrap estimates for confidence intervals in asr performance evaluation. In Acoustics, Speech, and Signal Processing, 2004. Proceedings. (ICASSP '04). IEEE International Conference on, vol. 1, pp. 1:I–409–12.Garcia, P., & Vidal, E. (1990). Inference of k-testable languages in the strict sense and application to syntactic pattern recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(9), 920-925. doi:10.1109/34.57687Jurafsky D. , Shriberg E. and Biasca D. 1997. Switchboard SWBD-DAMSL shallow discourse function annotation coders manual - draft 13. Technical Report 97-01, University of Colorado Institute of Cognitive Science.Gorin, A. ., Riccardi, G., & Wright, J. . (1997). How may I help you? Speech Communication, 23(1-2), 113-127. doi:10.1016/s0167-6393(97)00040-xWalker, M. A. (2000). An Application of Reinforcement Learning to Dialogue Strategy Selection in a Spoken Dialogue System for Email. Journal of Artificial Intelligence Research, 12, 387-416. doi:10.1613/jair.713Core M. G. and Allen J. F. 2007. Coding dialogues with the DAMSL annotation scheme. In Fall Symposium on Communicative Action in Humans and Machines. American Association for Artificial Intelligence, pp. 28–35.Fukada T. , Koll D. , Waibel A. and Tanigaki K. 1998. Probabilistic dialogue act extraction for concept based multilingual translation systems. ICSLP 98 2771–4

Evaluation of a hierarchical reinforcement learning spoken dialogue system

We describe an evaluation of spoken dialogue strategies designed using hierarchical reinforcement learning agents. The dialogue strategies were learnt in a simulated environment and tested in a laboratory setting with 32 users. These dialogues were used to evaluate three types of machine dialogue behaviour: hand-coded, fully-learnt and semi-learnt. These experiments also served to evaluate the realism of simulated dialogues using two proposed metrics contrasted with ‘Precision-Recall’. The learnt dialogue behaviours used the Semi-Markov Decision Process (SMDP) model, and we report the first evaluation of this model in a realistic conversational environment. Experimental results in the travel planning domain provide evidence to support the following claims: (a) hierarchical semi-learnt dialogue agents are a better alternative (with higher overall performance) than deterministic or fully-learnt behaviour; (b) spoken dialogue strategies learnt with highly coherent user behaviour and conservative recognition error rates (keyword error rate of 20%) can outperform a reasonable hand-coded strategy; and (c) hierarchical reinforcement learning dialogue agents are feasible and promising for the (semi) automatic design of optimized dialogue behaviours in larger-scale systems

Displacement of One Stimulus Class Over Another Stimulus Class: A Systematic Replication

Previous researchers have found that individuals with intellectual and developmental disabilities tend to prefer edible over leisure stimuli and that leisure stimuli generally function as less effective reinforcers than edible stimuli, regardless of the preference patterns observed during a combined-class multiple-stimulus without replacement (MSWO) assessment. However, researchers have often arbitrarily selected items to include in these preference assessments and have not investigated this phenomenon with typically developing children. In Study 1, we evaluated the preference for leisure and edible stimuli in a combined-class MSWO assessment with 15 typically developing children. Five of 15 participants preferred edible stimuli over leisure stimuli, 3 of 15 participants preferred leisure stimuli over edible stimuli, and the remaining seven of 15 participants did not prefer one stimulus class over another. In Study 2, we compared the reinforcer potency of displaced stimuli and the stimuli that displaced them with 7 of 8 participants who showed displacement of one stimulus class over the other. Four of 7 participants allocated more responding to the free-operant task associated with the top-ranked stimulus identified in the combined-class MSWO, while 3 of 7 participants showed no differences in responding to the free-operant task regardless of ranking of the reinforcer delivered

An Application of Reinforcement Learning to Dialogue Strategy Selection in a Spoken Dialogue System for Email

This paper describes a novel method by which a spoken dialogue system can learn to choose an optimal dialogue strategy from its experience interacting with human users. The method i