Challenging Neural Dialogue Models with Natural Data: Memory Networks Fail on Incremental Phenomena

Natural, spontaneous dialogue proceeds incrementally on a word-by-word basis; and it contains many sorts of disfluency such as mid-utterance/sentence hesitations, interruptions, and self-corrections. But training data for machine learning approaches to dialogue processing is often either cleaned-up or wholly synthetic in order to avoid such phenomena. The question then arises of how well systems trained on such clean data generalise to real spontaneous dialogue, or indeed whether they are trainable at all on naturally occurring dialogue data. To answer this question, we created a new corpus called bAbI+ by systematically adding natural spontaneous incremental dialogue phenomena such as restarts and self-corrections to the Facebook AI Research's bAbI dialogues dataset. We then explore the performance of a state-of-the-art retrieval model, MemN2N, on this more natural dataset. Results show that the semantic accuracy of the MemN2N model drops drastically; and that although it is in principle able to learn to process the constructions in bAbI+, it needs an impractical amount of training data to do so. Finally, we go on to show that an incremental, semantic parser -- DyLan -- shows 100% semantic accuracy on both bAbI and bAbI+, highlighting the generalisation properties of linguistically informed dialogue models.Comment: 9 pages, 3 figures, 2 tables. Accepted as a full paper for SemDial 201

Parsing Manually Detected andNnormalized Disfluencies in Spoken Estonian

Proceedings of the 16th Nordic Conference of Computational Linguistics NODALIDA-2007. Editors: Joakim Nivre, Heiki-Jaan Kaalep, Kadri Muischnek and Mare Koit. University of Tartu, Tartu, 2007. ISBN 978-9985-4-0513-0 (online) ISBN 978-9985-4-0514-7 (CD-ROM) pp. 363-366

Guidelines for digital storytelling for Arab children

Children are getting more exposed to various technologies in teaching-learning. Various types of teaching-learning have been designed, including interactive digital storytelling. In Malaysia, local children have been clear about story-based learning materials. However, the situation is a little bit different with Arab children. Because the number of Arab children migrating into Malaysia is increasing, for following their parents who are studying at higher levels, they have to also make themselves familiar with the local scenario. In accordance, this study is initiates, to identify their acceptance towards story-based learning materials, or specifically interactive digital storytelling. Hence, this study reacts proactively, by approaching Arab children asking for their feedback on whether they have any desire for interactive digital storytelling. Through a series of interviews, this study found that they have a strong desire and tendency. Then, the following objectives have been stated: (1) to determine the components for the interactive digital storytelling for Arab children, (2) to design and develop a prototype of the interactive digital storytelling, and (3) to observe on how the Arab children experience the interactive digital storytelling. User-centered design (UCD) approach has been gone through in ensuring that the objectives are achieved. The process of determining the components for the interactive digital storytelling was carried out by directly involving Arab children and their teachers from three preschools in Changlun and Sintok. It was similar with the efforts in determining the contents, and interface design until the prototype development. Having the prototype ready, user testing was carried out to explore the way Arab children experience the prototype. All the processes involved various techniques through observation, interviews, and noting. Specifically, the user testing involved qualitative and empirical data. Qualitative data were gathered through observation, meanwhile the empirical data were gathered using Computer System Usability Questionnaire (CSUQ) tool. In the end, having data processed, the findings show that Arab children are highly satisfied with the prototype. Scientifically, the developed prototype is a mirror of the obtained guidelines, obtained through the UCD seminars. Hence, the positive acceptance on the prototype reflects positive acceptance on the guidelines, as the main contribution of this study. Besides the guidelines as the main contribution of this study, the developed prototype is also a wonderful contribution to the Arab children and their teacher. They will be using it as part of their teaching and learning material

Conceptual model for usable multi-modal mobile assistance during Umrah

Performing Umrah is very demanding and to be performed in very crowded environments. In response to that, many efforts have been initiated to overcome the difficulties faced by pilgrims. However, those efforts focus on acquiring initial perspective and background knowledge before going to Mecca. Findings of preliminary study show that those efforts do not support multi-modality for user interaction. Nowadays the computational capabilities in mobile phones enable it to serve people in various aspects of daily life. Consequently, the mobile phone penetration has increased dramatically in the last decade. Hence, this study aims to propose a comprehensive conceptual model for usable multimodal mobile assistance during Umrah called Multi-model Mobile Assistance during Umrah (MMA-U). Thus, four (4) supporting objectives are formulated, and the Design Science Research Methodology has been adopted. For the usability of MMA-U, Systematic Literature Review (SLR) indicates ten (10) attributes: usefulness, errors rate, simplicity, reliability, ease of use, safety, flexibility, accessibility, attitude, and acceptability. Meanwhile, the content and comparative analysis result in five (5) components that construct the conceptual model of MMA-U: structural, content composition, design principles, development approach, technology, and the design and usability theories. Then, the MMA-U has been reviewed and well-accepted by 15 experts. Later, the MMA-U was incorporated into a prototype called Personal Digital Mutawwif (PDM). The PDM was developed for the purpose of user test in the field. The findings indicate that PDM facilitates the execution of Umrah and successfully meet pilgrims’ needs and expectations. Also, the pilgrims were satisfied and felt that they need to have PDM. In fact, they would recommend PDM to their friends, which mean that use of PDM is safe and suitable while performing Umrah. As a conclusion, the theoretical contribution; the conceptual model of MMA-U; provides guidelines for developing multimodal content mobile applications during Umrah

Data-efficient methods for dialogue systems

Conversational User Interface (CUI) has become ubiquitous in everyday life, in consumer-focused products like Siri and Alexa or more business-oriented customer support automation solutions. Deep learning underlies many recent breakthroughs in dialogue systems but requires very large amounts of training data, often annotated by experts — and this dramatically increases the cost of deploying such systems in production setups and reduces their flexibility as software products. Trained with smaller data, these methods end up severely lacking robustness to various phenomena of spoken language (e.g. disfluencies), out-of-domain input, and often just have too little generalisation power to other tasks and domains. In this thesis, we address the above issues by introducing a series of methods for bootstrapping robust dialogue systems from minimal data. Firstly, we study two orthogonal approaches to dialogue: a linguistically informed model (DyLan) and a machine learning-based one (MemN2N) — from the data efficiency perspective, i.e. their potential to generalise from minimal data and robustness to natural spontaneous input. We outline the steps to obtain data-efficient solutions with either approach and proceed with the neural models for the rest of the thesis. We then introduce the core contributions of this thesis, two data-efficient models for dialogue response generation: the Dialogue Knowledge Transfer Network (DiKTNet) based on transferable latent dialogue representations, and the Generative-Retrieval Transformer (GRTr) combining response generation logic with a retrieval mechanism as the fallback. GRTr ranked first at the Dialog System Technology Challenge 8 Fast Domain Adaptation task. Next, we the problem of training robust neural models from minimal data. As such, we look at robustness to disfluencies and propose a multitask LSTM-based model for domain-general disfluency detection. We then go on to explore robustness to anomalous, or out-of-domain (OOD) input. We address this problem by (1) presenting Turn Dropout, a data-augmentation technique facilitating training for anomalous input only using in-domain data, and (2) introducing VHCN and AE-HCN, autoencoder-augmented models for efficient training with turn dropout based on the Hybrid Code Networks (HCN) model family. With all the above work addressing goal-oriented dialogue, our final contribution in this thesis focuses on social dialogue where the main objective is maintaining natural, coherent, and engaging conversation for as long as possible. We introduce a neural model for response ranking in social conversation used in Alana, the 3rd place winner in the Amazon Alexa Prize 2017 and 2018. For our model, we employ a novel technique of predicting the dialogue length as the main objective for ranking. We show that this approach matches the performance of its counterpart based on the conventional, human rating-based objective — and surpasses it given more raw dialogue transcripts, thus reducing the dependence on costly and cumbersome dialogue annotations.EPSRC project BABBLE (grant EP/M01553X/1)

READING RECOVERY CHILDREN AND EARLY LITERACY DEVELOPMENT: INVESTIGATION INTO PHONOLOGICAL AWARENESS, ORTHOGRAPHIC KNOWLEDGE, ORAL READING PROCESSING, AND READING COMPREHENSION PROCESSING

Marie Clay (1993) designed Reading Recovery tutoring to accelerate the early literacy development of low-performing, six-year-old children so that they achieve average levels of classroom performance. Approximately one third of the proportion of the first cohort of U.S. children who receive Reading Recovery tutoring at the beginning of a school year respond poorly to it (Gómez-Bellengé, Rodgers, & Fullerton, 2003). They fail to meet the criteria for successful performance and their Reading Recovery teachers recommend them for additional assessment and/or consideration for other supplemental instruction. An emerging program of research suggests that recommended children struggle in early literacy development. This study compared recommended to discontinued Reading Recovery children on phonological awareness and orthographic knowledge at pre- and post-tutoring, and oral reading processing and reading comprehension processing at post-tutoring. The sample consisted of 29 recommended children and 26 discontinued children who were taught by 16 trained Reading Recovery teachers in a single school district. This study contributes to the understanding of recommended children's early literacy development. Analysis of phonological awareness and orthographic knowledge composite data revealed that recommended children demonstrated less overall phonological awareness and overall orthographic knowledge than discontinued children and that recommended and discontinued children combined displayed gains from pre- to post-tutoring at statistically significant levels. Analyses of the phonological awareness composite data revealed that recommended children performed at a level below discontinued children on rhyme awareness at pre-tutoring, phonological skeletal structure awareness at pre- and post-tutoring, and graphophonemic awareness with respect to beginning phonemes at pre- and post-tutoring and ending phonemes at pre-tutoring at statistically significant levels. Analyses of the orthographic knowledge composite data revealed that recommended children performed at a level below discontinued children on orthographic acceptability knowledge at pre- tutoring and spelling knowledge at post-tutoring at statistically significant levels. Analyses of oral reading processing data at post-tutoring revealed that recommended children read stories with less accuracy, more overall errors, more substitutions, less fluency, and at a slower rate than discontinued children at statistically significant levels. An analysis of reading comprehension processing data at post-tutoring revealed that the two groups comprehended the stories nearly equivalently