Using Mixed Media Tools for Eliciting Discourse in Indigenous Languages

Prosody plays a vital role in communication, but is one of the most widely neglected topics in language documentation. This omission is doubly detrimental since intonation is unrecoverable from transcribed texts, the most prevalent data sources for many indigenous languages. One of the underlying reasons for the dearth of prosodic data is methodological. Modern technology has removed technical barriers to recording the appropriate data, but traditional methods of elicitation still inhibit accurate documentation of linguistic structures at or above the phrasal level. In addition, these methods do not facilitate the mobilization of linguistic documentation. In this paper, we present techniques that we have developed that address both these concerns: 1) eliciting prosodic data for theoretical analysis, and 2) producing linguistic materials that can be useful for educators and curriculum developers. Highlighting advantages and disadvantages, we compare traditional elicitation and text-gathering methods with two non-traditional methodologies using non-verbal stimuli. These two non-traditional methodologies are aimed at collecting: 1) spontaneous conversation (either unguided, or task-oriented), and 2) partly scripted conversation (aided by multimedia tools). The methodologies are illustrated with original fieldwork on focus and intonation in two related, endangered Interior Salish languages – Nlhe7kepmxcín (Thompson) and St’át’imcets (Lillooet).National Foreign Language Resource Cente

Evaluation of Interventions in Blended Learning Using a Communication Skills Serious Game

Serious games often employ a scripted dialogue for player interaction with a virtual character. In our serious game Communicate, a domain expert develops a structured, scripted scenario as a sequence of potential interactions in an authoring tool. A player is often a student learning communication skills and a virtual character represents a person that a student talks to. In the original version of Communicate, a player `converses' with a virtual character by clicking on one of the multiple statement options. Since 2018, we perform blended learning sessions for final year computer science students using Communicate. Our goal is to improve these sessions and in this paper, we apply the action research method over three semesters to iteratively improve these blended learning sessions. In the first semester, our baseline, we conduct sessions where students play a scenario in multiple choice format. In the second semester, we enhance Communicate by enabling a student to enter open text input in an improved scenario. In the third semester, we enhance a session by incorporating peer teaching. Students fill in an evaluation survey after a session and we compare the evaluation of students from the three semesters. Results show that student ratings are significantly higher in sessions incorporating peer teaching compared to the baseline

Interpreting Human Responses in Dialogue Systems using Fuzzy Semantic Similarity Measures

Dialogue systems are automated systems that interact with humans using natural language. Much work has been done on dialogue management and learning using a range of computational intelligence based approaches, however the complexity of human dialogue in different contexts still presents many challenges. The key impact of work presented in this paper is to use fuzzy semantic similarity measures embedded within a dialogue system to allow a machine to semantically comprehend human utterances in a given context and thus communicate more effectively with a human in a specific domain using natural language. To achieve this, perception based words should be understood by a machine in context of the dialogue. In this work, a simple question and answer dialogue system is implemented for a café customer satisfaction feedback survey. Both fuzzy and crisp semantic similarity measures are used within the dialogue engine to assess the accuracy and robustness of rule firing. Results from a 32 participant study, show that the fuzzy measure improves rule matching within the dialogue system by 21.88% compared with the crisp measure known as STASIS, thus providing a more natural and fluid dialogue exchange

Development of an Arabic conversational intelligent tutoring system for education of children with autism spectrum disorder

Children with Autism Spectrum Disorder (ASD) are affected in different degrees in terms of their level of intellectual ability. Some people with Asperger syndrome or high functioning autism are very intelligent academically but they still have difficulties in social and communication skills. In recent years, many of these pupils are taught within mainstream schools. However, the process of facilitating their learning and participation remains a complex and poorly understood area of education. Although many teachers in mainstream schools are firmly committed to the principles of inclusive education, they do not feel that they have the necessary training and support to provide adequately for pupils with ASD. One solution for this problem is to use a virtual tutor to supplement the education of pupils with ASD in mainstream schools. This thesis describes research to develop a Novel Arabic Conversational Intelligent Tutoring System (CITS), called LANA, for children with ASD, which delivers topics related to the science subject by engaging with the user in Arabic language. The Visual, Auditory, and Kinaesthetic (VAK) learning style model is used in LANA to adapt to the children’s learning style by personalising the tutoring session. Development of an Arabic Conversational Agent has many challenges. Part of the challenge in building such a system is the requirement to deal with the grammatical features and the morphological nature of the Arabic language. The proposed novel architecture for LANA uses both pattern matching (PM) and a new Arabic short text similarity (STS) measure to extract facts from user’s responses to match rules in scripted conversation in a particular domain (Science). In this research, two prototypes of an Arabic CITS were developed (LANA-I) and (LANA-II). LANA-I was developed and evaluated with 24 neurotypical children to evaluate the effectiveness and robustness of the system engine. LANA-II was developed to enhance LANA-I by addressing spelling mistakes and words variation with prefix and suffix. Also in LANA-II, TEACCH method was added to the user interface to adapt the tutorial environment to the autistic students learning, and the knowledge base was expanded by adding a new tutorial. An evaluation methodology and experiment were designed to evaluate the enhanced components of LANA-II architecture. The results illustrated a statistically significant impact on the effectiveness of LANA-II engine when compared to LANA-I. In addition, the results indicated a statistically significant improvement on the autistic students learning gain with adapting to their learning styles indicating that LANA-II can be adapted to autistic children’s learning styles and enhance their learning

Arabic conversational agent for modern Islamic education

This thesis presents research that combines the benefits of intelligent tutoring systems (ITS), Arabic conversational agents (CA) and learning theories by constructing a novel Arabic conversational intelligent tutoring system (CITS) called Abdullah. Abdullah CITS is a software program intended to deliver a tutorial to students aged between 10 and 12 years old, that covers the essential topics in Islam using natural language. The CITS aims to mimic a human Arabic tutor by engaging the students in dialogue using Modern standard Arabic language (MSA), whilst also allowing conversation and discussion in classical Arabic language (CAL). Developing a CITS for the Arabic language faces many challenges due to the complexity of the morphological system, non-standardization of the written text, ambiguity, and lack of resources. However, the main challenge for the developed Arabic CITS is how the user utterances are recognized and responded to by the CA, as well as how the domain is scripted and maintained. This research presents a novel Arabic CA and accompanying a scripting language that use a form of pattern matching, to handle users’ conversations when the user converse in MSA. A short text similarity measure is used within Abdullah CITS to extract the responses from CAL resources such as the Quran, Hadith, and Tafsir if there are no matching patterns with the Arabic conversation agent’s scripts. Abdullah CITS is able to capture the user’s level of knowledge and adapt the tutoring session and tutoring style to suit that particular learner’s level of knowledge. This is achieved through the inclusion of several learning theories and methods such as Gagne’s learning theory, Piaget learning theory, and storytelling method. These learning theories and methods implemented within Abdullah’s CITS architecture, are applied to personalise a tutorial to an individual learner. This research presents the first Arabic CITS, which utilises established learning typically employed in a classroom environment. The system was evaluated through end user testing with the target age group in schools both in Jordan and in the UK. Empirical experimentation has produced some positive results, indicating that Abdullah CITS is gauging the individual learner’s knowledge level and adapting the tutoring session to ensure learning gain is achieved

Hexa: Self-Improving for Knowledge-Grounded Dialogue System

A common practice in knowledge-grounded dialogue generation is to explicitly utilize intermediate steps (e.g., web-search, memory retrieval) with modular approaches. However, data for such steps are often inaccessible compared to those of dialogue responses as they are unobservable in an ordinary dialogue. To fill in the absence of these data, we develop a self-improving method to improve the generative performances of intermediate steps without the ground truth data. In particular, we propose a novel bootstrapping scheme with a guided prompt and a modified loss function to enhance the diversity of appropriate self-generated responses. Through experiments on various benchmark datasets, we empirically demonstrate that our method successfully leverages a self-improving mechanism in generating intermediate and final responses and improves the performances on the task of knowledge-grounded dialogue generation

Proceedings of QG2010: The Third Workshop on Question Generation

These are the peer-reviewed proceedings of "QG2010, The Third Workshop on Question Generation". The workshop included a special track for "QGSTEC2010: The First Question Generation Shared Task and Evaluation Challenge". QG2010 was held as part of The Tenth International Conference on Intelligent Tutoring Systems (ITS2010)

A Survey of Available Corpora For Building Data-Driven Dialogue Systems: The Journal Version

During the past decade, several areas of speech and language understanding have witnessed substantial breakthroughs from the use of data-driven models. In the area of dialogue systems, the trend is less obvious, and most practical systems are still built through significant engineering and expert knowledge. Nevertheless, several recent results suggest that data-driven approaches are feasible and quite promising. To facilitate research in this area, we have carried out a wide survey of publicly available datasets suitable for data-driven learning of dialogue systems. We discuss important characteristics of these datasets, how they can be used to learn diverse dialogue strategies, and their other potential uses. We also examine methods for transfer learning between datasets and the use of external knowledge. Finally, we discuss appropriate choice of evaluation metrics for the learning objective