Modular Approach to Machine Reading Comprehension: Mixture of Task-Aware Experts

In this work we present a Mixture of Task-Aware Experts Network for Machine Reading Comprehension on a relatively small dataset. We particularly focus on the issue of common-sense learning, enforcing the common ground knowledge by specifically training different expert networks to capture different kinds of relationships between each passage, question and choice triplet. Moreover, we take inspi ration on the recent advancements of multitask and transfer learning by training each network a relevant focused task. By making the mixture-of-networks aware of a specific goal by enforcing a task and a relationship, we achieve state-of-the-art results and reduce over-fitting

Deeper Understanding of Tutorial Dialogues and Student Assessment

Bloom (1984) reported two standard deviation improvement with human tutoring which inspired many researchers to develop Intelligent Tutoring Systems (ITSs) that are as effective as human tutoring. However, recent studies suggest that the 2-sigma result was misleading and that current ITSs are as good as human tutors. Nevertheless, we can think of 2 standard deviations as the benchmark for tutoring effectiveness of ideal expert tutors. In the case of ITSs, there is still the possibility that ITSs could be better than humans.One way to improve the ITSs would be identifying, understanding, and then successfully implementing effective tutorial strategies that lead to learning gains. Another step towards improving the effectiveness of ITSs is an accurate assessment of student responses. However, evaluating student answers in tutorial dialogues is challenging. The student answers often refer to the entities in the previous dialogue turns and problem description. Therefore, the student answers should be evaluated by taking dialogue context into account. Moreover, the system should explain which parts of the student answer are correct and which are incorrect. Such explanation capability allows the ITSs to provide targeted feedback to help students reflect upon and correct their knowledge deficits. Furthermore, targeted feedback increases learners\u27 engagement, enabling them to persist in solving the instructional task at hand on their own. In this dissertation, we describe our approach to discover and understand effective tutorial strategies employed by effective human tutors while interacting with learners. We also present various approaches to automatically assess students\u27 contributions using general methods that we developed for semantic analysis of short texts. We explain our work using generic semantic similarity approaches to evaluate the semantic similarity between individual learner contributions and ideal answers provided by experts for target instructional tasks. We also describe our method to assess student performance based on tutorial dialogue context, accounting for linguistic phenomena such as ellipsis and pronouns. We then propose an approach to provide an explanatory capability for assessing student responses. Finally, we recommend a novel method based on concept maps for jointly evaluating and interpreting the correctness of student responses

Retrospective and Prospective Mixture-of-Generators for Task-oriented Dialogue Response Generation

Dialogue response generation (DRG) is a critical component of task-oriented dialogue systems (TDSs). Its purpose is to generate proper natural language responses given some context, e.g., historical utterances, system states, etc. State-of-the-art work focuses on how to better tackle DRG in an end-to-end way. Typically, such studies assume that each token is drawn from a single distribution over the output vocabulary, which may not always be optimal. Responses vary greatly with different intents, e.g., domains, system actions. We propose a novel mixture-of-generators network (MoGNet) for DRG, where we assume that each token of a response is drawn from a mixture of distributions. MoGNet consists of a chair generator and several expert generators. Each expert is specialized for DRG w.r.t. a particular intent. The chair coordinates multiple experts and combines the output they have generated to produce more appropriate responses. We propose two strategies to help the chair make better decisions, namely, a retrospective mixture-of-generators (RMoG) and prospective mixture-of-generators (PMoG). The former only considers the historical expert-generated responses until the current time step while the latter also considers possible expert-generated responses in the future by encouraging exploration. In order to differentiate experts, we also devise a global-and-local (GL) learning scheme that forces each expert to be specialized towards a particular intent using a local loss and trains the chair and all experts to coordinate using a global loss. We carry out extensive experiments on the MultiWOZ benchmark dataset. MoGNet significantly outperforms state-of-the-art methods in terms of both automatic and human evaluations, demonstrating its effectiveness for DRG.Comment: The paper is accepted by 24th European Conference on Artificial Intelligenc

End-to-End Goal-Oriented Conversational Agent for Risk Awareness

Traditional development of goal-oriented conversational agents typically require a lot of domain-specific handcrafting, which precludes scaling up to different domains; end-to-end systems would escape this limitation because they can be trained directly from dialogues. The very promising success recently obtained in end-to-end chatbots development could carry over to goal-oriented settings: applying deep learning models for building robust and scalable goal-oriented dialog systems directly from corpora of conversations is a challenging task and an open research area. For this reason, I decided that it would have been more relevant in the context of a master's thesis to experiment and get acquainted with these new promising methodologies - although not yet ready for production - rather than investing time in hand-crafting dialogue rules for a domain-specific solution. My thesis work had the following macro objectives: (i) investigate the latest research works concerning goal-oriented conversational agents development; (ii) choose a reference study, understand it and implement it with an appropriate technology; (iii) apply what learnt to a particular domain of interest. As a reference framework I chose the end-to-end memory networks (MemN2N) (Sukhbaatar et al., 2015) because it has proven to be particularly promising and has been used as a baseline for many recent works. Not having real dialogues available for training though, I took care of synthetically generating a corpora of conversations, taking a cue from the Dialog bAbI dataset for restaurant reservations (Bordes et al., 2016) and adapting it to the new domain of interest of risk awareness. Finally, I built a simple prototype which exploited the pre-trained dialog model in order to advise users about risk through an anthropomorphic talking avatar interface