TOWARDS BUILDING INTELLIGENT COLLABORATIVE PROBLEM SOLVING SYSTEMS

Historically, Collaborative Problem Solving (CPS) systems were more focused on Human Computer Interaction (HCI) issues, such as providing good experience of communication among the participants. Whereas, Intelligent Tutoring Systems (ITS) focus both on HCI issues as well as leveraging Artificial Intelligence (AI) techniques in their intelligent agents. This dissertation seeks to minimize the gap between CPS systems and ITS by adopting the methods used in ITS researches. To move towards this goal, we focus on analyzing interactions with textual inputs in online learning systems such as DeepTutor and Virtual Internships (VI) to understand their semantics and underlying intents. In order to address the problem of assessing the student generated short text, this research explores firstly data driven machine learning models coupled with expert generated as well as general text analysis features. Secondly it explores method to utilize knowledge graph embedding for assessing student answer in ITS. Finally, it also explores a method using only standard reference examples generated by human teacher. Such method is useful when a new system has been deployed and no student data were available.To handle negation in tutorial dialogue, this research explored a Long Short Term Memory (LSTM) based method. The advantage of this method is that it requires no human engineered features and performs comparably well with other models using human engineered features.Another important analysis done in this research is to find speech acts in conversation utterances of multiple players in VI. Among various models, a noise label trained neural network model performed better in categorizing the speech acts of the utterances.The learners\u27 professional skill development in VI is characterized by the distribution of SKIVE elements, the components of epistemic frames. Inferring the population distribution of these elements could help to assess the learners\u27 skill development. This research sought a Markov method to infer the population distribution of SKIVE elements, namely the stationary distribution of the elements.While studying various aspects of interactions in our targeted learning systems, we motivate our research to replace the human mentor or tutor with intelligent agent. Introducing intelligent agent in place of human helps to reduce the cost as well as scale up the system

Measuring Semantic Similarity: Representations and Methods

This dissertation investigates and proposes ways to quantify and measure semantic similarity between texts. The general approach is to rely on linguistic information at various levels, including lexical, lexico-semantic, and syntactic. The approach starts by mapping texts onto structured representations that include lexical, lexico-semantic, and syntactic information. The representation is then used as input to methods designed to measure the semantic similarity between texts based on the available linguistic information.While world knowledge is needed to properly assess semantic similarity of texts, in our approach world knowledge is not used, which is a weakness of it.We limit ourselves to answering the question of how successfully one can measure the semantic similarity of texts using just linguistic information.The lexical information in the original texts is retained by using the words in the corresponding representations of the texts. Syntactic information is encoded using dependency relations trees, which represent explicitly the syntactic relations between words. Word-level semantic information is relatively encoded through the use of semantic similarity measures like WordNet Similarity or explicitly encoded using vectorial representations such as Latent Semantic Analysis (LSA). Several methods are being studied to compare the representations, ranging from simple lexical overlap, to more complex methods such as comparing semantic representations in vector spaces as well as syntactic structures. Furthermore, a few powerful kernel models are proposed to use in combination with Support Vector Machine (SVM) classifiers for the case in which the semantic similarity problem is modeled as a classification task

The Value of Everything: Ranking and Association with Encyclopedic Knowledge

This dissertation describes WikiRank, an unsupervised method of assigning relative values to elements of a broad coverage encyclopedic information source in order to identify those entries that may be relevant to a given piece of text. The valuation given to an entry is based not on textual similarity but instead on the links that associate entries, and an estimation of the expected frequency of visitation that would be given to each entry based on those associations in context. This estimation of relative frequency of visitation is embodied in modifications to the random walk interpretation of the PageRank algorithm. WikiRank is an effective algorithm to support natural language processing applications. It is shown to exceed the performance of previous machine learning algorithms for the task of automatic topic identification, providing results comparable to that of human annotators. Second, WikiRank is found useful for the task of recognizing text-based paraphrases on a semantic level, by comparing the distribution of attention generated by two pieces of text using the encyclopedic resource as a common reference. Finally, WikiRank is shown to have the ability to use its base of encyclopedic knowledge to recognize terms from different ontologies as describing the same thing, and thus allowing for the automatic generation of mapping links between ontologies. The conclusion of this thesis is that the "knowledge access heuristic" is valuable and that a ranking process based on a large encyclopedic resource can form the basis for an extendable general purpose mechanism capable of identifying relevant concepts by association, which in turn can be effectively utilized for enumeration and comparison at a semantic level

Development of an Arabic conversational intelligent tutoring system for education of children with autism spectrum disorder

Children with Autism Spectrum Disorder (ASD) are affected in different degrees in terms of their level of intellectual ability. Some people with Asperger syndrome or high functioning autism are very intelligent academically but they still have difficulties in social and communication skills. In recent years, many of these pupils are taught within mainstream schools. However, the process of facilitating their learning and participation remains a complex and poorly understood area of education. Although many teachers in mainstream schools are firmly committed to the principles of inclusive education, they do not feel that they have the necessary training and support to provide adequately for pupils with ASD. One solution for this problem is to use a virtual tutor to supplement the education of pupils with ASD in mainstream schools. This thesis describes research to develop a Novel Arabic Conversational Intelligent Tutoring System (CITS), called LANA, for children with ASD, which delivers topics related to the science subject by engaging with the user in Arabic language. The Visual, Auditory, and Kinaesthetic (VAK) learning style model is used in LANA to adapt to the children’s learning style by personalising the tutoring session. Development of an Arabic Conversational Agent has many challenges. Part of the challenge in building such a system is the requirement to deal with the grammatical features and the morphological nature of the Arabic language. The proposed novel architecture for LANA uses both pattern matching (PM) and a new Arabic short text similarity (STS) measure to extract facts from user’s responses to match rules in scripted conversation in a particular domain (Science). In this research, two prototypes of an Arabic CITS were developed (LANA-I) and (LANA-II). LANA-I was developed and evaluated with 24 neurotypical children to evaluate the effectiveness and robustness of the system engine. LANA-II was developed to enhance LANA-I by addressing spelling mistakes and words variation with prefix and suffix. Also in LANA-II, TEACCH method was added to the user interface to adapt the tutorial environment to the autistic students learning, and the knowledge base was expanded by adding a new tutorial. An evaluation methodology and experiment were designed to evaluate the enhanced components of LANA-II architecture. The results illustrated a statistically significant impact on the effectiveness of LANA-II engine when compared to LANA-I. In addition, the results indicated a statistically significant improvement on the autistic students learning gain with adapting to their learning styles indicating that LANA-II can be adapted to autistic children’s learning styles and enhance their learning

Adaptive computer assisted assessment of free-text students' answers: an approach to automatically generate students' conceptual models

Tesis doctoral inédita. Universidad Autónoma de Madrid, Escuela Politécnica Superior, mayo de 200

Representation learning for dialogue systems

Cette thèse présente une série de mesures prises pour étudier l’apprentissage de représentations (par exemple, l’apprentissage profond) afin de mettre en place des systèmes de dialogue et des agents de conversation virtuels. La thèse est divisée en deux parties générales. La première partie de la thèse examine l’apprentissage des représentations pour les modèles de dialogue génératifs. Conditionnés sur une séquence de tours à partir d’un dialogue textuel, ces modèles ont la tâche de générer la prochaine réponse appropriée dans le dialogue. Cette partie de la thèse porte sur les modèles séquence-à-séquence, qui est une classe de réseaux de neurones profonds génératifs. Premièrement, nous proposons un modèle d’encodeur-décodeur récurrent hiérarchique ("Hierarchical Recurrent Encoder-Decoder"), qui est une extension du modèle séquence-à-séquence traditionnel incorporant la structure des tours de dialogue. Deuxièmement, nous proposons un modèle de réseau de neurones récurrents multi-résolution ("Multiresolution Recurrent Neural Network"), qui est un modèle empilé séquence-à-séquence avec une représentation stochastique intermédiaire (une "représentation grossière") capturant le contenu sémantique abstrait communiqué entre les locuteurs. Troisièmement, nous proposons le modèle d’encodeur-décodeur récurrent avec variables latentes ("Latent Variable Recurrent Encoder-Decoder"), qui suivent une distribution normale. Les variables latentes sont destinées à la modélisation de l’ambiguïté et l’incertitude qui apparaissent naturellement dans la communication humaine. Les trois modèles sont évalués et comparés sur deux tâches de génération de réponse de dialogue: une tâche de génération de réponses sur la plateforme Twitter et une tâche de génération de réponses de l’assistance technique ("Ubuntu technical response generation task"). La deuxième partie de la thèse étudie l’apprentissage de représentations pour un système de dialogue utilisant l’apprentissage par renforcement dans un contexte réel. Cette partie porte plus particulièrement sur le système "Milabot" construit par l’Institut québécois d’intelligence artificielle (Mila) pour le concours "Amazon Alexa Prize 2017". Le Milabot est un système capable de bavarder avec des humains sur des sujets populaires à la fois par la parole et par le texte. Le système consiste d’un ensemble de modèles de récupération et de génération en langage naturel, comprenant des modèles basés sur des références, des modèles de sac de mots et des variantes des modèles décrits ci-dessus. Cette partie de la thèse se concentre sur la tâche de sélection de réponse. À partir d’une séquence de tours de dialogues et d’un ensemble des réponses possibles, le système doit sélectionner une réponse appropriée à fournir à l’utilisateur. Une approche d’apprentissage par renforcement basée sur un modèle appelée "Bottleneck Simulator" est proposée pour sélectionner le candidat approprié pour la réponse. Le "Bottleneck Simulator" apprend un modèle approximatif de l’environnement en se basant sur les trajectoires de dialogue observées et le "crowdsourcing", tout en utilisant un état abstrait représentant la sémantique du discours. Le modèle d’environnement est ensuite utilisé pour apprendre une stratégie d’apprentissage du renforcement par le biais de simulations. La stratégie apprise a été évaluée et comparée à des approches concurrentes via des tests A / B avec des utilisateurs réel, où elle démontre d’excellente performance.This thesis presents a series of steps taken towards investigating representation learning (e.g. deep learning) for building dialogue systems and conversational agents. The thesis is split into two general parts. The first part of the thesis investigates representation learning for generative dialogue models. Conditioned on a sequence of turns from a text-based dialogue, these models are tasked with generating the next, appropriate response in the dialogue. This part of the thesis focuses on sequence-to-sequence models, a class of generative deep neural networks. First, we propose the Hierarchical Recurrent Encoder-Decoder model, which is an extension of the vanilla sequence-to sequence model incorporating the turn-taking structure of dialogues. Second, we propose the Multiresolution Recurrent Neural Network model, which is a stacked sequence-to-sequence model with an intermediate, stochastic representation (a "coarse representation") capturing the abstract semantic content communicated between the dialogue speakers. Third, we propose the Latent Variable Recurrent Encoder-Decoder model, which is a variant of the Hierarchical Recurrent Encoder-Decoder model with latent, stochastic normally-distributed variables. The latent, stochastic variables are intended for modelling the ambiguity and uncertainty occurring naturally in human language communication. The three models are evaluated and compared on two dialogue response generation tasks: a Twitter response generation task and the Ubuntu technical response generation task. The second part of the thesis investigates representation learning for a real-world reinforcement learning dialogue system. Specifically, this part focuses on the Milabot system built by the Quebec Artificial Intelligence Institute (Mila) for the Amazon Alexa Prize 2017 competition. Milabot is a system capable of conversing with humans on popular small talk topics through both speech and text. The system consists of an ensemble of natural language retrieval and generation models, including template-based models, bag-of-words models, and variants of the models discussed in the first part of the thesis. This part of the thesis focuses on the response selection task. Given a sequence of turns from a dialogue and a set of candidate responses, the system must select an appropriate response to give the user. A model-based reinforcement learning approach, called the Bottleneck Simulator, is proposed for selecting the appropriate candidate response. The Bottleneck Simulator learns an approximate model of the environment based on observed dialogue trajectories and human crowdsourcing, while utilizing an abstract (bottleneck) state representing high-level discourse semantics. The learned environment model is then employed to learn a reinforcement learning policy through rollout simulations. The learned policy has been evaluated and compared to competing approaches through A/B testing with real-world users, where it was found to yield excellent performance

Digital writing technologies in higher education : theory, research, and practice

This open access book serves as a comprehensive guide to digital writing technology, featuring contributions from over 20 renowned researchers from various disciplines around the world. The book is designed to provide a state-of-the-art synthesis of the developments in digital writing in higher education, making it an essential resource for anyone interested in this rapidly evolving field. In the first part of the book, the authors offer an overview of the impact that digitalization has had on writing, covering more than 25 key technological innovations and their implications for writing practices and pedagogical uses. Drawing on these chapters, the second part of the book explores the theoretical underpinnings of digital writing technology such as writing and learning, writing quality, formulation support, writing and thinking, and writing processes. The authors provide insightful analysis on the impact of these developments and offer valuable insights into the future of writing. Overall, this book provides a cohesive and consistent theoretical view of the new realities of digital writing, complementing existing literature on the digitalization of writing. It is an essential resource for scholars, educators, and practitioners interested in the intersection of technology and writing

Towards adaptive argumentation learning systems : theoretical and practical considerations in the design of argumentation learning systems

This dissertation addresses four issues of pivotal importance in realizing the promises of adaptive argumentation learning systems: (1) User interface: How can argumentation user interfaces be designed to effectively structure and support problem solving, peer interaction, and learning? (2) Software architecture: How can software architectures of adaptive argumentation learning systems be designed to be employable across different argumentation domains and application scenarios in a flexible and cost-effective manner? (3) Diagnostics: How can user behavior be analyzed, automatically and accurately, to drive automated adaptations and help generation? (4) Adaptation: How can strategies for automated adaptation and support be designed to promote problem solving, peer interaction, and learning in an optimal fashion? Regarding issue (1), this dissertation investigates argument diagrams and structured discussion interfaces, two areas of focal interest in argumentation learning research during the past decades. The foundation for such structuring approaches is given by theories of learning and teaching with knowledge representations (theory of representational guidance) and collaboration scripts (script theory of guidance in computer-supported collaborative learning). This dissertation brings these two strands of research together and presents a computer-based learning environment that combines both approaches to support students in conducting high-quality discussions of controversial texts. An empirical study confirms that this combined approach has positive impact on the quality of discussions, thus, underpins the theoretical basis of the approach. Regarding issue (2), this dissertation presents a software framework for enhancing argumentation systems with adaptive support mechanisms. Adaptive support functionality of past argumentation systems has been tailored to particular domains and application scenarios. A novel software framework is presented that abstracts from the specific demands of different domains and application scenarios to provide a more general approach. The approach comprises an extensive configuration subsystem that allows the flexible definition of intelligent software agents, that is, software components able to reason and act autonomously to help students engage in fruitful learning activities. A graphical authoring tool has been conceptualized and implemented to simplify the process of defining and administering software agents beyond what has been achieved with the provided framework system. Among other things, the authoring tool allows, for the first time, specifying relevant patterns in argument diagrams using a graphical language. Empirical results indicate the high potential of the authoring approach but also challenges for future research. Regarding issue (3), the dissertation investigates two alternative approaches to automatically analyzing argumentation learning activities: the knowledge-driven and the data-driven analysis method. The knowledge-driven approach utilizes a pattern search component to identify relevant structures in argument diagrams based on declarative pattern specifications. The capabilities and appropriateness of this approach are demonstrated through three exemplary applications, for which pedagogically relevant patterns have been defined and implemented within the component. The approach proves particularly useful for patterns of limited complexity in scenarios with sufficient expert knowledge available. The data-driven approach is based on machine learning techniques, which have been employed to induce computational classifiers for important aspects of graphical online discussions, such as off-topic contributions, reasoned claims, and question-answer interactions. Validation results indicate that this approach can be realistically used even for complex classification tasks involving natural language. This research constitutes the first investigation on the use of machine learning techniques to analyze diagram-based educational discussions. The dissertation concludes with discussing the four addressed research challenges in the broader context of existing theories and empirical results. The pros and cons of different options in the design of argumentation learning systems are juxtaposed; areas for future research are identified. This final part of the dissertation gives researchers and practitioners a synopsis of the current state of the art in the design of argumentation learning systems and its theoretical and empirical underpinning. Special attention is paid to issue (4), with an in-depth discussion of existing adaptation approaches and corresponding empirical results.Diese Dissertationsschrift behandelt die folgenden vier Fragestellungen, welche bei der Realisierung adaptiver Argumentationssysteme von zentraler Bedeutung sind: (1) Benutzerschnittstelle: Wie müssen Benutzerschnittstellen beschaffen sein, um Problemlöse-, Kooperations- und Lernprozesse effektiv zu strukturieren und zu unterstützen? (2) Softwarearchitektur: Wie können die Funktionalitäten eines adaptiven Argumentationslernsystems in eine Softwarearchitektur abgebildet werden, welche flexibel und mit angemessenem Aufwand in verschiedenen Bereichen und Szenarien einsetzbar ist? (3) Diagnostik: Wie kann Benutzerverhalten automatisch und mit hoher Genauigkeit analysiert werden, um automatisierte Anpassungen und Hilfestellungen effektiv zu steuern? (4) Adaption: Wie sollten automatisierte Anpassungen und Hilfestellungen ausgestaltet werden, um Problemlöse-, Kooperations- und Lernprozesse optimal zu unterstützen? Hinsichtlich Fragestellung (1) untersucht diese Arbeit Argumentationsdiagramme und strukturierte Onlinediskussionen, zwei Schwerpunkte der Forschung zu Lernsystemen für Argumentation der vergangenen Jahre. Die Grundlage solcher Strukturierungsansätze bilden Theorien zum Lehren und Lernen mit Wissensrepräsentationen (theory of representational guidance) und Kooperationsskripten (script theory of guidance in computer-supported collaborative learning). Diese Arbeit führt beide Forschungsstränge in einer neuartigen Lernumgebung zusammen, die beide Ansätze vereint, um Lernende beim Diskutieren kontroverser Texte zu unterstützen. Eine empirische Untersuchung zeigt, dass sich dieser kombinierte Ansatz positiv auf die Diskussionsqualität auswirkt und bekräftigt damit die zu Grunde liegenden theoretischen Annahmen. Hinsichtlich Fragestellung (2) stellt diese Arbeit ein Software-Rahmensystem zur Bereitstellung adaptiver Unterstützungsmechanismen in Argumentationssystemen vor. Das Rahmensystem abstrahiert von domänen- und anwendungsspezifischen Besonderheiten und stellt damit einen generelleren Ansatz im Vergleich zu früheren Systemen dar. Der Ansatz umfasst ein umfangreiches Konfigurationssystem zur Definition intelligenter Softwareagenten, d. h. Softwarekomponenten, die eigeständig schlussfolgern und handeln, um Lernprozesse zu unterstützen. Um das Definieren und Administrieren von Softwareagenten über das bereitgestellte Rahmensystem hinaus zu vereinfachen, wurde ein grafisches Autorenwerkzeug konzipiert und entwickelt. Unter anderem erlaubt dieses erstmals, relevante Muster in Argumentationsdiagrammen ohne Programmierung mittels einer grafischen Sprache zu spezifizieren. Empirische Befunde zeigen neben dem hohen Potential des Ansatzes auch die Notwendigkeit weiterführender Forschung. Hinsichtlich Fragestellung (3) untersucht diese Arbeit zwei alternative Ansätze zur automatisierten Analyse von Lernaktivitäten im Bereich Argumentation: die wissensbasierte und die datenbasierte Analysemethodik. Der wissensbasierte Ansatz wurde mittels einer Softwarekomponente zur Mustersuche in Argumentationsdiagrammen umgesetzt, welche auf Grundlage deklarativer Musterbeschreibungen arbeitet. Die Möglichkeiten und Eignung des Ansatzes werden anhand von drei Beispielszenarien demonstriert, für die verschiedenartige, pädagogisch relevante Muster innerhalb der entwickelten Softwarekomponente definiert wurden. Der Ansatz erweist sich insbesondere als nützlich für Muster eingeschränkter Komplexität in Szenarien, für die Expertenwissen in ausreichendem Umfang verfügbar ist. Der datenbasierte Ansatz wurde mittels maschineller Lernverfahren umgesetzt. Mit deren Hilfe wurden Klassifikationsroutinen zur Analyse zentraler Aspekte von Onlinediskussionen, wie beispielsweise themenfremde Beiträge, begründete Aussagen und Frage-Antwort-Interaktionen, algorithmisch hergeleitet. Validierungsergebnisse zeigen, dass sich dieser Ansatz selbst für komplexe Klassifikationsprobleme eignet, welche die Berücksichtigung natürlicher Sprache erfordern. Dies ist die erste Arbeit zum Einsatz maschineller Lernverfahren zur Analyse von diagrammbasierten Lerndiskussionen. Die Arbeit schließt mit einer Diskussion des aktuellen Forschungsstands hinsichtlich der vier Fragestellungen im breiteren Kontext existierender Theorien und empirischer Befunde. Die Vor- und Nachteile verschiedener Optionen für die Gestaltung von Lernsystemen für Argumentation werden gegenübergestellt und zukünftige Forschungsfelder vorgeschlagen. Dieser letzte Teil der Arbeit bietet Forschern und Anwendern einen umfassenden Überblick des aktuellen Forschungsstands bezüglich des Designs computerbasierter Argumentationslernsysteme und den zugrunde liegenden lehr- und lerntheoretischen Erkenntnissen. Insbesondere wird auf Fragestellung (4) vertiefend eingegangen und bisherige Adaptionsansätze einschließlich entsprechender empirischer Befunde erörtert

Impact of corpus size and dimensionality of LSA spaces from Wikipedia articles on autotutor answer evaluation

Latent Semantic Analysis (LSA) plays an important role in analyzing text data from education settings. LSA represents meaning of words and sets of words by vectors from a k-dimensional space generated from a selected corpus. While the impact of the value of k has been investigated by many researchers, the impact of the selection of documents and the size of the corpus has never been systematically investigated. This paper tackles this problem based on the performance of LSA in evaluating learners’ answers to AutoTutor, a conversational intelligent tutoring system. We report the impact of document sources (Wikipedia vs TASA), selection algorithms (keyword based vs random), corpus size (from 2000 to 30000 documents) and number of dimensions (from 2 to 1000). Two AutoTutor tasks are used to evaluate the performance of different LSA spaces: a phrase level answer assessment (responses to focal prompt questions) and a sentence level answer assessment (responses to hints). We show that a sufficiently large (e.g., 20,000 to 30,000 documents) randomly selected Wikipedia corpus with high enough dimensions (about 300) could provide a reasonably good space. A specifically selected domain corpus could have significantly better performance with a relatively smaller corpus size (about 8000 documents) and much lower dimensionality (around 17). The widely used TASA corpus (37,651 documents scientifically sampled) performs equally well as a randomly selected large Wikipedia corpus (20,000 to 30,000) with a sufficiently high dimensionality (e.g., k\u3e=300)

Impact of corpus size and dimensionality of LSA spaces from Wikipedia articles on autotutor answer evaluation

Latent Semantic Analysis (LSA) plays an important role in analyzing text data from education settings. LSA represents meaning of words and sets of words by vectors from a k-dimensional space generated from a selected corpus. While the impact of the value of k has been investigated by many researchers, the impact of the selection of documents and the size of the corpus has never been systematically investigated. This paper tackles this problem based on the performance of LSA in evaluating learners’ answers to AutoTutor, a conversational intelligent tutoring system. We report the impact of document sources (Wikipedia vs TASA), selection algorithms (keyword based vs random), corpus size (from 2000 to 30000 documents) and number of dimensions (from 2 to 1000). Two AutoTutor tasks are used to evaluate the performance of different LSA spaces: a phrase level answer assessment (responses to focal prompt questions) and a sentence level answer assessment (responses to hints). We show that a sufficiently large (e.g., 20,000 to 30,000 documents) randomly selected Wikipedia corpus with high enough dimensions (about 300) could provide a reasonably good space. A specifically selected domain corpus could have significantly better performance with a relatively smaller corpus size (about 8000 documents) and much lower dimensionality (around 17). The widely used TASA corpus (37,651 documents scientifically sampled) performs equally well as a randomly selected large Wikipedia corpus (20,000 to 30,000) with a sufficiently high dimensionality (e.g., k\u3e=300)