FAST: Feature-Aware Student Knowledge Tracing

Various kinds of e-learning systems, such as Massively Open Online Courses and intelligent tutoring systems, are now producing amounts of feature-rich data from students solving items at different levels of proficiency over time. To analyze such data, researchers often use Knowledge Tracing [4], a 20-year old method that has become the de-facto standard for inferring student’s knowledge from performance data. Knowledge Tracing uses Hidden Markov Models (HMM) to estimate the latent cognitive state (student’s knowledge) from the student’s performance answering items. Since the original Knowledge Tracing formulation does not allow to model general features, a considerable amount of research has focused on ad-hoc modifications to the Knowledge Tracing algorithm to enable modeling a specific feature of interest. This has led to a plethora of different Knowledge Tracing reformulations for very specific purposes. For example, Pardos et al. [5] proposed a new model to measure the effect of students’ individual characteristics, Beck et al. [2] modified Knowledge Tracing to assess the effect of help in a tutor system, and Xu and Mostow [7] proposed a new model that allows measuring the effect of subskills. These ad hoc models are successful for their own specific purpose, but they do not generalize to arbitrary features. Other student modeling methods which allow more flexible features have been proposed. For example, Performance Factor Analysis [6] uses logistic regression to model arbitrary features, but unfortunately it does not make inferences of whether the student has learned a skill. We present FAST (Feature-Aware Student knowledge Tracing), a novel method that allows general features into Knowledge Tracing. FAST combines Performance Factor Analysis (logistic regression) with Knowledge Tracing, by leveraging on previous work on unsupervised learning with features [3]. Therefore, FAST is able to infer student’s knowledge, like Knowledge Tracing does, while also allowing for arbitrary features, like Performance Factor Analysis does. FAST allows general features into Knowledge Tracing by replacing the generative emission probabilities (often called guess and slip probabilities) with logistic regression [3], so that these probabilities can change with time to infer student’s knowledge. FAST allows arbitrary features to train the logistic regression model and the HMM jointly. Training the parameters simultaneously enables FAST to learn from the features. This differs from using regression to analyze the slip and guess probabilities [1]. To validate our approach, we use data collected from real students interacting with a tutor. We present experimental results comparing FAST with Knowledge Tracing and Performance Factor Analysis. We conduct experiments with our model using features like item difficulty, prior successes and failures of a student for the skill (or multiple skills) associated with the item, according to the formulation of Performance Factor Analysis

Knowledge Tracing: A Review of Available Technologies

As a student modeling technique, knowledge tracing is widely used by various intelligent tutoring systems to infer and trace the individual’s knowledge state during the learning process. In recent years, various models were proposed to get accurate and easy-to-interpret results. To make sense of the wide Knowledge tracing (KT) modeling landscape, this paper conducts a systematic review to provide a detailed and nuanced discussion of relevant KT techniques from the perspective of assumptions, data, and algorithms. The results show that most existing KT models consider only a fragment of the assumptions that relate to the knowledge components within items and student’s cognitive process. Almost all types of KT models take “quize data” as input, although it is insufficient to reflect a clear picture of students’ learning process. Dynamic Bayesian network, logistic regression and deep learning are the main algorithms used by various knowledge tracing models. Some open issues are identified based on the analytics of the reviewed works and discussed potential future research directions

Modelo de inferência dos parâmetros causais : incorporação da probabilidade de conhecimento nas atribuições das causas dos sucessos ou fracassos em atividades cognitivas e sua relação com as ações dos estudantes

Orientador : Prof. Dr. Andrey Ricardo PimentelDissertação (mestrado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa: Curitiba, 23/10/2015Inclui referências : f. 64-68Resumo: O processo cognitivo é uma atividade que envolve vários componentes na aquisição, organização e uso da informação. A aprendizagem é uma tarefa imprecisa uma vez que o processo cognitivo envolve fatores internos dos seres humanos, como o estado afetivo e motivacional. Neste trabalho foi desenvolvido um modelo para a inferência das probabilidades das causas dos sucessos (acertos) ou fracassos (erros) em atividades cognitivas. As atribuições causais foram retiradas da teoria da "Atribuição Causal". Esta teoria apresenta as percepções causais e sua relação com a emoção, motivação e expectativa. As fórmulas foram modeladas utilizando o nível de conhecimento do estudante que é calculado de forma probabilística pelo modelo Bayesian Knowledge Tracing (BKT). Também é utilizada uma extensão do BKT que explora as ações futuras do estudante para determinar o nível e o momento da aprendizagem. Destaca-se que o modelo proposto faz a especialização dos parâmetros do modelo BKT em novos parâmetros visando aprofundar o diagnóstico dos fatores influenciadores do processo cognitivo dos estudantes. Prevendo essas percepções causais pretende-se colaborar com o processo de identificação comportamental do estudante: antes da ocorrência da ação para a previsão de sua resposta, ou após a ação, atuando como um auxiliar na decisão em estratégias de ensino. Nesta dissertação foi simulado o impacto dos valores inferidos para a previsão da resposta do discente através de bases de dados dos históricos de ações de estudantes com um Sistema Tutor Inteligente (STI) em produção.Abstract: The cognitive process is an activity that involves various components in the acquisition, organization and use of information. Learning is an imprecise task since the cognitive process involves internal factors of humans as the emotional and motivational state. In this work, we developed a model for inference of probability causes of successes or failures in cognitive activities. The causal attributions were based on the psychology theory of "attributional theory of motivation and emotion". This theory introduces the causal perceptions and their relationship with the emotion, motivation and expectation. The formulas were modeled using the level of student's knowledge which is calculated in a probabilistic manner by the model Bayesian Knowledge Tracing (BKT). Also we use an extension of the BKT which explores future actions of students to set the level and the moment of learning. We highlight that the proposed model uses the BKT parameters to create new parameters, more specialized, aiming to identify factors influencing the cognitive process of students. Anticipating these causal perceptions, we intend to collaborate with the student's behavioral identification process: before the action for students answer prediction, or after the action, acting as a subsidiary on the decision of education strategy. In this thesis it was simulated the impact of inferred values for the prediction of students action through an historical data base of students actions of a production Intelligent Tutoring System (ITS)

Des méta-modèles pour guider l’élicitation des connaissances en EIAH : contributions à l’enseignement de méthodes et à la personnalisation des activités

Les travaux présentés dans ce mémoire d'habilitation à diriger des recherches portent sur l’élicitation des connaissances dans le cadre de l’ingénierie des EIAH (Environnements Informatiques pour l’Apprentissage Humain). Deux thématiques de recherche ont été explorées : l’enseignement de méthodes de résolution de problèmes et la personnalisation des EIAH. Les contributions à l’élicitation des connaissances dans ces deux thématiques sont des modèles et outils permettant à un utilisateur humain de définir les connaissances nécessaires au système pour proposer à l’apprenant un contenu pédagogique personnalisé, que ce soit un exercice, une rétroaction ou une recommandation.L’approche choisie pour répondre à la problématique de l’élicitation des connaissances est de proposer, pour chacune des questions de recherche abordées, un méta-modèle des connaissances à acquérir, indépendant du domaine d’apprentissage. Ce méta-modèle permet de guider l’utilisateur humain (concepteur, expert, auteur, enseignant) dans la définition d’un modèle de connaissances, qui sera lui dépendant du domaine. Le méta-modèle proposé permet également de définir un moteur de raisonnement associé, capable d’exploiter tout modèle de connaissances conforme au méta-modèle. Ce moteur de raisonnement exploite le modèle de connaissances défini par l’utilisateur, afin d’accomplir les tâches nécessaires à l’accompagnement par l’EIAH d’une activité d’apprentissage.En ce qui concerne l’enseignement de méthodes, les architectures proposées, rassemblant méta-modèles et moteurs de raisonnement, permettent de définir, dans un domaine donné, une méthode de résolution de problèmes et les connaissances destinées à accompagner l’élève dans son apprentissage de la méthode. Dans un domaine donné, une méthode de résolution de problèmes est constituée par un ensemble de classes de problème et d’outils de résolution associés à ces classes. Nous avons proposé le cycle AMBRE, mis en œuvre dans plusieurs EIAH, et qui incite l’apprenant à résoudre des problèmes par analogie afin d’acquérir les classes de problèmes de la méthode.Pour ce qui est de la personnalisation des EIAH, l’objectif de ces recherches est d’adapter à chaque apprenant les activités qui lui sont proposées au sein d’un EIAH. Nous avons proposé des méta-modèles et des outils fondés sur ces méta-modèles, outils destinés à un utilisateur ne possédant pas forcément de compétences poussées en informatique, comme un enseignant ou un auteur de MOOC. Ces outils lui permettent de mettre en place un processus de personnalisation complet, en définissant d’une part comment élaborer des profils d’apprenant à partir des traces de l’activité des apprenants avec l’EIAH, dans le but d’identifier les besoins de chacun, en définissant d’autre part des modèles d’exercices permettant la génération d’activités répondant à des besoins spécifiques, et en précisant enfin selon quelle stratégie affecter des exercices adaptés au profil de chaque apprenant

Apprentissage automatique pour l'assistance au suivi d'étudiants en ligne : approches classique et bio-inspirée

Cette thèse a pris la forme d’un partenariat entre l’équipe VORTEX du laboratoire de recherche en informatique IRIT et l’entreprise Andil, spécialisée dans l'informatique pour l'e-learning. Ce partenariat est conclu autour d’une thèse CIFRE, dispositif soutenu par l’État via l’ANRT. La doctorante, Angela Bovo, a travaillé au sein de l'Université Toulouse 1 Capitole. Un partenariat a également été noué avec l'institut de formation Juriscampus, qui nous a fourni des données issues de formations réelles pour nos expérimentations. Notre objectif principal avec ce projet était d'améliorer les possibilités de suivi des étudiants en cours de formation en ligne pour éviter leur décrochage ou leur échec. Nous avons proposé des possibilités de suivi par apprentissage automatique classique en utilisant comme données les traces d'activité des élèves. Nous avons également proposé, à partir de nos données, des indicateurs de comportement des apprenants. Avec Andil, nous avons conçu et réalisé une application web du nom de GIGA, déjà commercialisée et appréciée par les responsables de formation, qui implémente ces propositions et qui a servi de base à de premières expériences de partitionnement de données qui semblent permettre d'identifier les étudiants en difficulté ou en voie d'abandon. Ce projet a également été lancé avec l'objectif d'étudier les possibilités de l'algorithme d'apprentissage automatique inspiré du cerveau humain Hierarchical Temporal Memory (HTM), dans sa version Cortical Learning Algorithm (CLA), dont les hypothèses fondatrices sont bien adaptées à notre problème. Nous avons proposé des façons d'adapter HTM-CLA à des fonctionnalités d'apprentissage automatique classique (partitionnement, classification, régression, prédiction), afin de comparer ses résultats à ceux fournis par les autres algorithmes plus classiques ; mais aussi de l'utiliser comme base d'un moteur de génération de comportement, qui pourrait être utilisé pour créer un tuteur virtuel intelligent chargé de conseiller les apprenants en temps réel. Les implémentations ne sont toutefois pas encore parvenues à produire des résultats probants.This Ph.D. took the shape of a partnership between the VORTEX team in the computer science research laboratory IRIT and the company Andil, which specializes in software for e-learning. This partnership was concluded around a CIFRE Ph.D. This plan is subsidized by the French state through the ANRT. The Ph.D. student, Angela Bovo, worked in Université Toulouse 1 Capitole. Another partnership was built with the training institute Juriscampus, which gave us access to data from real trainings for our experiments. Our main goal for this project was to improve the possibilities for monitoring students in an e-learning training to keep them from falling behind or giving up. We proposed ways to do such monitoring with classical machine learning methods, with the logs from students' activity as data. We also proposed, using the same data, indicators of students' behaviour. With Andil, we designed and produced a web application called GIGA, already marketed and sold, and well appreciated by training managers, which implements our proposals and served as a basis for first clustering experiments which seem to identify well students who are failing or about to give up. Another goal of this project was to study the capacities of the human brain inspired machine learning algorithm Hierarchical Temporal Memory (HTM), in its Cortical Learning Algorithm (CLA) version, because its base hypotheses are well adapted to our problem. We proposed ways to adapt HTM-CLA to classical machine learning functionalities (clustering, classification, regression, prediction), in order to compare its results to those of more classical algorithms; but also to use it as a basis for a behaviour generation engine, which could be used to create an intelligent tutoring system tasked with advising students in real time. However, our implementations did not get to the point of conclusive results