FAST: Feature-Aware Student Knowledge Tracing

Various kinds of e-learning systems, such as Massively Open Online Courses and intelligent tutoring systems, are now producing amounts of feature-rich data from students solving items at different levels of proficiency over time. To analyze such data, researchers often use Knowledge Tracing [4], a 20-year old method that has become the de-facto standard for inferring student’s knowledge from performance data. Knowledge Tracing uses Hidden Markov Models (HMM) to estimate the latent cognitive state (student’s knowledge) from the student’s performance answering items. Since the original Knowledge Tracing formulation does not allow to model general features, a considerable amount of research has focused on ad-hoc modifications to the Knowledge Tracing algorithm to enable modeling a specific feature of interest. This has led to a plethora of different Knowledge Tracing reformulations for very specific purposes. For example, Pardos et al. [5] proposed a new model to measure the effect of students’ individual characteristics, Beck et al. [2] modified Knowledge Tracing to assess the effect of help in a tutor system, and Xu and Mostow [7] proposed a new model that allows measuring the effect of subskills. These ad hoc models are successful for their own specific purpose, but they do not generalize to arbitrary features. Other student modeling methods which allow more flexible features have been proposed. For example, Performance Factor Analysis [6] uses logistic regression to model arbitrary features, but unfortunately it does not make inferences of whether the student has learned a skill. We present FAST (Feature-Aware Student knowledge Tracing), a novel method that allows general features into Knowledge Tracing. FAST combines Performance Factor Analysis (logistic regression) with Knowledge Tracing, by leveraging on previous work on unsupervised learning with features [3]. Therefore, FAST is able to infer student’s knowledge, like Knowledge Tracing does, while also allowing for arbitrary features, like Performance Factor Analysis does. FAST allows general features into Knowledge Tracing by replacing the generative emission probabilities (often called guess and slip probabilities) with logistic regression [3], so that these probabilities can change with time to infer student’s knowledge. FAST allows arbitrary features to train the logistic regression model and the HMM jointly. Training the parameters simultaneously enables FAST to learn from the features. This differs from using regression to analyze the slip and guess probabilities [1]. To validate our approach, we use data collected from real students interacting with a tutor. We present experimental results comparing FAST with Knowledge Tracing and Performance Factor Analysis. We conduct experiments with our model using features like item difficulty, prior successes and failures of a student for the skill (or multiple skills) associated with the item, according to the formulation of Performance Factor Analysis

Unsupervised Neural Hidden Markov Models

In this work, we present the first results for neuralizing an Unsupervised Hidden Markov Model. We evaluate our approach on tag in- duction. Our approach outperforms existing generative models and is competitive with the state-of-the-art though with a simpler model easily extended to include additional context.Comment: accepted at EMNLP 2016, Workshop on Structured Prediction for NLP. Oral presentatio

Token and Type Constraints for Cross-Lingual Part-of-Speech Tagging

We consider the construction of part-of-speech taggers for resource-poor languages. Recently, manually constructed tag dictionaries from Wiktionary and dictionaries projected via bitext have been used as type constraints to overcome the scarcity of annotated data in this setting. In this paper, we show that additional token constraints can be projected from a resource-rich source language to a resource-poor target language via word-aligned bitext. We present several models to this end; in particular a partially observed conditional random field model, where coupled token and type constraints provide a partial signal for training. Averaged across eight previously studied Indo-European languages, our model achieves a 25% relative error reduction over the prior state of the art. We further present successful results on seven additional languages from different families, empirically demonstrating the applicability of coupled token and type constraints across a diverse set of languages

General Features in Knowledge Tracing to Model Multiple Subskills, Temporal Item Response Theory, and Expert Knowledge

Knowledge Tracing is the de-facto standard for inferring student knowledge from performance data. Unfortunately, it does not allow modeling the feature-rich data that is now possible to collect in modern digital learning environments. Because of this, many ad hoc Knowledge Tracing variants have been proposed to model a specific feature of interest. For example, variants have studied the effect of students’ individual characteristics, the effect of help in a tutor, and subskills. These ad hoc models are successful for their own specific purpose, but are specified to only model a single specific feature. We present FAST (Feature Aware Student knowledge Tracing), an efficient, novel method that allows integrating general features into Knowledge Tracing. We demonstrate FAST’s flexibility with three examples of feature sets that are relevant to a wide audience. We use features in FAST to model (i) multiple subskill tracing, (ii) a temporal Item Response Model implementation, and (iii) expert knowledge. We present empirical results using data collected from an Intelligent Tutoring System. We report that using features can improve up to 25% in classification performance of the task of predicting student performance. Moreover, for fitting and inferencing, FAST can be 300 times faster than models created in BNT-SM, a toolkit that facilitates the creation of ad hoc Knowledge Tracing variants

Unsupervised Chunking Based on Graph Propagation from Bilingual Corpus

This paper presents a novel approach for unsupervised shallow parsing model trained on the unannotated Chinese text of parallel Chinese-English corpus. In this approach, no information of the Chinese side is applied. The exploitation of graph-based label propagation for bilingual knowledge transfer, along with an application of using the projected labels as features in unsupervised model, contributes to a better performance. The experimental comparisons with the state-of-the-art algorithms show that the proposed approach is able to achieve impressive higher accuracy in terms of F-score

Unsupervised POS induction with word embeddings

Abstract Unsupervised word embeddings have been shown to be valuable as features in supervised learning problems; however, their role in unsupervised problems has been less thoroughly explored. In this paper, we show that embeddings can likewise add value to the problem of unsupervised POS induction. In two representative models of POS induction, we replace multinomial distributions over the vocabulary with multivariate Gaussian distributions over word embeddings and observe consistent improvements in eight languages. We also analyze the effect of various choices while inducing word embeddings on "downstream" POS induction results