Parametric Constraints for Bayesian Knowledge Tracing from First Principles

Bayesian Knowledge Tracing (BKT) is a probabilistic model of a learner's state of mastery corresponding to a knowledge component. It considers the learner's state of mastery as a "hidden" or latent binary variable and updates this state based on the observed correctness of the learner's response using parameters that represent transition probabilities between states. BKT is often represented as a Hidden Markov Model and the Expectation-Maximization (EM) algorithm is used to infer these parameters. However, this algorithm can suffer from several issues including producing multiple viable sets of parameters, settling into a local minima, producing degenerate parameter values, and a high computational cost during fitting. This paper takes a "from first principles" approach to deriving constraints that can be imposed on the BKT parameter space. Starting from the basic mathematical truths of probability and building up to the behaviors expected of the BKT parameters in real systems, this paper presents a mathematical derivation that results in succinct constraints that can be imposed on the BKT parameter space. Since these constraints are necessary conditions, they can be applied prior to fitting in order to reduce computational cost and the likelihood of issues that can emerge from the EM procedure. In order to see that promise through, the paper further introduces a novel algorithm for estimating BKT parameters subject to the newly defined constraints. While the issue of degenerate parameter values has been reported previously, this paper is the first, to our best knowledge, to derive the constrains from first principles while also presenting an algorithm that respects those constraints

Keystroke-level analysis to estimate time to process pages in online learning environments

It is challenging for students to plan their work sessions in online environments, as it is very difficult to make estimates on how much material there is to cover. In order to simplify this estimation, we have extended the Keystroke-level analysis model with individual reading speed of text, figures, and questions. This was used to estimate how long students might take to work through pages in an online learning environment. The estimates from the model were compared to data collected from 902 volunteer students. Despite the huge differences in reported reading speeds between students, the presented model performs reasonably well and could be used to give learners feedback on how long it takes to work through pages in online learning environments. This feedback could be used to support students’ motivation and effort regulation as they work through online course components. Although the model performs reasonably well, we propose giving feedback in the form of intervals to indicate the uncertainty of the estimates.QC 20170627</p

Estimating the minimum number of opportunities needed for all students to achieve predicted mastery

We have conducted a study on how many opportunities are necessary, on average, for learners to achieve mastery of a skill, also called a knowledge component (KC), as defined in the Open Learning Initiative (OLI) digital courseware. The study used datasets from 74 different course instances in four topic areas comprising 3813 students and 1.2 million transactions. The analysis supports our claim that the number of opportunities to reach mastery gives us new information on both students and the development of course components. Among the conclusions are a minimum of seven opportunities are necessary for each knowledge component, more if the prior knowledge among students are uneven within a course. The number of KCs in a course increases the number of opportunities needed. The number of opportunities to reach mastery can be used to identify KCs that are outliers that may be in need of better explanations or further instruction.QC 20180917</p

Developing Rubrics to Grade Student Performance

What is a rubric