Dynamic Bayeian Inference Networks and Hidden Markov Models for Modeling Learning Progressions over Multiple Time Points

The current study examines the performance of a Bayesian Inference Network (BIN) for modeling Learning Progressions (LP) as a longitudinal design approach. Recently, Learning Progressions, defined by measurable pathways that a student may follow in building their knowledge and gaining expertise over time (National Research Council, 2007; Shin, Stevens, Short & Krajcik, 2009), have captured attention in mathematics and science education (Learning Progressions in Science Conference, 2009). While substantive, psychological, instructional, and task developmental aspects has been proposed in the LP framework, few assessment design frameworks have been designed to link the theory embodied in a progression, tasks that provide evidence about a student's level on that progression, and psychometric models that can link them. Specially, few psychometric models have been proposed to characterize the relationship between student performance and levels on learning progressions in a longitudinal design approach. This dissertation introduces an approach to modeling LPs over multiple time points using Bayesian Inference Networks, referred to as dynamic Bayesian Inference Networks (DBINs). The DBINs are a framework for modeling LPs over time by integrating the theory embodying LPs, assessment design, and interpretation of student performances. The technical aspects of this dissertation cover the fundamental concepts of the graphical model for constructing a DBIN. It is shown that this modeling strategy for change over multiple time points is equivalent to a hidden Markov model. An expectation-maximization (EM) algorithm is presented for estimating the parameters in the model. Two simulation studies are conducted that focus on the construction of a simple DBIN model and an expanded DBIN model with a covariate. The extension that incorporates a covariate for students is useful for studying the effect of instructional treatments, students' background, and motivation on a student's LP. An application illustrates the ideas with real data from the domain of beginning computer network engineering drawn from work in the Cisco Networking Academy

Optimising ITS behaviour with Bayesian networks and decision theory

We propose and demonstrate a methodology for building tractable normative intelligent tutoring systems (ITSs). A normative ITS uses a Bayesian network for long-term student modelling and decision theory to select the next tutorial action. Because normative theories are a general framework for rational behaviour, they can be used to both define and apply learning theories in a rational, and therefore optimal, way. This contrasts to the more traditional approach of using an ad-hoc scheme to implement the learning theory. A key step of the methodology is the induction and the continual adaptation of the Bayesian network student model from student performance data, a step that is distinct from other recent Bayesian net approaches in which the network structure and probabilities are either chosen beforehand by an expert, or by efficiency considerations. The methodology is demonstrated by a description and evaluation of CAPIT, a normative constraint-based tutor for English capitalisation and punctuation. Our evaluation results show that a class using the full normative version of CAPIT learned the domain rules at a faster rate than the class that used a non-normative version of the same system

Dynamic Key-Value Memory Networks for Knowledge Tracing

Knowledge Tracing (KT) is a task of tracing evolving knowledge state of students with respect to one or more concepts as they engage in a sequence of learning activities. One important purpose of KT is to personalize the practice sequence to help students learn knowledge concepts efficiently. However, existing methods such as Bayesian Knowledge Tracing and Deep Knowledge Tracing either model knowledge state for each predefined concept separately or fail to pinpoint exactly which concepts a student is good at or unfamiliar with. To solve these problems, this work introduces a new model called Dynamic Key-Value Memory Networks (DKVMN) that can exploit the relationships between underlying concepts and directly output a student's mastery level of each concept. Unlike standard memory-augmented neural networks that facilitate a single memory matrix or two static memory matrices, our model has one static matrix called key, which stores the knowledge concepts and the other dynamic matrix called value, which stores and updates the mastery levels of corresponding concepts. Experiments show that our model consistently outperforms the state-of-the-art model in a range of KT datasets. Moreover, the DKVMN model can automatically discover underlying concepts of exercises typically performed by human annotations and depict the changing knowledge state of a student.Comment: To appear in 26th International Conference on World Wide Web (WWW), 201

Effects of network topology on the OpenAnswer’s Bayesian model of peer assessment

The paper investigates if and how the topology of the peer assessment network can affect the performance of the Bayesian model adopted in Ope nAnswer. Performance is evaluated in terms of the comparison of predicted grades with actual teacher’s grades. The global network is built by interconnecting smaller subnetworks, one for each student, where intra subnetwork nodes represent student's characteristics, and peer assessment assignments make up inter subnetwork connections and determine evidence propagation. A possible subset of teacher graded answers is dynamically determined by suitable selec tion and stop rules. The research questions addressed are: RQ1) “does the topology (diameter) of the network negatively influence the precision of predicted grades?”̀ in the affirmative case, RQ2) “are we able to reduce the negative effects of high diameter networks through an appropriate choice of the subset of students to be corrected by the teacher?” We show that RQ1) OpenAnswer is less effective on higher diameter topologies, RQ2) this can be avoided if the subset of corrected students is chosen considering the network topology

Hierarchical Models for Relational Event Sequences

Interaction within small groups can often be represented as a sequence of events, where each event involves a sender and a recipient. Recent methods for modeling network data in continuous time model the rate at which individuals interact conditioned on the previous history of events as well as actor covariates. We present a hierarchical extension for modeling multiple such sequences, facilitating inferences about event-level dynamics and their variation across sequences. The hierarchical approach allows one to share information across sequences in a principled manner---we illustrate the efficacy of such sharing through a set of prediction experiments. After discussing methods for adequacy checking and model selection for this class of models, the method is illustrated with an analysis of high school classroom dynamics

Modeling Infection with Multi-agent Dynamics

Developing the ability to comprehensively study infections in small populations enables us to improve epidemic models and better advise individuals about potential risks to their health. We currently have a limited understanding of how infections spread within a small population because it has been difficult to closely track an infection within a complete community. The paper presents data closely tracking the spread of an infection centered on a student dormitory, collected by leveraging the residents' use of cellular phones. The data are based on daily symptom surveys taken over a period of four months and proximity tracking through cellular phones. We demonstrate that using a Bayesian, discrete-time multi-agent model of infection to model real-world symptom reports and proximity tracking records gives us important insights about infec-tions in small populations