Integrating knowledge tracing and item response theory: A tale of two frameworks

Traditionally, the assessment and learning science commu-nities rely on different paradigms to model student performance. The assessment community uses Item Response Theory which allows modeling different student abilities and problem difficulties, while the learning science community uses Knowledge Tracing, which captures skill acquisition. These two paradigms are complementary - IRT cannot be used to model student learning, while Knowledge Tracing assumes all students and problems are the same. Recently, two highly related models based on a principled synthesis of IRT and Knowledge Tracing were introduced. However, these two models were evaluated on different data sets, using different evaluation metrics and with different ways of splitting the data into training and testing sets. In this paper we reconcile the models' results by presenting a unified view of the two models, and by evaluating the models under a common evaluation metric. We find that both models are equivalent and only differ in their training procedure. Our results show that the combined IRT and Knowledge Tracing models offer the best of assessment and learning sciences - high prediction accuracy like the IRT model, and the ability to model student learning like Knowledge Tracing

Integrated Stochastic and Literate Based Driven Approaches in Learning Style Identification for Personalized E-Learning Purpose

This paper presents integrated stochastic and literate based driven approaches in learning style identification for personalized e-learning purpose. Shifting a paradigm in education from teacher learning to student learning center has encouraged that learning should follow and tailor learners’ characteristics in the form of personalized e-learning. There are several aspects to describe a condition of learners such as prior knowledge, learning goals, learning styles, cognitive ability, learning interest, and motivation. Even though, in many studies of the personalized e-learning, the learning style plays a significant role. In terms of e-learning, implementing several methods for identifying learner style becomes more challenging. Artificial intelligence and machine learning method give good accuracy, but they still have some issues in computation. Additionally, the stationary method is very hard to represent non-deterministic and dynamic data. Therefore, this research proposes the learning style identification by integrating stochastic and literate based driven approaches. Hidden Markov Model (HMM) and the Naïve Bayes as the Stochastic Approach have been implemented. Subsequently, learner behavior as the literate based data is used to get hints during accessing the learning objects. The proposed model has been implemented to VARK learning style. The accuracy is calculated by comparing the model results with the questionnaire results. When Using the HMM, the proposed model gives accuracy in the range of 95% up to 96.67%. Additionally, when using the Naïve Bayes; the accuracy is 93.33%. The results give better accuracy compared to previous studies. In conclusion, the proposed model is promising for modeling learner style in personalized e-learning

A Causal-Comparative Study on the Efficacy of Intelligent Tutoring Systems on Middle-Grade Math Achievement

This study is a quantitative examination of intelligent tutoring systems in two similar suburban middle schools (grades 6-8) in the Southeastern United States. More specifically, it is a causal-comparative study purposed with examining the efficacy of intelligent tutoring systems as they relate to math achievement for students at two similar middle schools in the Midlands of South Carolina. The independent variable, use of an intelligent tutoring system in math instruction, is defined as the supplementary use of two intelligent tutoring systems, Pearson’s Math Digits and IXL, for math instruction. The dependent variable is math achievement as determined by the Measures of Academic Progress (MAP) SC 6+Math test. The student data examined is archived MAP SC 6+ Math scores from the 2017-2018 school year. A one-way ANCOVA was used to compare the mean achievement gain scores of both groups, students whose math instruction included intelligent tutoring systems and students whose math instruction did not include intelligent tutoring systems, to establish whether or not there was any statistically significant difference between the adjusted population means of the two independent groups. The results showed that the adjusted mean of posttest scores of students who did not receive math instruction that involved an intelligent tutoring system were significantly higher than those who did

Adaptive hypermedia for education and training

Adaptive hypermedia (AH) is an alternative to the traditional, one-size-fits-all approach in the development of hypermedia systems. AH systems build a model of the goals, preferences, and knowledge of each individual user; this model is used throughout the interaction with the user to adapt to the needs of that particular user (Brusilovsky, 1996b). For example, a student in an adaptive educational hypermedia system will be given a presentation that is adapted specifically to his or her knowledge of the subject (De Bra & Calvi, 1998; Hothi, Hall, & Sly, 2000) as well as a suggested set of the most relevant links to proceed further (Brusilovsky, Eklund, & Schwarz, 1998; Kavcic, 2004). An adaptive electronic encyclopedia will personalize the content of an article to augment the user's existing knowledge and interests (Bontcheva & Wilks, 2005; Milosavljevic, 1997). A museum guide will adapt the presentation about every visited object to the user's individual path through the museum (Oberlander et al., 1998; Stock et al., 2007). Adaptive hypermedia belongs to the class of user-adaptive systems (Schneider-Hufschmidt, Kühme, & Malinowski, 1993). A distinctive feature of an adaptive system is an explicit user model that represents user knowledge, goals, and interests, as well as other features that enable the system to adapt to different users with their own specific set of goals. An adaptive system collects data for the user model from various sources that can include implicitly observing user interaction and explicitly requesting direct input from the user. The user model is applied to provide an adaptation effect, that is, tailor interaction to different users in the same context. In different kinds of adaptive systems, adaptation effects could vary greatly. In AH systems, it is limited to three major adaptation technologies: adaptive content selection, adaptive navigation support, and adaptive presentation. The first of these three technologies comes from the fields of adaptive information retrieval (IR) and intelligent tutoring systems (ITS). When the user searches for information, the system adaptively selects and prioritizes the most relevant items (Brajnik, Guida, & Tasso, 1987; Brusilovsky, 1992b)

THE ROLE OF SIMULATION IN SUPPORTING LONGER-TERM LEARNING AND MENTORING WITH TECHNOLOGY

Mentoring is an important part of professional development and longer-term learning. The nature of longer-term mentoring contexts means that designing, developing, and testing adaptive learning sys-tems for use in this kind of context would be very costly as it would require substantial amounts of fi-nancial, human, and time resources. Simulation is a cheaper and quicker approach for evaluating the impact of various design and development decisions. Within the Artificial Intelligence in Education (AIED) research community, however, surprisingly little attention has been paid to how to design, de-velop, and use simulations in longer-term learning contexts. The central challenge is that adaptive learning system designers and educational practitioners have limited guidance on what steps to consider when designing simulations for supporting longer-term mentoring system design and development deci-sions. My research work takes as a starting point VanLehn et al.’s [1] introduction to applications of simulated students and Erickson et al.’s [2] suggested approach to creating simulated learning envi-ronments. My dissertation presents four research directions using a real-world longer-term mentoring context, a doctoral program, for illustrative purposes. The first direction outlines a framework for guid-ing system designers as to what factors to consider when building pedagogical simulations, fundamen-tally to answer the question: how can a system designer capture a representation of a target learning context in a pedagogical simulation model? To illustrate the feasibility of this framework, this disserta-tion describes how to build, the SimDoc model, a pedagogical model of a longer-term mentoring learn-ing environment – a doctoral program. The second direction builds on the first, and considers the issue of model fidelity, essentially to answer the question: how can a system designer determine a simulation model’s fidelity to the desired granularity level? This dissertation shows how data from a target learning environment, the research literature, and common sense are combined to achieve SimDoc’s medium fidelity model. The third research direction explores calibration and validation issues to answer the question: how many simulation runs does it take for a practitioner to have confidence in the simulation model’s output? This dissertation describes the steps taken to calibrate and validate the SimDoc model, so its output statistically matches data from the target doctoral program, the one at the university of Saskatchewan. The fourth direction is to demonstrate the applicability of the resulting pedagogical model. This dissertation presents two experiments using SimDoc to illustrate how to explore pedagogi-cal questions concerning personalization strategies and to determine the effectiveness of different men-toring strategies in a target learning context. Overall, this dissertation shows that simulation is an important tool in the AIED system design-ers’ toolkit as AIED moves towards designing, building, and evaluating AIED systems meant to support learners in longer-term learning and mentoring contexts. Simulation allows a system designer to exper-iment with various design and implementation decisions in a cost-effective and timely manner before committing to these decisions in the real world

Multi-Armed Bandits for Intelligent Tutoring Systems

We present an approach to Intelligent Tutoring Systems which adaptively personalizes sequences of learning activities to maximize skills acquired by students, taking into account the limited time and motivational resources. At a given point in time, the system proposes to the students the activity which makes them progress faster. We introduce two algorithms that rely on the empirical estimation of the learning progress, RiARiT that uses information about the difficulty of each exercise and ZPDES that uses much less knowledge about the problem. The system is based on the combination of three approaches. First, it leverages recent models of intrinsically motivated learning by transposing them to active teaching, relying on empirical estimation of learning progress provided by specific activities to particular students. Second, it uses state-of-the-art Multi-Arm Bandit (MAB) techniques to efficiently manage the exploration/exploitation challenge of this optimization process. Third, it leverages expert knowledge to constrain and bootstrap initial exploration of the MAB, while requiring only coarse guidance information of the expert and allowing the system to deal with didactic gaps in its knowledge. The system is evaluated in a scenario where 7-8 year old schoolchildren learn how to decompose numbers while manipulating money. Systematic experiments are presented with simulated students, followed by results of a user study across a population of 400 school children