Evaluating and improving adaptive educational systems with learning curves

Personalised environments such as adaptive educational systems can be evaluated and compared using performance curves. Such summative studies are useful for determining whether or not new modifications enhance or degrade performance. Performance curves also have the potential to be utilised in formative studies that can shape adaptive model design at a much finer level of granularity. We describe the use of learning curves for evaluating personalised educational systems and outline some of the potential pitfalls and how they may be overcome. We then describe three studies in which we demonstrate how learning curves can be used to drive changes in the user model. First, we show how using learning curves for subsets of the domain model can yield insight into the appropriateness of the model’s structure. In the second study we use this method to experiment with model granularity. Finally, we use learning curves to analyse a large volume of user data to explore the feasibility of using them as a reliable method for fine-tuning a system’s model. The results of these experiments demonstrate the successful use of performance curves in formative studies of adaptive educational systems

Logistic Knowledge Tracing: A Constrained Framework for Learner Modeling

Adaptive learning technology solutions often use a learner model to trace learning and make pedagogical decisions. The present research introduces a formalized methodology for specifying learner models, Logistic Knowledge Tracing (LKT), that consolidates many extant learner modeling methods. The strength of LKT is the specification of a symbolic notation system for alternative logistic regression models that is powerful enough to specify many extant models in the literature and many new models. To demonstrate the generality of LKT, we fit 12 models, some variants of well-known models and some newly devised, to 6 learning technology datasets. The results indicated that no single learner model was best in all cases, further justifying a broad approach that considers multiple learner model features and the learning context. The models presented here avoid student-level fixed parameters to increase generalizability. We also introduce features to stand in for these intercepts. We argue that to be maximally applicable, a learner model needs to adapt to student differences, rather than needing to be pre-parameterized with the level of each student's ability

An Online Tutor for Astronomy: The GEAS Self-Review Library

We introduce an interactive online resource for use by students and college instructors in introductory astronomy courses. The General Education Astronomy Source (GEAS) online tutor guides students developing mastery of core astronomical concepts and mathematical applications of general astronomy material. It contains over 12,000 questions, with linked hints and solutions. Students who master the material quickly can advance through the topics, while under-prepared or hesitant students can focus on questions on a certain topic for as long as needed, with minimal repetition. Students receive individual accounts for study and course instructors are provided with overview tracking information, by time and by topic, for entire cohorts of students. Diagnostic tools support self-evaluation and close collaboration between instructor and student, even for distance learners. An initial usage study shows clear trends in performance which increase with study time, and indicates that distance learners using these materials perform as well as or better than a comparison cohort of on-campus astronomy students. We are actively seeking new collaborators to use this resource in astronomy courses and other educational venues.Comment: 15 pages, 9 figures; Vogt, N. P., and A. S. Muise. 2015. An online tutor for general astronomy: The GEAS self-review library. Cogent Education, 2 (1

A quantitative evaluation of the AVITEWRITE model of handwriting learning

Much sensory-motor behavior develops through imitation, as during the learning of handwriting by children. Such complex sequential acts are broken down into distinct motor control synergies, or muscle groups, whose activities overlap in time to generate continuous, curved movements that obey an intense relation between curvature and speed. The Adaptive Vector Integration to Endpoint (AVITEWRITE) model of Grossberg and Paine (2000) proposed how such complex movements may be learned through attentive imitation. The model suggest how frontal, parietal, and motor cortical mechanisms, such as difference vector encoding, under volitional control from the basal ganglia, interact with adaptively-timed, predictive cerebellar learning during movement imitation and predictive performance. Key psycophysical and neural data about learning to make curved movements were simulated, including a decrease in writing time as learning progresses; generation of unimodal, bell-shaped velocity profiles for each movement synergy; size scaling with isochrony, and speed scaling with preservation of the letter shape and the shapes of the velocity profiles; an inverse relation between curvature and tangential velocity; and a Two-Thirds Power Law relation between angular velocity and curvature. However, the model learned from letter trajectories of only one subject, and only qualitative kinematic comparisons were made with previously published human data. The present work describes a quantitative test of AVITEWRITE through direct comparison of a corpus of human handwriting data with the model's performance when it learns by tracing human trajectories. The results show that model performance was variable across subjects, with an average correlation between the model and human data of 89+/-10%. The present data from simulations using the AVITEWRITE model highlight some of its strengths while focusing attention on areas, such as novel shape learning in children, where all models of handwriting and learning of other complex sensory-motor skills would benefit from further research.Defense Advanced Research Projects Agency and the Office of Naval Research (N00014-95-1-0409); National Institutes of Health (1-R29-DC02952-01); Office of Naval Research (N00014-92-J-1309, N00014-01-1-0624); Air Force Office of Scientific Research (F49620-01-1-0397); National Institute of Neurological Disorders and Stroke (NS 33173

Machine Learning Approach for an Advanced Agent-based Intelligent Tutoring System

Machine Learning Approach for an Advanced Agent-based Intelligent Tutoring System Roya Aminikia Learning Management Systems (LMSs) are digital frameworks that provide curriculum, training materials, and corresponding assessments to guarantee an effective learning process. Although these systems are capable of distributing the learning content, they do not support dynamic learning processes and do not have the capability to communicate with human learners who are required to interact in a dynamic environment during the learning process. To create this process and support the interaction feature, LMSs are equipped with Intelligent Tutoring Systems (ITSs). The main objective of an ITS is to facilitate students’ movement towards their learning goals through virtual tutoring. When equipped with ITSs, LMSs operate as dynamic systems to provide students with access to a tutor who is available anytime during the learning session. The crucial issues we address in this thesis are how to set up a dynamic LMS, and how to design the logical structure behind an ITS. Artificial intelligence, multi-agent technology and machine learning provide powerful theories and foundations that we leverage to tackle these issues. We designed and implemented the new concept of Pedagogical Agent (PA) as the main part of our ITS. This agent uses an evaluation procedure to compare each particular student, in terms of performance, with their peers to develop a worthwhile guidance. The agent captures global knowledge of students’ feature measurements during students’ guiding process. Therefore, the PA retains an updated status, called image, of each specific student at any moment. The agent uses this image for the purpose of diagnosing students’ skills to implement a specific correct instruction. To develop the infrastructure of the agent decision making algorithm, we laid out a protocol (decision tree) to select the best individual direction. The significant capability of the agent is the ability to update its functionality by looking at a student’s image at run time. We also applied two supervised machine learning methods to improve the decision making protocol performance in order to maximize the effect of the collaborating mechanism between students and the ITS. Through these methods, we made the necessary modifications to the decision making structure to promote students’ performance by offering prompts during the learning sessions. The conducted experiments showed that the proposed system is able to efficiently classify students into learners with high versus low performance. Deployment of such a model enabled the PA to use different decision trees while interacting with students of different learning skills. The performance of the system has been shown by ROC curves and details regarding combination of different attributes used in the two machine learning algorithms are discussed, along with the correlation of key attributes that contribute to the accuracy and performance of the decision maker components

A Multi-Gene Genetic Programming Application for Predicting Students Failure at School

Several efforts to predict student failure rate (SFR) at school accurately still remains a core problem area faced by many in the educational sector. The procedure for forecasting SFR are rigid and most often times require data scaling or conversion into binary form such as is the case of the logistic model which may lead to lose of information and effect size attenuation. Also, the high number of factors, incomplete and unbalanced dataset, and black boxing issues as in Artificial Neural Networks and Fuzzy logic systems exposes the need for more efficient tools. Currently the application of Genetic Programming (GP) holds great promises and has produced tremendous positive results in different sectors. In this regard, this study developed GPSFARPS, a software application to provide a robust solution to the prediction of SFR using an evolutionary algorithm known as multi-gene genetic programming. The approach is validated by feeding a testing data set to the evolved GP models. Result obtained from GPSFARPS simulations show its unique ability to evolve a suitable failure rate expression with a fast convergence at 30 generations from a maximum specified generation of 500. The multi-gene system was also able to minimize the evolved model expression and accurately predict student failure rate using a subset of the original expressionComment: 14 pages, 9 figures, Journal paper. arXiv admin note: text overlap with arXiv:1403.0623 by other author