Carelessness and Affect in an Intelligent Tutoring System for Mathematics

We investigate the relationship between students’ affect and their frequency of careless errors while using an Intelligent Tutoring System for middle school mathematics. A student is said to have committed a careless error when the student’s answer is wrong despite knowing the skill required to provide the correct answer. We operationalize the probability that an error is careless through the use of an automated detector, developed using educational data mining, which infers the probability that an error involves carelessness rather than not knowing the relevant skill. This detector is then applied to log data produced by high-school students in the Philippines using a Cognitive Tutor for scatterplots. We study the relationship between carelessness and affect, triangulating between the detector of carelessness and field observations of affect. Surprisingly, we find that carelessness is common among students who frequently experience engaged concentration. This finding implies that a highly engaged student may paradoxically become overconfident or impulsive, leading to more careless errors. In contrast, students displaying confusion or boredom make fewer careless errors. Further analysis over time suggests that confused and bored students have lower learning overall. Thus, their mistakes appear to stem from a genuine lack of knowledge rather than carelessness

When Does Disengagement Correlate with Performance in Spoken Dialog Computer Tutoring?

In this paper we investigate how student disengagement relates to two performance metrics in a spoken dialog computer tutoring corpus, both when disengagement is measured through manual annotation by a trained human judge, and also when disengagement is measured through automatic annotation by the system based on a machine learning model. First, we investigate whether manually labeled overall disengagement and six different disengagement types are predictive of learning and user satisfaction in the corpus. Our results show that although students’ percentage of overall disengaged turns negatively correlates both with the amount they learn and their user satisfaction, the individual types of disengagement correlate differently: some negatively correlate with learning and user satisfaction, while others don’t correlate with eithermetric at all. Moreover, these relationships change somewhat depending on student prerequisite knowledge level. Furthermore, using multiple disengagement types to predict learning improves predictive power. Overall, these manual label-based results suggest that although adapting to disengagement should improve both student learning and user satisfaction in computer tutoring, maximizing performance requires the system to detect and respond differently based on disengagement type. Next, we present an approach to automatically detecting and responding to user disengagement types based on their differing correlations with correctness. Investigation of ourmachine learningmodel of user disengagement shows that its automatic labels negatively correlate with both performance metrics in the same way as the manual labels. The similarity of the correlations across the manual and automatic labels suggests that the automatic labels are a reasonable substitute for the manual labels. Moreover, the significant negative correlations themselves suggest that redesigning ITSPOKE to automatically detect and respond to disengagement has the potential to remediate disengagement and thereby improve performance, even in the presence of noise introduced by the automatic detection process

Analysis of Student Behavior and Score Prediction in Assistments Online Learning

Understanding and analyzing student behavior is paramount in enhancing online learning, and this thesis delves into the subject by presenting an in-depth analysis of student behavior and score prediction in the ASSISTments online learning platform. We used data from the EDM Cup 2023 Kaggle Competition to answer four key questions. First, we explored how students seeking hints and explanations affect their performance in assignments, shedding light on the role of guidance in learning. Second, we looked at the connection between students mastering specific skills and their performance in related assignments, giving insights into the effectiveness of curriculum alignment. Third, we identified important features from student activity data to improve grade prediction, helping identify at-risk students early and monitor their progress. Lastly, we used graph representation learning to understand complex relationships in the data, leading to more accurate predictive models. This research enhances our understanding of data mining in online learning, with implications for personalized learning and support mechanisms

Clustering student interaction data using Bloom's Taxonomy to find predictive reading patterns

In modern educational technology we have the ability to capture click-stream interaction data from a student as they work on educational problems within an online environment. This provides us with an opportunity to identify student behaviours within the data (captured by the online environment) that are predictive of student success or failure. The constraints that exist within an educational setting provide the ability to associate these student behaviours to specific educational outcomes. This information could be then used to inform environments that support student learning while improving a student’s metacognitive skills. In this dissertation, we describe how reading behaviour clusters were extracted in an experiment in which students were embedded in a learning environment where they read documents and answered questions. We tracked their keystroke level behaviour and then applied clustering techniques to find pedagogically meaningful clusters. The key to finding these clusters were categorizing the questions as to their level in Bloom’s educational taxonomy: different behaviour patterns predicted success and failure in answering questions at various levels of Bloom. The clusters found in the first experiment were confirmed through two further experiments that explored variations in the number, type, and length of documents and the kinds of questions asked. In the final experiment, we also went beyond the actual keystrokes and explored how the pauses between keystrokes as a student answers a question can be utilized in the process of determining student success. This research suggests that it should be possible to diagnose learner behaviour even in “ill-defined” domains like reading. It also suggests that Bloom’s taxonomy can be an important (even necessary) input to such diagnosis

The Role of Artificial Intelligence in Educating Novice Programmers

Programming is an inherently difficult skill to acquire and develop. Those who attempt to learn programming may be easily discouraged. The current landscape for computer science education does not address the needs of every novice programmer. Literature reports a discrepancy between student misconceptions and instructors’ perceptions of those misconceptions. Those who can afford a one-on-one human tutor perform on average two standard deviations better than those who learn via conventional methods, suggesting there is a need for a comparable, cheaper replacement. As a result, a number of intelligent tutoring systems have been developed for the purpose of teaching introductory programming concepts and replicating the benefits of one-on-one human tutoring. In this thesis, we analyze and discuss the literature pertaining to student misconceptions, selecting five fundamental misconception categories for introductory programming to demonstrate the effectiveness of existing intelligent tutoring systems. The features of existing intelligent tutoring systems are discussed and analyzed with respect to their effectiveness in addressing student misconceptions. Finally, we highlight the current gap in research on intelligent tutoring systems, hypothesizing the architecture and features of an ideal intelligent tutoring system for introductory programming.Electrical and Computer Engineerin