Boredom and student modeling in intelligent tutoring systems

Over the past couple decades, intelligent tutoring systems (ITSs) have become popular in education. ITSs are effective at helping students learn (VanLehn, 2011; Razzaq, Mendicino & Heffernan, 2008; Koedinger et al, 1997) and help researchers understand how students learn. Such research has included modeling how students learn (Corbett & Anderson, 1995), the effectiveness of help given within an ITS (Beck et al, 2008), the difficulty of different problems (Pardos & Heffernan, 2011), and predicting long-term outcomes like college attendance (San Pedro et al, 2013a), among many other studies. While most studies have focused on ITSs from a cognitive perspective, a growing number of researchers are paying attention to the motivational and affective aspects of tutoring, which have been recognized as important components of human tutoring (Lepper et al, 1993). Recent work has shown that student affect within an ITS can be detected, even without physical sensors or cameras (D’Mello et al, 2008; Conati & Maclaren, 2009; Sabourin et al, 2011; San Pedro et al, 2013b). Initial studies with these sensor-less affect detectors have shown that certain problematic affective states, such as boredom, confusion and frustration, are prevalent within ITSs (Baker et al, 2010b). Boredom in particular has been linked to negative learning outcomes (Pekrun et al, 2010; Farmer & Sundberg, 1986) and long-term disengagement (Farrell, 1988). Therefore, reducing or responding effectively to these affective states within ITSs may improve both short- and long-term learning outcomes. This work is an initial attempt to determine what causes boredom in ITSs. First, we determine which is more responsible for boredom in ITSs: the content in the system, or the students themselves. Based on the findings of that analysis, we conduct a randomized controlled trial to determine the effects of monotony on student boredom. In addition to the work on boredom, we also perform analyses that concern student modeling, specifically how to improve Knowledge Tracing (Corbett & Anderson, 1995), a popular student model used extensively in real systems like the Cognitive Tutors (Koedinger et al, 1997) and in educational research

Predicting Students Performance Based on Their Reading Behaviors

E-learning systems can support students in the on-line classroom environment by providing different learning materials. However, recent studies find that students may misuse such systems with a variety of strategies. One particular misused strategy, gaming the system, has repeatedly been found to negatively affect the students’ learning results. Unfortunately, methods to quantitatively capture such behavior are poorly developed, making it difficult to predict students learning outcomes. In this work, we tackle this problem based on a study of the 567,193 records of the 71 students’ reading behaviors from two classes in the academic year 2016. We first quantify the extent to which students misused the system and then predict their class performance based on the quantified results. Our results demonstrated that such misbehavior in the E-learning system can be quantified as a probability and then further used as a significant factor to predict students class learning outcomes with high accuracy

Understanding and Supporting Vocabulary Learners via Machine Learning on Behavioral and Linguistic Data

This dissertation presents various machine learning applications for predicting different cognitive states of students while they are using a vocabulary tutoring system, DSCoVAR. We conduct four studies, each of which includes a comprehensive analysis of behavioral and linguistic data and provides data-driven evidence for designing personalized features for the system. The first study presents how behavioral and linguistic interactions from the vocabulary tutoring system can be used to predict students' off-task states. The study identifies which predictive features from interaction signals are more important and examines different types of off-task behaviors. The second study investigates how to automatically evaluate students' partial word knowledge from open-ended responses to definition questions. We present a technique that augments modern word-embedding techniques with a classic semantic differential scaling method from cognitive psychology. We then use this interpretable semantic scale method for predicting students' short- and long-term learning. The third and fourth studies show how to develop a model that can generate more efficient training curricula for both human and machine vocabulary learners. The third study illustrates a deep-learning model to score sentences for a contextual vocabulary learning curriculum. We use pre-trained language models, such as ELMo or BERT, and an additional attention layer to capture how the context words are less or more important with respect to the meaning of the target word. The fourth study examines how the contextual informativeness model, originally designed to develop curricula for human vocabulary learning, can also be used for developing curricula for various word embedding models. We identify sentences predicted as low informative for human learners are also less helpful for machine learning algorithms. Having a rich understanding of user behaviors, responses, and learning stimuli is imperative to develop an intelligent online system. Our studies demonstrate interpretable methods with cross-disciplinary approaches to understand various cognitive states of students during learning. The analysis results provide data-driven evidence for designing personalized features that can maximize learning outcomes. Datasets we collected from the studies will be shared publicly to promote future studies related to online tutoring systems. And these findings can also be applied to represent different user states observed in other online systems. In the future, we believe our findings can help to implement a more personalized vocabulary learning system, to develop a system that uses non-English texts or different types of inputs, and to investigate how the machine learning outputs interact with students.PHDInformationUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/162999/1/sjnam_1.pd

Student Modeling in Intelligent Tutoring Systems

After decades of development, Intelligent Tutoring Systems (ITSs) have become a common learning environment for learners of various domains and academic levels. ITSs are computer systems designed to provide instruction and immediate feedback, which is customized to individual students, but without requiring the intervention of human instructors. All ITSs share the same goal: to provide tutorial services that support learning. Since learning is a very complex process, it is not surprising that a range of technologies and methodologies from different fields is employed. Student modeling is a pivotal technique used in ITSs. The model observes student behaviors in the tutor and creates a quantitative representation of student properties of interest necessary to customize instruction, to respond effectively, to engage students¡¯ interest and to promote learning. In this dissertation work, I focus on the following aspects of student modeling. Part I: Student Knowledge: Parameter Interpretation. Student modeling is widely used to obtain scientific insights about how people learn. Student models typically produce semantically meaningful parameter estimates, such as how quickly students learn a skill on average. Therefore, parameter estimates being interpretable and plausible is fundamental. My work includes automatically generating data-suggested Dirichlet priors for the Bayesian Knowledge Tracing model, in order to obtain more plausible parameter estimates. I also proposed, implemented, and evaluated an approach to generate multiple Dirichlet priors to improve parameter plausibility, accommodating the assumption that there are subsets of skills which students learn similarly. Part II: Student Performance: Student Performance Prediction. Accurately predicting student performance is one of the most desired features common evaluations for student modeling. for an ITS. The task, however, is very challenging, particularly in predicting a student¡¯s response on an individual problem in the tutor. I analyzed the components of two common student models to determine which aspects provide predictive power in classifying student performance. I found that modeling the student¡¯s overall knowledge led to improved predictive accuracy. I also presented an approach, which, rather than assuming students are drawn from a single distribution, modeled multiple distributions of student performances to improve the model¡¯s accuracy. Part III: Wheel-spinning: Student Future Failure in Mastery Learning. One drawback of the mastery learning framework is its possibility to leave a student stuck attempting to learn a skill he is unable to master. We refer to this phenomenon of students being given practice with no improvement as wheel-spinning. I analyzed student wheel-spinning across different tutoring systems and estimated the scope of the problem. To investigate the negative consequences of see what wheel-spinning could have done to students, I investigated the relationships between wheel-spinning and two other constructs of interest about students: efficiency of learning and ¡°gaming the system¡±. In addition, I designed a generic model of wheel-spinning, which uses features easily obtained by most ITSs. The model can be well generalized to unknown students with high accuracy classifying mastery and wheel-spinning problems. When used as a detector, the model can detect wheel-spinning in its early stage with satisfying satisfactory precision and recall

Does gamification work for boys and girls?: An exploratory study with a virtual learning environment

SIGAPP ACM Special Interest Group on Applied ComputingThe development and use of Virtual Learning Environments (VLE) has increased considerably over the past decades. Following that trend, many research findings have shown the benefits of using VLE during the learning process. Nevertheless, there are important problems that hinder their use requiring further investigation. Among them, one of the main problems is the inappropriate use of these systems by students. The boredom, lack of interest, monotony, lack of motivation, among other factors, ultimately causes students to behave inappropriately and lead them to a lower performance. In this context, the proposed study investigates whether it is possible to reduce undesirable behaviors and increase performance of students through the use of game mechanics (i.e. gamification). We develop a VLE, E-Game, that can turn on/off several game mechanics, such as points, badges, levels and so on. A case study was conducted with two groups of students to investigate their behavior during their interaction with E-Game with and without gamification. The results indicate that the gamification implemented by E-Game contributed to improve student performance in the case of boys. Yet, improvement was not observed in the case of girls. Furthermore, it was not possible to conclude whether the use of gamification helps to prevent inappropriate student behavior, and therefore, further studies and experiments are needed

Does the Use of Learning Management Systems With Hypermedia Mean Improved Student Learning Outcomes?

Learning management systems (LMSs) that incorporate hypermedia Smart Tutoring Systems and personalized student feedback can increase self-regulated learning (SRL), motivation, and effective learning. These systems are studied with the following aims: (1) to verify whether the use of LMS with hypermedia Smart Tutoring Systems improves student learning outcomes; (2) to verify whether the learning outcomes will be grouped into performance clusters (Satisfactory, Good, and Excellent); and (3) to verify whether those clusters will group together the different learning outcomes assessed in four different evaluation procedures. Use of the LMS with hypermedia Smart Tutoring Systems was studied among students of Health Sciences, all of whom had similar test results in the use of metacognitive skills. It explained 38% of the variance in student learning outcomes in the evaluation procedures. Likewise, three clusters that grouped the learning outcomes in relation to the variable ‘Use of an LMS with hypermedia Smart Tutoring Systems vs. No use’ explained 60.4% of the variance. Each cluster grouped the learning outcomes in the different evaluation procedures. In conclusion, LMS with hypermedia Smart Tutoring Systems in Moodle increased the effectiveness of student learning outcomes, above all in the individual quiz-type tests. It also facilitated personalized learning and respect for the individual pace of student-learning. Hence, modules for the analysis of supervised, unsupervised and multivariate learning should be incorporated into the Moodle platform to provide teaching tools that will undoubtedly contribute to improvements in student learning outcomes.The Research Funding Program 2018 of the Vice-Rectorate for Research and Knowledge Transfer of the University of Burgos