Carelessness and Affect in an Intelligent Tutoring System for Mathematics

We investigate the relationship between students’ affect and their frequency of careless errors while using an Intelligent Tutoring System for middle school mathematics. A student is said to have committed a careless error when the student’s answer is wrong despite knowing the skill required to provide the correct answer. We operationalize the probability that an error is careless through the use of an automated detector, developed using educational data mining, which infers the probability that an error involves carelessness rather than not knowing the relevant skill. This detector is then applied to log data produced by high-school students in the Philippines using a Cognitive Tutor for scatterplots. We study the relationship between carelessness and affect, triangulating between the detector of carelessness and field observations of affect. Surprisingly, we find that carelessness is common among students who frequently experience engaged concentration. This finding implies that a highly engaged student may paradoxically become overconfident or impulsive, leading to more careless errors. In contrast, students displaying confusion or boredom make fewer careless errors. Further analysis over time suggests that confused and bored students have lower learning overall. Thus, their mistakes appear to stem from a genuine lack of knowledge rather than carelessness

Carelessness and Affect in an Intelligent Tutoring System for Mathematics

We investigate the relationship between students' affect and their frequency of careless errors while using an Intelligent Tutoring System for middle school mathematics. A student is said to have committed a careless error when the student's answer is wrong despite knowing the skill required to provide the correct answer. We operationalize the probability that an error is careless through the use of an automated detector, developed using educational data mining, which infers the probability that an error involves carelessness rather than not knowing the relevant skill. This detector is then applied to log data produced by high-school students in the Philippines using a Cognitive Tutor for scatterplots. We study the relationship between carelessness and affect, triangulating between the detector of carelessness and field observations of affect. Surprisingly, we find that carelessness is common among students who frequently experience engaged concentration. This finding implies that a highly engaged student may paradoxically become overconfident or impulsive, leading to more careless errors. In contrast, students displaying confusion or boredom make fewer careless errors. Further analysis over time suggests that confused and bored students have lower learning overall. Thus, their mistakes appear to stem from a genuine lack of knowledge rather than carelessness

Detecting students who are conducting inquiry Without Thinking Fastidiously (WTF) in the Context of Microworld Learning Environments

In recent years, there has been increased interest and research on identifying the various ways that students can deviate from expected or desired patterns while using educational software. This includes research on gaming the system, player transformation, haphazard inquiry, and failure to use key features of the learning system. Detection of these sorts of behaviors has helped researchers to better understand these behaviors, thus allowing software designers to develop interventions that can remediate them and/or reduce their negative impacts on student learning. This work addresses two types of student disengagement: carelessness and a behavior we term WTF (“Without Thinking Fastidiously”) behavior. Carelessness is defined as not demonstrating a skill despite knowing it; we measured carelessness using a machine learned model. In WTF behavior, the student is interacting with the software, but their actions appear to have no relationship to the intended learning task. We discuss the detector development process, validate the detectors with human labels of the behavior, and discuss implications for understanding how and why students conduct inquiry without thinking fastidiously while learning in science inquiry microworlds. Following this work we explore the relationship between student learner characteristics and the aforementioned disengaged behaviors carelessness and WTF. Our goal was to develop a deeper understanding of which learner characteristics correlate to carelessness or WTF behavior. Our work examines three alternative methods for predicting carelessness and WTF behaviors from learner characteristics: simple correlations, k-means clustering, and decision tree rule learners

Predicting Math Success in an Online Tutoring System Using Language Data and Click-Stream Variables: A Longitudinal Analysis

Previous studies have demonstrated strong links between students\u27 linguistic knowledge, their affective language patterns and their success in math. Other studies have shown that demographic and click-stream variables in online learning environments are important predictors of math success. This study builds on this research in two ways. First, it combines linguistics and click-stream variables along with demographic information to increase prediction rates for math success. Second, it examines how random variance, as found in repeated participant data, can explain math success beyond linguistic, demographic, and click-stream variables. The findings indicate that linguistic, demographic, and click-stream factors explained about 14% of the variance in math scores. These variables mixed with random factors explained about 44% of the variance

Characterizing Productive Perseverance Using Sensor-Free Detectors of Student Knowledge, Behavior, and Affect

Failure is a necessary step in the process of learning. For this reason, there has been a myriad of research dedicated to the study of student perseverance in the presence of failure, leading to several commonly-cited theories and frameworks to characterize productive and unproductive representations of the construct of persistence. While researchers are in agreement that it is important for students to persist when struggling to learn new material, there can be both positive and negative aspects of persistence. What is it, then, that separates productive from unproductive persistence? The purpose of this work is to address this question through the development, extension, and study of data-driven models of student affect, behavior, and knowledge. The increased adoption of computer-based learning platforms in real classrooms has led to unique opportunities to study student learning at both fine levels of granularity and longitudinally at scale. Prior work has leveraged machine learning methods, existing learning theory, and previous education research to explore various aspects of student learning. These include the development of sensor-free detectors that utilize only the student interaction data collected through such learning platforms. Building off of the considerable amount of prior research, this work employs state-of-the-art machine learning methods in conjunction with the large scale granular data collected by computer-based learning platforms in alignment with three goals. First, this work focuses on the development of student models that study learning through the use of advancements in student modeling and deep learning methodologies. Second, this dissertation explores the development of tools that incorporate such models to support teachers in taking action in real classrooms to promote productive approaches to learning. Finally, this work aims to complete the loop in utilizing these detector models to better understand the underlying constructs that are being measured through their application and their connection to productive perseverance and commonly-observed learning outcomes

Comprendiendo el potencial y los desafíos del Big Data en las escuelas y la educación

In recent years, the world has experienced a huge revolution centered around the gathering and application of big data in various fields. This has affected many aspects of our daily life, including government, manufacturing, commerce, health, communication, entertainment, and many more. So far, education has benefited only a little from the big data revolution. In this article, we review the potential of big data in the context of education systems. Such data may include log files drawn from online learning environments, messages on online discussion forums, answers to open-ended questions, grades on various tasks, demographic and administrative information, speech, handwritten notes, illustrations, gestures and movements, neurophysiologic signals, eye movements, and many more. Analyzing this data, it is possible to calculate a wide range of measurements of the learning process and to support various educational stakeholders with informed decision-making. We offer a framework for better understanding of how big data can be used in education. The framework comprises several elements that need to be addressed in this context: defining the data; formulating data-collecting and storage apparatuses; data analysis and the application of analysis products. We further review some key opportunities and some important challenges of using big data in educationEn los últimos años, el mundo ha experimentado una gran revolución centrada en la recopilación y aplicación de big data en varios campos. Esto ha afectado muchos aspectos de nuestra vida diaria, incluidos el gobierno, la manufactura, el comercio, la salud, la comunicación, el entretenimiento y muchos más. Hasta ahora, la educación se ha beneficiado muy poco de la revolución del big data. En este artículo revisamos el potencial de los macrodatos en el contexto de los sistemas educativos. Dichos datos pueden incluir archivos de registro extraídos de entornos de aprendizaje en línea, mensajes en foros de discusión en línea, respuestas a preguntas abiertas, calificaciones en diversas tareas, información demográfica y administrativa, discurso, notas escritas a mano, ilustraciones, gestos y movimientos, señales neurofisiológicas, movimientos oculares y muchos más. Analizando estos datos es posible calcular una amplia gama de mediciones del proceso de aprendizaje y apoyar a diversos interesados educativos con una toma de decisiones informada. Ofrecemos un marco para una mejor comprensión de cómo se puede utilizar el big data en la educación. El marco comprende varios elementos que deben abordarse en este contexto: definición de los datos; formulación de aparatos de recolección y almacenamiento de datos; análisis de datos y aplicación de productos de análisis. Además, revisamos algunas oportunidades clave y algunos desafíos importantes del uso de big data en la educació