UNDERSTANDING STUDENT BEHAVIORS USING IMMEDIATE FEEDBACK FEATURES IN A BLENDED LEARNING ENVIRONMENT

Feedback serves to close the gap between learners’ current understanding and the desired understanding. Informative feedback can keep students from holding onto misconceptions, actively engage learners in knowledge acquisition, and increase confidence and motivation to learn. Yet, in the context of higher education, it is usually not possible for instructors to provide timely feedback to every individual student. This is especially difficult in first-year foundational courses due to the large number of students. Online learning platforms offer a solution by providing students immediate feedback during the course of their interactions with formative assessment tools (e.g., online homework, quizzes, embedded questions in lecture videos). However, how students choose to interact with these features and how these features influence students’ learning experiences have not been well understood. Even less is known about student behaviors with these immediate feedback features in a blended learning class

Measuring Semantic Textual Similarity and Automatic Answer Assessment in Dialogue Based Tutoring Systems

This dissertation presents methods and resources proposed to improve onmeasuring semantic textual similarity and their applications in student responseunderstanding in dialogue based Intelligent Tutoring Systems. In order to predict the extent of similarity between given pair of sentences,we have proposed machine learning models using dozens of features, such as thescores calculated using optimal multi-level alignment, vector based compositionalsemantics, and machine translation evaluation methods. Furthermore, we haveproposed models towards adding an interpretation layer on top of similaritymeasurement systems. Our models on predicting and interpreting the semanticsimilarity have been the top performing systems in SemEval (a premier venue for thesemantic evaluation) for the last three years. The correlations between our models\u27predictions and the human judgments were above 0.80 for several datasets while ourmodels being very robust than many other top performing systems. Moreover, wehave proposed Bayesian. We have also proposed a novel Neural Network based word representationmapping approach which allows us to map the vector based representation of a wordfound in one model to the another model where the word representation is missing,effectively pooling together the vocabularies and corresponding representationsacross models. Our experiments show that the model coverage increased by few toseveral times depending on which model\u27s vocabulary is taken as a reference. Also,the transformed representations were well correlated to the native target modelvectors showing that the mapped representations can be used with condence tosubstitute the missing word representations in the target model. models to adapt similarity models across domains. Furthermore, we have proposed methods to improve open-ended answersassessment in dialogue based tutoring systems which is very challenging because ofthe variations in student answers which often are not self contained and need thecontextual information (e.g., dialogue history) in order to better assess theircorrectness. In that, we have proposed Probabilistic Soft Logic (PSL) modelsaugmenting semantic similarity information with other knowledge. To detect intra- and inter-sentential negation scope and focus in tutorialdialogs, we have developed Conditional Random Fields (CRF) models. The resultsindicate that our approach is very effective in detecting negation scope and focus intutorial dialogue context and can be further developed to augment the naturallanguage understanding systems. Additionally, we created resources (datasets, models, and tools) for fosteringresearch in semantic similarity and student response understanding inconversational tutoring systems

A Multifaceted Consideration of Motivation and Learning within ASSISTments

An approach to education gaining popularity in the modern classroom, adaptive tutoring systems offer interactive learning environments in which students can access immediate feedback and rich tutoring while teachers can achieve organized assessment for targeted interventions. Yet despite the benefits that these systems provide, a number of questions remain regarding the optimal inner workings of adaptive platforms. What is the recipe for optimal student performance within these platforms? What elements should be taken into consideration when designing these learning environments? Can facets of these platforms be harnessed to increase students’ motivation to learn and to improve both immediate and robust learning gains? This thesis combines work conducted over the past two years through versatile approaches toward the goal of enhancing student motivation and learning within the ASSISTments platform. Approaches considered include a) enhancing motivation and performance through altered feedback using hypermedia elements, b) instilling motivational messages alongside media enhanced content and feedback, c) allowing students to choose their feedback medium, thereby exerting control over their assignment, d) altering content delivery by interleaving skills to enhance solution strategy development, and e) establishing partial credit assessments to drive motivation and proper system usage while enhancing student modeling. After a brief introduction regarding the main tenants of this research, each chapter highlights a randomized controlled trial focused around one of these approaches. All studies presented have been conducted or are still running within ASSISTments. Much of this work has already been published at peer reviewed conference venues, some with stringent acceptance rates as low as 25% for full papers. Two of the studies presented here are second iterations of previously published work that are still in progress, and only preliminary analyses are available. A chapter on conclusions and future work is included to discuss the contributions that have been made to the Learning Sciences community thus far, and to briefly discuss potential directions for my continued research

Student Modeling in Intelligent Tutoring Systems

After decades of development, Intelligent Tutoring Systems (ITSs) have become a common learning environment for learners of various domains and academic levels. ITSs are computer systems designed to provide instruction and immediate feedback, which is customized to individual students, but without requiring the intervention of human instructors. All ITSs share the same goal: to provide tutorial services that support learning. Since learning is a very complex process, it is not surprising that a range of technologies and methodologies from different fields is employed. Student modeling is a pivotal technique used in ITSs. The model observes student behaviors in the tutor and creates a quantitative representation of student properties of interest necessary to customize instruction, to respond effectively, to engage students¡¯ interest and to promote learning. In this dissertation work, I focus on the following aspects of student modeling. Part I: Student Knowledge: Parameter Interpretation. Student modeling is widely used to obtain scientific insights about how people learn. Student models typically produce semantically meaningful parameter estimates, such as how quickly students learn a skill on average. Therefore, parameter estimates being interpretable and plausible is fundamental. My work includes automatically generating data-suggested Dirichlet priors for the Bayesian Knowledge Tracing model, in order to obtain more plausible parameter estimates. I also proposed, implemented, and evaluated an approach to generate multiple Dirichlet priors to improve parameter plausibility, accommodating the assumption that there are subsets of skills which students learn similarly. Part II: Student Performance: Student Performance Prediction. Accurately predicting student performance is one of the most desired features common evaluations for student modeling. for an ITS. The task, however, is very challenging, particularly in predicting a student¡¯s response on an individual problem in the tutor. I analyzed the components of two common student models to determine which aspects provide predictive power in classifying student performance. I found that modeling the student¡¯s overall knowledge led to improved predictive accuracy. I also presented an approach, which, rather than assuming students are drawn from a single distribution, modeled multiple distributions of student performances to improve the model¡¯s accuracy. Part III: Wheel-spinning: Student Future Failure in Mastery Learning. One drawback of the mastery learning framework is its possibility to leave a student stuck attempting to learn a skill he is unable to master. We refer to this phenomenon of students being given practice with no improvement as wheel-spinning. I analyzed student wheel-spinning across different tutoring systems and estimated the scope of the problem. To investigate the negative consequences of see what wheel-spinning could have done to students, I investigated the relationships between wheel-spinning and two other constructs of interest about students: efficiency of learning and ¡°gaming the system¡±. In addition, I designed a generic model of wheel-spinning, which uses features easily obtained by most ITSs. The model can be well generalized to unknown students with high accuracy classifying mastery and wheel-spinning problems. When used as a detector, the model can detect wheel-spinning in its early stage with satisfying satisfactory precision and recall

Characterizing Productive Perseverance Using Sensor-Free Detectors of Student Knowledge, Behavior, and Affect

Failure is a necessary step in the process of learning. For this reason, there has been a myriad of research dedicated to the study of student perseverance in the presence of failure, leading to several commonly-cited theories and frameworks to characterize productive and unproductive representations of the construct of persistence. While researchers are in agreement that it is important for students to persist when struggling to learn new material, there can be both positive and negative aspects of persistence. What is it, then, that separates productive from unproductive persistence? The purpose of this work is to address this question through the development, extension, and study of data-driven models of student affect, behavior, and knowledge. The increased adoption of computer-based learning platforms in real classrooms has led to unique opportunities to study student learning at both fine levels of granularity and longitudinally at scale. Prior work has leveraged machine learning methods, existing learning theory, and previous education research to explore various aspects of student learning. These include the development of sensor-free detectors that utilize only the student interaction data collected through such learning platforms. Building off of the considerable amount of prior research, this work employs state-of-the-art machine learning methods in conjunction with the large scale granular data collected by computer-based learning platforms in alignment with three goals. First, this work focuses on the development of student models that study learning through the use of advancements in student modeling and deep learning methodologies. Second, this dissertation explores the development of tools that incorporate such models to support teachers in taking action in real classrooms to promote productive approaches to learning. Finally, this work aims to complete the loop in utilizing these detector models to better understand the underlying constructs that are being measured through their application and their connection to productive perseverance and commonly-observed learning outcomes

Inclusion in Practice: An explanatory study of how patterns of classroom discourse shape processes of educational inclusion in Tanzanian secondary school classrooms

While access to secondary school is increasingly available to a larger and more heterogeneous population of learners, expansion in enrolment masks the reality that educational inequalities are widening. Using a comparative and in-depth qualitative research design, this dissertation explores processes of educational inclusion and exclusion as they unfold within the classroom walls, where it is teachers who determine how access to the curriculum and classroom culture is created and sustained. Data generated through 12 lesson observations and accompanying stimulated recall interviews enable consideration of how patterns of teacher discourse influence teacher capacity to foster inclusivity, and why such patterns emerge. Data gathered through key informant interviews and focus group discussions support the scrutinization and triangulation of tentative findings drawn from the observation and stimulated recall data. This research is premised on the need to expand our knowledge of what inclusive education can mean in classrooms typically characterized by material and human deprivation, and in teaching approaches most often defined by a strong pedagogic tradition of knowledge transmission. The findings support the construction of an explanatory framework revealing the ways in which patterns of classroom discourse shape teacher capacity to foster inclusion in Tanzanian secondary school classrooms. They reveal the importance of three moments embedded within each teacher-student interaction: The moment within which a turn is allocated, the moment within which a student response is interpreted by a teacher, and the moment within which a teacher then follows up. The findings demonstrate the power of the multiple continua of discursive strategies within which teachers move, revealing the specific decisions and respective trade-offs which influence teacher capacity to foster processes of educational inclusion. The findings are significant for expanding our understanding of the intersection between pedagogy and inclusion in the Tanzanian context, generating insight into that which teachers do, the reasons for why they do it, and the contextually relevant levers for change.N/

A Human-Centric System for Symbolic Reasoning About Code

While testing and tracing on specific input values are useful starting points for students to understand program behavior, ultimately students need to be able to reason rigorously and logically about the correctness of their code on all inputs without having to run the code. Symbolic reasoning is reasoning abstractly about code using arbitrary symbolic input values, as opposed to specific concrete inputs. The overarching goal of this research is to help students learn symbolic reasoning, beginning with code containing simple assertions as a foundation and proceeding to code involving data abstractions and loop invariants. Toward achieving this goal, this research has employed multiple experiments across five years at three institutions: a large, public university, an HBCU (Historically Black Colleges and Universities), and an HSI (Hispanic Serving Institution). A total of 862 students participated across all variations of the study. Interactive, online tools can enhance student learning because they can provide targeted help that would be prohibitively expensive without automation. The research experiments employ two such symbolic reasoning tools that had been developed earlier and a newly designed human-centric reasoning system (HCRS). The HCRS is a first step in building a generalized tutor that achieves a level of resolution necessary to identify difficulties and suggest appropriate interventions. The experiments show the value of tools in pinpointing and classifying difficulties in learning symbolic reasoning, as well as in learning design-by-contract assertions and applying them to develop loop invariants for code involving objects. Statistically significant results include the following. Students are able to learn symbolic reasoning with the aid of instruction and an online tool. Motivation improves student perception and attitude towards symbolic reasoning. Tool usage improves student performance on symbolic reasoning, their explanations of the larger purpose of code segments, and self-efficacy for all subpopulations

Modes and Mechanisms of Game-like Interventions in Intelligent Tutoring Systems

While games can be an innovative and a highly promising approach to education, creating effective educational games is a challenge. It requires effectively integrating educational content with game attributes and aligning cognitive and affective outcomes, which can be in conflict with each other. Intelligent Tutoring Systems (ITS), on the other hand, have proven to be effective learning environments that are conducive to strong learning outcomes. Direct comparisons between tutoring systems and educational games have found digital tutors to be more effective at producing learning gains. However, tutoring systems have had difficulties in maintaining students€™ interest and engagement for long periods of time, which limits their ability to generate learning in the long-term. Given the complementary benefits of games and digital tutors, there has been considerable effort to combine these two fields. This dissertation undertakes and analyzes three different ways of integrating Intelligent Tutoring Systems and digital games. We created three game-like systems with cognition, metacognition and affect as their primary target and mode of intervention. Monkey\u27s Revenge is a game-like math tutor that offers cognitive tutoring in a game-like environment. The Learning Dashboard is a game-like metacognitive support tool for students using Mathspring, an ITS. Mosaic comprises a series of mini-math games that pop-up within Mathspring to enhance students\u27 affect. The methodology consisted of multiple randomized controlled studies ran to evaluate each of these three interventions, attempting to understand their effect on students€™ performance, affect and perception of the intervention and the system that embeds it. Further, we used causal modeling to further explore mechanisms of action, the inter-relationships between student€™s incoming characteristics and predispositions, their mechanisms of interaction with the tutor, and the ultimate learning outcomes and perceptions of the learning experience

Supporting students’ confidence judgement through visualising alignment in open learner models

Supporting students’ knowledge monitoring skills, a component of metacognition, can help students regulate their own learning. This thesis investigates the alignment of learners’ confidence in their knowledge with a computer’s assessment of their knowledge, visualised using an Open Learner Model (OLM). The research explored students’ preferred method for visualising inconsistent data (e.g. misalignment) in an OLM, and the ways in which visualising alignment can influence student interaction with the computer. The thesis demonstrates that visualising alignment in Open Learner Models signifi-cantly increases students’ confidence compared to a control condition. In particular, visualising alignment benefited low-achieving students, in terms of knowledge monitoring and this was associated with improvements in their performance. Students showed a preference towards the visualisations that provides an overview of the in-formation (i.e. opacity) rather than ones, which provide detailed information. Graph-ical representation is shown to be more beneficial in motivating students to interact with the system than text-based representation of the same information in the con-text of representing the alignment within OLMs

Applying science of learning in education: Infusing psychological science into the curriculum

The field of specialization known as the science of learning is not, in fact, one field. Science of learning is a term that serves as an umbrella for many lines of research, theory, and application. A term with an even wider reach is Learning Sciences (Sawyer, 2006). The present book represents a sliver, albeit a substantial one, of the scholarship on the science of learning and its application in educational settings (Science of Instruction, Mayer 2011). Although much, but not all, of what is presented in this book is focused on learning in college and university settings, teachers of all academic levels may find the recommendations made by chapter authors of service. The overarching theme of this book is on the interplay between the science of learning, the science of instruction, and the science of assessment (Mayer, 2011). The science of learning is a systematic and empirical approach to understanding how people learn. More formally, Mayer (2011) defined the science of learning as the “scientific study of how people learn” (p. 3). The science of instruction (Mayer 2011), informed in part by the science of learning, is also on display throughout the book. Mayer defined the science of instruction as the “scientific study of how to help people learn” (p. 3). Finally, the assessment of student learning (e.g., learning, remembering, transferring knowledge) during and after instruction helps us determine the effectiveness of our instructional methods. Mayer defined the science of assessment as the “scientific study of how to determine what people know” (p.3). Most of the research and applications presented in this book are completed within a science of learning framework. Researchers first conducted research to understand how people learn in certain controlled contexts (i.e., in the laboratory) and then they, or others, began to consider how these understandings could be applied in educational settings. Work on the cognitive load theory of learning, which is discussed in depth in several chapters of this book (e.g., Chew; Lee and Kalyuga; Mayer; Renkl), provides an excellent example that documents how science of learning has led to valuable work on the science of instruction. Most of the work described in this book is based on theory and research in cognitive psychology. We might have selected other topics (and, thus, other authors) that have their research base in behavior analysis, computational modeling and computer science, neuroscience, etc. We made the selections we did because the work of our authors ties together nicely and seemed to us to have direct applicability in academic settings