Stealth Assessment of Self-Regulative Behaviors within a Game-Based Environment

abstract: Students' ability to regulate and control their behaviors during learning has been shown to be a critical skill for academic success. However, researchers often struggle with ways to capture the nuances of this ability, often solely relying on self-report measures. This thesis proposal employs a novel approach to investigating variations in students' ability to self-regulate by using process data from the game-based Intelligent Tutoring System (ITS) iSTART-ME. This approach affords a nuanced examination of how students' regulate their interactions with game-based features at both a coarse-grained and fine-grain levels and the ultimate impact that those behaviors have on in-system performance and learning outcomes (i.e., self-explanation quality). This thesis is comprised of two submitted manuscripts that examined how a group of 40 high school students chose to engage with game-based features and how those interactions influenced their target skill performance. Findings suggest that in-system log data has the potential to provide stealth assessments of students' self-regulation while learning.Dissertation/ThesisM.A. Psychology 201

Rich environments for active learning: a definition

Rich Environments for Active Learning, or REALs, are comprehensive instructional systems that evolve from and are consistent with constructivist philosophies and theories. To embody a constructivist view of learning, REALs: promote study and investigation within authentic contexts; encourage the growth of student responsibility, initiative, decision making, and intentional learning; cultivate collaboration among students and teachers; utilize dynamic, interdisciplinary, generative learning activities that promote higher-order thinking processes to help students develop rich and complex knowledge structures; and assess student progress in content and learning-to-learn within authentic contexts using realistic tasks and performances. REALs provide learning activities that engage students in a continuous collaborative process of building and reshaping understanding as a natural consequence of their experiences and interactions within learning environments that authentically reflect the world around them. In this way, REALs are a response to educational practices that promote the development of inert knowledge, such as conventional teacher-to-student knowledge-transfer activities. In this article, we describe and organize the shared elements of REALs, including the theoretical foundations and instructional strategies to provide a common ground for discussion. We compare existing assumptions underlying education with new assumptions that promote problem-solving and higher-level thinking. Next, we examine the theoretical foundation that supports these new assumptions. Finally, we describe how REALs promote these new assumptions within a constructivist framework, defining each REAL attribute and providing supporting examples of REAL strategies in action

Modeling Student Affective State Patterns during Self-Regulated Learning in Physics Playground

This dissertation research focuses on investigating the incidence of student self-regulated learning behavior, and examines patterns in student affective states that accompany such self-regulated behavior. This dissertation leverages prediction models of student affective states in the Physics Playground educational game platform to identify common patterns in student affective states during use of self-regulated learning behavior. In Study 1, prediction models of student affective states are developed in the context of the educational game environment Physics Playground, using affective state observations and computer log data that had already been collected as part of a larger project. The performances of student affective state prediction models generated using a combination of the computer log and observational data are then compared against those of similar prediction models generated using video data collected at the same time. In Study 2, I apply these affective state prediction models to generate predictions of student affective states on a broader set of data collected from students participants playing Physics Playground. In parallel, I define aggregated behavioral features that represent the self-observation and strategic planning components of self-regulated learning. Affective state predictions are then mapped to playground level attempts that contain these self-regulated learning behavioral features, and sequential pattern mining is applied to the affective state predictions to identify the most common patterns in student emotions. Findings from Study 1 demonstrate that both video data and interaction log data can be used to predict student affective states with significant accuracy. Since the video data is a direct measure of student emotions, it shows better performance across most affective states. However, the interaction log data can be collected natively by Physics Playground and is able to be generalized more easily to other learning environments. Findings from Study 2 suggest that self-regulatory behavior is closely associated with sustained periods of engaged concentration and .self-regulated learning behaviors are associated with transitions from negative affective states (confusion, frustration, and boredom) to the positive engaged concentration state. The results of this dissertation project demonstrate the power of measuring student affective states in real time and examining the temporal relationship to self-regulated learning behavior within an unstructured educational game platform. These results thus provide a building block for future research on the real-time assessment of student emotions and its relationship with self-regulated learning behaviors, particularly within online student-centered and self-directed learning contexts

Power to the Teachers:An Exploratory Review on Artificial Intelligence in Education

This exploratory review attempted to gather evidence from the literature by shedding light on the emerging phenomenon of conceptualising the impact of artificial intelligence in education. The review utilised the PRISMA framework to review the analysis and synthesis process encompassing the search, screening, coding, and data analysis strategy of 141 items included in the corpus. Key findings extracted from the review incorporate a taxonomy of artificial intelligence applications with associated teaching and learning practice and a framework for helping teachers to develop and self-reflect on the skills and capabilities envisioned for employing artificial intelligence in education. Implications for ethical use and a set of propositions for enacting teaching and learning using artificial intelligence are demarcated. The findings of this review contribute to developing a better understanding of how artificial intelligence may enhance teachers’ roles as catalysts in designing, visualising, and orchestrating AI-enabled teaching and learning, and this will, in turn, help to proliferate AI-systems that render computational representations based on meaningful data-driven inferences of the pedagogy, domain, and learner models

Applying science of learning in education: Infusing psychological science into the curriculum

The field of specialization known as the science of learning is not, in fact, one field. Science of learning is a term that serves as an umbrella for many lines of research, theory, and application. A term with an even wider reach is Learning Sciences (Sawyer, 2006). The present book represents a sliver, albeit a substantial one, of the scholarship on the science of learning and its application in educational settings (Science of Instruction, Mayer 2011). Although much, but not all, of what is presented in this book is focused on learning in college and university settings, teachers of all academic levels may find the recommendations made by chapter authors of service. The overarching theme of this book is on the interplay between the science of learning, the science of instruction, and the science of assessment (Mayer, 2011). The science of learning is a systematic and empirical approach to understanding how people learn. More formally, Mayer (2011) defined the science of learning as the “scientific study of how people learn” (p. 3). The science of instruction (Mayer 2011), informed in part by the science of learning, is also on display throughout the book. Mayer defined the science of instruction as the “scientific study of how to help people learn” (p. 3). Finally, the assessment of student learning (e.g., learning, remembering, transferring knowledge) during and after instruction helps us determine the effectiveness of our instructional methods. Mayer defined the science of assessment as the “scientific study of how to determine what people know” (p.3). Most of the research and applications presented in this book are completed within a science of learning framework. Researchers first conducted research to understand how people learn in certain controlled contexts (i.e., in the laboratory) and then they, or others, began to consider how these understandings could be applied in educational settings. Work on the cognitive load theory of learning, which is discussed in depth in several chapters of this book (e.g., Chew; Lee and Kalyuga; Mayer; Renkl), provides an excellent example that documents how science of learning has led to valuable work on the science of instruction. Most of the work described in this book is based on theory and research in cognitive psychology. We might have selected other topics (and, thus, other authors) that have their research base in behavior analysis, computational modeling and computer science, neuroscience, etc. We made the selections we did because the work of our authors ties together nicely and seemed to us to have direct applicability in academic settings

Educational Theories and Learning Analytics : From Data to Knowledge

Under embargo until 17.01.21.acceptedVersio

The Effects of Self-Regulated Learning Strategy Instruction and Structured-Diary Use on Students\u27 Self-Regulated Learning Conduct and Academic Success in Online Community-College General Education Courses

Student success in community-college online courses remains a topic of concern within higher-education research. Online courses offer flexibility and opportunities for students to learn anytime and anywhere. Students who are not prepared for the anytime-anywhere format struggle in online courses. As enrollment in online courses increases, the rate at which students persist through courses with satisfactory academic success is inconsistent. Effective ways to promote student success in online courses is an area that remains under-researched. Self-regulated learning has been shown to promote online student success by supporting student engagement, learning strategy use, and consistent evaluation of academic performance through instructional interventions and practice adopting the self-regulated learning process. The mixed methods study examined the effect of self-regulated learning strategy interventions on students\u27 self-regulated learning conduct and academic success in community-college online courses. Two intact classes of community-college online students participated in the studies in two subsequent quarters. Both curriculum-embedded interventions included instruction in a self-regulated learning strategic framework focused on, goal setting, actions, monitoring, and evaluation of self-regulated learning processes, followed by weekly implementation of the framework throughout the duration of online courses. Students\u27 perceptions were assessed before and after intervention and compared with academic performance, final course grades. Additionally, students completed structured-diary responses to evaluate implementation of self-regulated learning process. Results indicated that increases in students\u27 perceptions of self-regulated learning behaviors postintervention were significant in Study 1, and not significant in Study 2. Increases in students\u27 perceptions of metacognition were significant in both studies. Relationships between final course grades and students\u27 perceptions postintervention were moderate and not significant. Structured-diary responses revealed that students set goals centered on completing course assignments and time management and employed several learning strategies in support of achieving goals. Students perceived the framework as straightforward, adaptable, and effective. Results suggest that self-regulated learning strategy intervention was successful in raising the metacognitive awareness and self-regulated learning skill levels of community-college online students. Increased metacognitive awareness and self-regulated learning skills positively contributed to students\u27 efficacy for academic success in online courses. Implications of these studies contribute to research examining self-regulated learning strategy instruction as a means for promoting online student success

Integrating knowledge tracing and item response theory: A tale of two frameworks

Traditionally, the assessment and learning science commu-nities rely on different paradigms to model student performance. The assessment community uses Item Response Theory which allows modeling different student abilities and problem difficulties, while the learning science community uses Knowledge Tracing, which captures skill acquisition. These two paradigms are complementary - IRT cannot be used to model student learning, while Knowledge Tracing assumes all students and problems are the same. Recently, two highly related models based on a principled synthesis of IRT and Knowledge Tracing were introduced. However, these two models were evaluated on different data sets, using different evaluation metrics and with different ways of splitting the data into training and testing sets. In this paper we reconcile the models' results by presenting a unified view of the two models, and by evaluating the models under a common evaluation metric. We find that both models are equivalent and only differ in their training procedure. Our results show that the combined IRT and Knowledge Tracing models offer the best of assessment and learning sciences - high prediction accuracy like the IRT model, and the ability to model student learning like Knowledge Tracing

Qualitative, quantitative, and data mining methods for analyzing log data to characterize students' learning strategies and behaviors [discussant]

This symposium addresses how different classes of research methods, all based upon the use of log data from educational software, can facilitate the analysis of students’ learning strategies and behaviors. To this end, four multi-method programs of research are discussed, including the use of qualitative, quantitative-statistical, quantitative-modeling, and educational data mining methods. The symposium presents evidence regarding the applicability of each type of method to research questions of different grain sizes, and provides several examples of how these methods can be used in concert to facilitate our understanding of learning processes, learning strategies, and behaviors related to motivation, meta-cognition, and engagement