Development of Self-Regulated Learning Skills Within Open-Ended Computer-Based Learning Environments for Science

Over the past decade, open-ended computer-based learning environments have been increasingly used to facilitate students’ learning of complex scientific topics. The non-linearity and open-endedness of these environments create learning opportunities for students, but can also impose challenges in terms of extraneous cognitive load and greater requirements for self-regulated learning (SRL). SRL is crucial for academic success in various educational settings. This dissertation explores how self-regulatory skills develop and the role of gender in the development of SRL skills in Virtual Performance Assessments (VPA), an immersive, open-ended virtual environment designed to assess middle school students’ science inquiry skills. Findings from three analyses combining educational data mining techniques with multilevel modeling indicated that students developed self-regulatory behaviors and strategies as they used VPA. For example, experience with VPA prepared students to adopt more efficient note-taking and note-reviewing strategies. Students who used VPA for the second time engaged in note-taking more frequently, noted a significantly higher quantity of unique information, used the control of variables strategy more frequently in note-taking, and reproduced more domain-specific declarative information in notes than students who used VPA for the first time, all of which have been found to be positively associated with science inquiry performance. Students also learned to exploit more available sources of information by applying learning strategies, in order to either solve inquiry problems, or to monitor and evaluate their solutions. Compared to the second-time users who focused primarily on answering the core inquiry question and selectively collected data, the first-time users’ behaviors showed the repetition and combination of exploratory actions such as talking with non-player characters and collecting data. In addition, consistent gender differences in SRL were observed in this study. Female students were more likely to take notes than male students; they took notes and reviewed notes more frequently and recorded a higher quantity of information in notes, especially information from the research kiosk. Females were also more likely to review notes or read research pages to assist them with the problem-solving and decision-making process than their male counterparts. Possibly due to the higher quantity of information recorded by female note-takers and their tendency to review notes over males, female students’ performance on science inquiry tasks improved across the course of using the two scenarios of VPA, whereas the male students’ science inquiry skills did not show improvement. Results from this dissertation study provide insights into the instructional design of personalized open-ended learning environments to facilitate self-regulated learning for both male and female students

A LEARNER INTERACTION STUDY OF DIFFERENT ACHIEVEMENT GROUPS IN MPOCS WITH LEARNING ANALYTICS TECHNIQUES

The purpose of this study was to conduct data-driven research by employing learning analytics methodology and Big Data in learning management systems (LMSs), and then to identify and compare learners’ interaction patterns in different achievement groups through different course processes in Massive Private Online Courses (MPOCs). Learner interaction is the foundation of a successful online learning experience. However, the uncertainties about the temporal and sequential patterns of online interaction and the lack of knowledge about using dynamic interaction traces in LMSs have prevented research on ways to improve interactive qualities and learning effectiveness in online learning. Also, most research focuses on the most popular online learning organization form, Massive Open Online Courses (MOOCs), and little online learning research has been conducted to investigate learners’ interaction behaviors in another important online learning organization form: MPOCs. To fill these needs, the study pays attention to investigate the frequent and effective interaction patterns in different achievement groups as well as in different course processes, and attaches importance to LMS trace data (log data) in better serving learners and instructors in online learning. Further, the learning analytics methodology and techniques are introduced here into online interaction research. I assume that learners with different achievements express different interaction characteristics. Therefore, the hypotheses in this study are: 1) the interaction activity patterns of the high-achievement group and the low-achievement group are different; 2) in both groups, interaction activity patterns evolve through different course processes (such as the learning process and the exam process). The final purpose is to find interaction activity patterns that characterize the different achievement groups in specific MPOCs courses. Some learning analytics approaches, including Hidden Markov models (HMMs) and other related measures, are taken into account to identify frequently occurring interaction activity sequence patterns of High/Low achievement groups in the Learning/Exam processes under MPOCs settings. The results demonstrate that High-achievement learners especially focused on content learning, assignments, and quizzes to consolidate their knowledge construction in both Learning and Exam processes, while Low-achievement learners significantly did not perform the same. Further, High-achievement learners adjusted their learning strategies based on the goals of different course processes; Low-achievement learners were inactive in the learning process and opportunistic in the exam process. In addition, despite achievements or course processes, all learners were most interested in checking their performance statements, but they engaged little in forum discussion and group learning. In sum, the comparative analysis implies that certain interaction patterns may distinguish the High-achievement learners from the Low-achievement ones, and learners change their patterns more or less based on different course processes. This study provides an attempt to conduct learner interaction research by employing learning analytics techniques. In the short term, the results will give in-depth knowledge of the dynamic interaction patterns of MPOCs learners. In the long term, the results will help learners to gain insight into and evaluate their learning, help instructors identify at-risk learners and adjust instructional strategies, help developers and administrators to build recommendation systems based on objective and comprehensive information, all of which in turn will help to improve the achievements of all learner groups in specific MPOC courses

Chapter 38 Learning Analytics

In this chapter, we present an overview of the field by articulating definitions and existing models of learning analytics. Case examples of learning analytics from Asian researchers are then summarized and reported. This is followed by an exploration of the key tensions in this field. The chapter concludes with a discussion of potential areas for future research in this area

The sequence matters: A systematic literature review of using sequence analysis in Learning Analytics

Describing and analysing sequences of learner actions is becoming more popular in learning analytics. Nevertheless, the authors found a variety of definitions of what a learning sequence is, of which data is used for the analysis, and which methods are implemented, as well as of the purpose and educational interventions designed with them. In this literature review, the authors aim to generate an overview of these concepts to develop a decision framework for using sequence analysis in educational research. After analysing 44 articles, the conclusions enable us to highlight different learning tasks and educational settings where sequences are analysed, identify data mapping models for different types of sequence actions, differentiate methods based on purpose and scope, and identify possible educational interventions based on the outcomes of sequence analysis.Comment: Submitted to the Journal of Learning Analytic

New measurement paradigms

This collection of New Measurement Paradigms papers represents a snapshot of the variety of measurement methods in use at the time of writing across several projects funded by the National Science Foundation (US) through its REESE and DR K–12 programs. All of the projects are developing and testing intelligent learning environments that seek to carefully measure and promote student learning, and the purpose of this collection of papers is to describe and illustrate the use of several measurement methods employed to achieve this. The papers are deliberately short because they are designed to introduce the methods in use and not to be a textbook chapter on each method. The New Measurement Paradigms collection is designed to serve as a reference point for researchers who are working in projects that are creating e-learning environments in which there is a need to make judgments about students’ levels of knowledge and skills, or for those interested in this but who have not yet delved into these methods

Examining scientific thinking processes in open-ended serious games through gameplay data

Research on scientific problem-solving emphasizes the importance of problem solving and scientific inquiry as central components of the twenty-first century skills. Research has shown that open-ended serious games can facilitate students’ development of specific skills and improve learning performance through scientific problem-solving. However, understanding how students learn these complex skills in a game environment is a major challenge, as much research depends on typical paper-and-pencil assessments and self-reported surveys or other traditional observational and quantitative methods. The participants of the study were 237 sixth graders from two middle schools in the Southwestern area of the United States. The students used an open-ended serious game called Alien Rescue as their science curriculum for three weeks. The purpose of this study is, first, to identify students’ navigation behavior patterns in cognitive processes between at-risk and non-at-risk students within Alien Rescue. To accomplish this purpose, this study intends to use gameplay data by incorporating the integrated method of lag sequential analysis and sequential pattern mining together with a statistical analysis. The findings confirmed that the integrated method helped to explore students’ latent navigation behaviors as well as discover the differences of problem-solving processes between non-at-risk and at-risk students. The second purpose of this study is to examine the relationship between students’ learning performance and their scientific inquiry behaviors, which emerged as students engaged with Probe Design Center in this serious game. The results showed that the game metrics developed in Probe Design Center improved the predictions of both in-game and after-game performance. The cluster analyses with game metrics confirmed four unique groups regarding students’ scientific inquiry behaviors in Probe Design Center. This study concluded that the integrated methods of serious games analytics enabled researchers to investigate in-depth cognitive processes and scientific inquiry behaviors within a specific cognitive tool, Probe Design Center, and discover unique behavior groups across different school settings. The researcher identified the challenges of at-risk students in their cognitive processes and highlighted the support needs for these students. Consequently, this study proposed an interactive dashboard using the data-driven evidences to provide teachers just-in-time information to support students’ cognitive processes.Curriculum and Instructio

Analyzing collaborative learning processes automatically

In this article we describe the emerging area of text classification research focused on the problem of collaborative learning process analysis both from a broad perspective and more specifically in terms of a publicly available tool set called TagHelper tools. Analyzing the variety of pedagogically valuable facets of learners’ interactions is a time consuming and effortful process. Improving automated analyses of such highly valued processes of collaborative learning by adapting and applying recent text classification technologies would make it a less arduous task to obtain insights from corpus data. This endeavor also holds the potential for enabling substantially improved on-line instruction both by providing teachers and facilitators with reports about the groups they are moderating and by triggering context sensitive collaborative learning support on an as-needed basis. In this article, we report on an interdisciplinary research project, which has been investigating the effectiveness of applying text classification technology to a large CSCL corpus that has been analyzed by human coders using a theory-based multidimensional coding scheme. We report promising results and include an in-depth discussion of important issues such as reliability, validity, and efficiency that should be considered when deciding on the appropriateness of adopting a new technology such as TagHelper tools. One major technical contribution of this work is a demonstration that an important piece of the work towards making text classification technology effective for this purpose is designing and building linguistic pattern detectors, otherwise known as features, that can be extracted reliably from texts and that have high predictive power for the categories of discourse actions that the CSCL community is interested in

Modeling Student Affective State Patterns during Self-Regulated Learning in Physics Playground

This dissertation research focuses on investigating the incidence of student self-regulated learning behavior, and examines patterns in student affective states that accompany such self-regulated behavior. This dissertation leverages prediction models of student affective states in the Physics Playground educational game platform to identify common patterns in student affective states during use of self-regulated learning behavior. In Study 1, prediction models of student affective states are developed in the context of the educational game environment Physics Playground, using affective state observations and computer log data that had already been collected as part of a larger project. The performances of student affective state prediction models generated using a combination of the computer log and observational data are then compared against those of similar prediction models generated using video data collected at the same time. In Study 2, I apply these affective state prediction models to generate predictions of student affective states on a broader set of data collected from students participants playing Physics Playground. In parallel, I define aggregated behavioral features that represent the self-observation and strategic planning components of self-regulated learning. Affective state predictions are then mapped to playground level attempts that contain these self-regulated learning behavioral features, and sequential pattern mining is applied to the affective state predictions to identify the most common patterns in student emotions. Findings from Study 1 demonstrate that both video data and interaction log data can be used to predict student affective states with significant accuracy. Since the video data is a direct measure of student emotions, it shows better performance across most affective states. However, the interaction log data can be collected natively by Physics Playground and is able to be generalized more easily to other learning environments. Findings from Study 2 suggest that self-regulatory behavior is closely associated with sustained periods of engaged concentration and .self-regulated learning behaviors are associated with transitions from negative affective states (confusion, frustration, and boredom) to the positive engaged concentration state. The results of this dissertation project demonstrate the power of measuring student affective states in real time and examining the temporal relationship to self-regulated learning behavior within an unstructured educational game platform. These results thus provide a building block for future research on the real-time assessment of student emotions and its relationship with self-regulated learning behaviors, particularly within online student-centered and self-directed learning contexts