Analytics of self-regulated learning: a temporal and sequential approach

In educational settings, the increasingly sophisticated use of digital technology has provided students with greater agency over their learning. This has focused educational research on the metacognitive and cognitive activities with which students engage to manage their learning and the achievement of their learning goals. This field of research is articulated as self-regulated learning (SRL) and has seen the development of several key theoretical models. Despite key differences, these models are broadly defined by thematic variations of the same fundamental phases: i) a preparatory phase; ii) a performance phase, and; iii) an appraisal phase. Given the phasic nature of these models, the conceptualisation of SRL as a phenomenon that unfolds in temporal space has gained much traction. In acknowledging this dimension of SRL, researchers are bound to address the methodological demands of process, sequence, and temporality. Learning Analytics research, however, is largely characterised by the use of statistical models for data interrogation and analysis. Despite their value, several researchers posit that the use of statistical methods imposes ontological limitations with respect to the temporal and sequential nature of SRL. Another challenge is that while learner data are mostly collected at the micro level, (e.g., page access, video view, quiz attempt), SRL theory is defined at a macro level (e.g., planning, monitoring, evaluation), highlighting a need to bridge this gap in order to provide meaningful results. This thesis aims to explore the methodological opportunities and address the theoretical challenges presented in the area of temporally focused SRL learning analytics. First, the thesis explores the corpus of research in the area. As such, we present a systematic review of literature that analyses the findings of studies that explore SRL through the lenses of order and sequence, to provide insights into the temporal dynamics of SRL. Second, the thesis demonstrates the use of a novel process mining method to analyse how certain temporal activity traits relate to academic performance. We determined that more strategically minded activity, embodying aspects self-regulation, generally demonstrated to be more successful than less disciplined reactive behaviours. Third, the thesis presents a methodological framework designed to embed our analyses in a model of SRL. It comprises the use of: i) micro-level processing to transform raw trace data into SRL processes; and ii) first order Markov models to explore the temporal associations between SRL processes. We call this the “Trace-SRL” framework. Fourth, using the Trace-SRL framework, the thesis explores the deployment of multiple analytic methods and posits that richer insights can be gained through a combined methodological perspective. Fifth, building on this theme, the thesis presents a systematic analysis of four process mining algorithms, as deployed in the exploration of common SRL event data, concluding that the choice of algorithm and metric is of key importance in temporally-focused SRL research, and that combined metrics can provide deeper insights than those presented individually. Finally, the thesis concludes with a discussion of practical implications, the significance of the results, and future research directions

Towards investigating the validity of measurement of self-regulated learning based on trace data

Contains fulltext : 250033.pdf (Publisher’s version ) (Open Access)Contemporary research that looks at self-regulated learning (SRL) as processes of learning events derived from trace data has attracted increasing interest over the past decade. However, limited research has been conducted that looks into the validity of trace-based measurement protocols. In order to fill this gap in the literature, we propose a novel validation approach that combines theory-driven and data-driven perspectives to increase the validity of interpretations of SRL processes extracted from trace-data. The main contribution of this approach consists of three alignments between trace data and think aloud data to improve measurement validity. In addition, we define the match rate between SRL processes extracted from trace data and think aloud as a quantitative indicator together with other three indicators (sensitivity, specificity and trace coverage), to evaluate the "degree" of validity. We tested this validation approach in a laboratory study that involved 44 learners who learned individually about the topic of artificial intelligence in education with the use of a technology-enhanced learning environment for 45 minutes. Following this new validation approach, we achieved an improved match rate between SRL processes extracted from trace-data and think aloud data (training set: 54.24%; testing set: 55.09%) compared to the match rate before applying the validation approach (training set: 38.97%; test set: 34.54%). By considering think aloud data as "reference point", this improvement of the match rate quantified the extent to which validity can be improved by using our validation approach. In conclusion, the novel validation approach presented in this study used both empirical evidence from think aloud data and rationale from our theoretical framework of SRL, which now, allows testing and improvement of the validity of trace-based SRL measurements.39 p

The sequence matters: A systematic literature review of using sequence analysis in Learning Analytics

Describing and analysing sequences of learner actions is becoming more popular in learning analytics. Nevertheless, the authors found a variety of definitions of what a learning sequence is, of which data is used for the analysis, and which methods are implemented, as well as of the purpose and educational interventions designed with them. In this literature review, the authors aim to generate an overview of these concepts to develop a decision framework for using sequence analysis in educational research. After analysing 44 articles, the conclusions enable us to highlight different learning tasks and educational settings where sequences are analysed, identify data mapping models for different types of sequence actions, differentiate methods based on purpose and scope, and identify possible educational interventions based on the outcomes of sequence analysis.Comment: Submitted to the Journal of Learning Analytic

Modeling temporality in person and variable centered approaches.

Learning analytics needs to pay more attention to the temporal aspect of learning processes, especially in self-regulated learning (SRL) research. In doing so, learning analytics models should incorporate both the duration and frequency of learning activities, the passage of time, and the temporal order of learning activities. However, where this exhortation is widely supported, there is less agreement on its consequences. Temporal aspects of learning processes could be presented as events, but does paying tribute to temporal aspects necessarily imply that event-based models are to replace variable-based models, and whether analytic discovery methods could or even should substitute traditional statistical methods? Our contribution will reason that we do not require such a paradigm shift to give temporal aspects the position it deserves. First, we argue that temporal aspects can be well integrated into variable-based models that apply statistical methods by carefully choosing appropriate time windows and granularity levels. Our second argument is that in addressing temporality in learning analytic models that describe authentic learning settings, heterogeneity is of crucial importance in both variable and event-based models. Variable-based person-centered modeling where a heterogeneous sample is split into homogeneous subsamples is suggested as a solution. Our conjecture is illustrated by an application of dispositional learning analytics, describing authentic learning processes over an eight-week full module of 2,360 students

Temporal learning analytics to explore traces of self-regulated learning behaviors and their associations with learning performance, cognitive load, and student engagement in an asynchronous online course

Self-regulated learning (SRL) plays a critical role in asynchronous online courses. In recent years, attention has been focused on identifying student subgroups with different patterns of online SRL behaviors and comparing their learning performance. However, there is limited research leveraging traces of SRL behaviors to detect student subgroups and examine the subgroup differences in cognitive load and student engagement. The current study tracked the engagement of 101 graduate students with SRL-enabling tools integrated into an asynchronous online course. According to the recorded SRL behaviors, this study identified two distinct student subgroups, using sequence analysis and cluster analysis: high SRL (H-SRL) and low SRL (L-SRL) groups. The H-SRL group showed lower extraneous cognitive load and higher learning performance, germane cognitive load, and cognitive engagement than the L-SRL group did. Additionally, this study articulated and compared temporal patterns of online SRL behaviors between the student subgroups combining lag sequential analysis and epistemic network analysis. The results revealed that both groups followed three phases of self-regulation but performed off-task behaviors. Additionally, the H-SRL group preferred activating mastery learning goals to improve ethical knowledge, whereas the L-SRL group preferred choosing performance-avoidance learning goals to pass the unit tests. The H-SRL group invested more in time management and notetaking, whereas the L-SRL group engaged more in surface learning approaches. This study offers researchers both theoretical and methodological insights. Additionally, our research findings help inform practitioners about how to design and deploy personalized SRL interventions in asynchronous online courses

Analytics of time management strategies in online learning environments: a novel methodological approach

The emergence of technology-supported education, e.g., blended and online, has changed the global higher education landscape. Importantly, the new learning modes involve more complex tasks and challenging ways of learning that require effective time management and strong self-regulation skills. In this regard, one of the most prevalent theoretical lenses to understand learning processes is Self-Regulated Learning (SRL). In reference to SRL models, time is a major resource in learning. The way learners schedule, plan, and enact tactics and strategies on their learning time could tremendously impact their academic achievement. However, the assessment of how learners make time-related decisions in learning is a daunting task, particularly given its latent nature and inherent autonomous learning capacity. One way to address this problem is to make use of unprecedented volumes of data collected by digital learning environments that are precisely timestamped records of actions that learners take while studying. This thesis presents a set of novel learning analytics methods for detecting and understanding time management strategies based on the analysis of digital trace data collected in online learning environments. First, the thesis proposes a new method to detect time management tactics and strategies using a combination of sequence mining and clustering techniques. The thesis also describes how time management tactics and strategies detected with this method are aligned with an SRL model that is used as a theoretical foundation of this thesis. Second, the thesis introduces a novel learning analytics method for the detection of time management tactics and strategies. This method uses a combination of process mining and clustering techniques followed by a complementary process mining technique that has a unique feature to bring insights into the temporal learning processes. This new method also has a strong potential to inform and enhance understanding of how learners make complex decisions about their learning. Third, the thesis investigates mutual connections between time management and learning strategies and their combined connections with academic performance using epistemic network analysis. This analysis provides empirical evidence that supports the proposition that time management is a critical characteristic of effective self-regulated learners. Fourth, the thesis proposes a novel method that integrates computational and visualization techniques to explore the frequency, connections, ordering, and the time of the execution of time management and learning tactics, which usually been done in isolation in the existing literature. Then, the thesis quantitatively and theoretically compare time management and learning strategies detected with this new method to explore the role of time management and learning strategies in learning as drawing on theories of educational psychology. Fifth, this new method was validated in a study that was conducted on the trace data of different learning modalities and interaction modes, where large cohorts are involved. This final study emphasizes the importance of multivocality approach in the study of time management and other relevant learning constructs. Finally, the thesis concludes with a discussion of practical implications, the significance of the results, and future research directions

Course Design Approaches and Behavioral Patterns in Massive Open Online Courses for Professional Learning

Despite their growing importance, differential, process-oriented research on Massive Open Online Courses (MOOCs) for professional learning is scarce. This paper explores learner behavior in Enterprise MOOCs using lag sequential analysis. Data from 13 MOOCs on business and technology-related topics with a total of N = 72,668 active learners were examined. Starting from consistent high-level behavioral patterns, a deeper analysis reveals variations in interaction sequences according to the underlying course design approach. Lecture-oriented, system interaction-oriented, and discussion-oriented courses share a set of common patterns but also differ in various interaction sequences. Results point towards an isolated role of video playbacks across all course clusters, consumerist patterns in lecture-oriented courses, and a positive influence of metacognitively oriented interactions on learning outcomes. Accordingly, initial design recommendations include integrating interactive instructional elements in videos, promoting learner engagement in lecture-oriented courses, and fostering metacognition. Connecting interaction and achievement data may uncover promising behavior patterns that can be further supported by course design. Based on the initial findings, implications for future research and development are discussed

Course design approaches and behavioral patterns in massive open online courses for professional learning

Despite their growing importance, differential, processoriented research on Massive Open Online Courses (MOOCs) for professional learning is scarce. This paper explores learner behavior in Enterprise MOOCs using lag sequential analysis. Data from 13 MOOCs on business and technology-related topics with a total of N = 72,668 active learners were examined. Starting from consistent high-level behavioral patterns, a deeper analysis reveals variations in interaction sequences according to the underlying course design approach. Lecture-oriented, system interaction-oriented, and discussion-oriented courses share a set of common patterns but also differ in various interaction sequences. Results point towards an isolated role of video playbacks across all course clusters, consumerist patterns in lecture-oriented courses, and a positive influence of metacognitively oriented interactions on learning outcomes. Accordingly, initial design recommendations include integrating interactive instructional elements in videos, promoting learner engagement in lecture-oriented courses, and fostering metacognition. Connecting interaction and achievement data may uncover promising behavior patterns that can be further supported by course design. Based on the initial findings, implications for future research and development are discussed

Mental Toughness in the Workplace

Within the realm of the modern work organization, the employers role on employees’ mental health has emerged as a critical factor concerning job attrition. Frequently, work organizations prioritize their productivity and output, rather than recognizing the importance of individual contributions as a reflection of their well-being. Research on workplace satisfaction and adversity management has failed to address the challenges that non-military or law enforcement populations face in retaining talent. Current research on mental toughness, resilience, and grit has overlooked the importance of civilian population training interventions, leaving a gap in the literature. This mixed-method study used an explanatory sequential design, combining quantitative survey data with phenomenology, to address this gap by examining the instances of workplace adversities of 144 non-military or law enforcement employees. The research questions explored the relationships between mental toughness, grit, workplace satisfaction and perceived adversity in the workplace, framed within the incremental theory and cognitive affective model (CAP). Quantitative data suggested that there was a positive correlation between mental toughness and grit, workplace satisfaction and mental toughness, and between workplace satisfaction and grit. Qualitative data analysis revealed three primary themes in the participants’ written stories when asked about their instances of workplace adversity: tone, emotionality, and temporal focus. The findings emphasize the need for employers to prioritize employee mental toughness and resilience interventions in an effort to bolster workplace satisfaction, minimize quitting intention, and retain talent while improving overall well-being