Stability and sensitivity of Learning Analytics based prediction models

Learning analytics seek to enhance the learning processes through systematic measurements of learning related data and to provide informative feedback to learners and educators. Track data from Learning Management Systems (LMS) constitute a main data source for learning analytics. This empirical contribution provides an application of Buckingham Shum and Deakin Crick’s theoretical framework of dispositional learning analytics: an infrastructure that combines learning dispositions data with data extracted from computer-assisted, formative assessments and LMSs. In two cohorts of a large introductory quantitative methods module, 2049 students were enrolled in a module based on principles of blended learning, combining face-to-face Problem-Based Learning sessions with e-tutorials. We investigated the predictive power of learning dispositions, outcomes of continuous formative assessments and other system generated data in modelling student performance and their potential to generate informative feedback. Using a dynamic, longitudinal perspective, computer-assisted formative assessments seem to be the best predictor for detecting underperforming students and academic performance, while basic LMS data did not substantially predict learning. If timely feedback is crucial, both use-intensity related track data from e-tutorial systems, and learning dispositions, are valuable sources for feedback generation

Student profiling in a dispositional learning analytics application using formative assessment

How learning disposition data can help us translating learning feedback from a learning analytics application into actionable learning interventions, is the main focus of this empirical study. It extends previous work where the focus was on deriving timely prediction models in a data rich context, encompassing trace data from learning management systems, formative assessment data, e-tutorial trace data as well as learning dispositions. In this same educational context, the current study investigates how the application of cluster analysis based on e-tutorial trace data allows student profiling into different at-risk groups, and how these at-risk groups can be characterized with the help of learning disposition data. It is our conjecture that establishing a chain of antecedent-consequence relationships starting from learning disposition, through student activity in e-tutorials and formative assessment performance, to course performance, adds a crucial dimension to current learning analytics studies: that of profiling students with descriptors that easily lend themselves to the design of educational interventions

Long-Term Impact of Lifelong Fitness: Examining Longitudinal Exercise Behavior in College Students

Over time, the United States population has gradually shifted to an increasingly inactive lifestyle, and there has been a decline in health behavior. Only 50% of the population meet the recommended guidelines for weekly physical activity. With this glaring increase of inactive lifestyles, programs designed to increase health behavior change have become crucial. One solution to this problem has been a required Lifelong Fitness class at George Fox University where new college students learn knowledge and skills to implement for healthier lifestyles during this transformative time. A multiple regression model predicting long-term exercise by pre-minutes of exercise, post-minutes of exercise, and current exercise self-efficacy was fit to data. The model accounted for 18% of variance shared in all the predictors. When all variables were entered in the model, pre-minutes of exercise and current exercise self-efficacy were significant. There were no significant changes in mean levels of exercise longitudinally, suggesting the course helped students maintain levels of activity, but not increase them

Predicting Student Success in a Self-Paced Mathematics MOOC

abstract: While predicting completion in Massive Open Online Courses (MOOCs) has been an active area of research in recent years, predicting completion in self-paced MOOCS, the fastest growing segment of open online courses, has largely been ignored. Using learning analytics and educational data mining techniques, this study examined data generated by over 4,600 individuals working in a self-paced, open enrollment college algebra MOOC over a period of eight months. Although just 4% of these students completed the course, models were developed that could predict correctly nearly 80% of the time which students would complete the course and which would not, based on each student’s first day of work in the online course. Logistic regression was used as the primary tool to predict completion and focused on variables associated with self-regulated learning (SRL) and demographic variables available from survey information gathered as students begin edX courses (the MOOC platform employed). The strongest SRL predictor was the amount of time students spent in the course on their first day. The number of math skills obtained the first day and the pace at which these skills were gained were also predictors, although pace was negatively correlated with completion. Prediction models using only SRL data obtained on the first day in the course correctly predicted course completion 70% of the time, whereas models based on first-day SRL and demographic data made correct predictions 79% of the time.Dissertation/ThesisDoctoral Dissertation Educational Technology 201

Exploring Relations Between Motivation, Metacognition, and Academic Achievement Through Variable-Centered, Person-Centered and Learning Analytic Methodologies

The three studies that comprise this dissertation examine relations between student characteristics, motivations, metacognitive learning processes, and academic achievement. Methodologically, the dissertation demonstrated the potential of multiple types of approaches and data resource types. By employing multiple approaches including variable-centered, person-centered, and learning analytics, researchers can understand learning processes from various angles. In addition, through this triangulation by multiple types of methodological approaches, educational theories could be more thoroughly verified and supported by various empirical findings. Multiple types of data resources are related to analytical methods. The purpose of the first paper was to examine relations between achievement goals and metacognitive learning behaviors using a clustering analysis and visualization. A clustering analysis conducted with achievement goals produced three goal profiles; 1) mastery-approach, 2) performance-approach, and 3) performance-avoidance identified three goal profiles. The profiles include High Approach, High Mastery, and High Goal Endorsement groups. The finding demonstrated that students in the High Mastery group, who had greater use of the self-assessment tool, obtained higher final grades than other groups could be explained from the perspective of SRL. In addition, learners motivated by mastery approach goals engaged in the greater use of self-assessment quizzes. Students in the High Mastery group also used the tools earlier than other two groups for exam 2. As the most frequently used pattern, sequential pattern mining discovered the repeated use of self-assessment quizzes to monitor their learning. More students in the High Mastery group employ this pattern of metacognitive events than students in the High Performance and High-Goal endorsement groups, particularly during sessions in weeks before exams. A subsequent analysis revealed that for all exams, students who conducted a repeated behavior pattern indicative of metacognitive monitoring and control outperformed those who did not. From the research, it is confirmed that the person-centered analysis provided authentic and generalizable groups and afforded observation of the learning behaviors of learners with typical combinations of goals. In addition, sequential patterns provide instructor more interesting information on learning processes than the frequency of accesses. The purpose of the second research was to identify motivational profiles based on multiple types of motivations including self-efficacy, achievement goals, and expectancy-value from an integrative perspective. For this research, a LPA was conducted with ten types of motivational constructs and three kinds of metacognitive learning processes. The LPA identified four motivational profiles; 1) High Cost, 2) High Performance Goals, 3) High Goals and Values, and 4) Low Performance Goals, and three metacognitive profiles; 1) Infrequent metacognitive processing. 2) Checking performance and planning, and 3) Self-assessment. Student demographic information significantly influenced the membership of motivational profiles. Older students tend to have higher self-efficacy, mastery-approach, and values, but low cost than younger ones. In addition, compared to Caucasian and Asian students, underrepresented students tend to be more motivated by higher goals and values than high cost or high performance goals. Lastly, female students are more likely to be members of High performance goals and High goals and values than High cost oriented and Low performance goals and cost than males. In terms of the relations profiles with academic achievement, Low Performance Goals group showed the best performance. Among metacognitive profile groups, students in Checking performance and planning, and Self-assessment demonstrated similar academic performance. The investigation of relations between two profile groups demonstrated that students in the High cost group are more likely to be a member of self-assessment group than checking performance and planning as well as of a member of an infrequent metacognitive process than checking performance and planning. In addition, students in high performance and goals and high goals and values groups relative to the low performance goals group more likely to be a member of the infrequent metacognitive process than checking performance and planning. The findings of this research provide authentic motivation status and metacognition learning process as well as their relations. Addition, this research figured out specific motivational profiles through the multiple types of motivations from the integrative perspective. Therefore, instructors can provide more effective and specific interventions to students who have difficulty utilizing metacognitive learning processes, considering motivational status based on multiple motivations. In addition, instructors can understand motivational profiles by demographics so at the beginning of the semester in which the information on students is not enough to identify students learning processes, they intervene students based on demographic information. The purpose of the third paper was to consider the relative importance of capturing demographic, motivational and metacognitive processes as potential predictors of learning outcomes, and appraises them alongside both traditional prediction modeling approaches in higher education, and emergent methods, sequence pattern mining, arising from the field of educational data mining. The sequence pattern mining discovered the repeated use of self-assessment quizzes in Biology and repeated use of planning contents in Math. A regression model with combined resource types demonstrated the improved predictive power than models with individual resource types. Also, theory-aligned behaviors designed based on metacognitive learning processes better improved the accuracy of the model than non-theory-aligned behaviors automatically provided by the system. Lastly, when applying the same prediction model, the model better explained the variance of academic achievement in Biology in which metacognitive supporting tools designed based on an educational theory than that in Math that has few theory-aligned behavior variables. Therefore, this study emphasizes the importance of existing ambient data from university systems. Also, log data generated by systems such as LMS allows researchers to examine the same data in different ways with no need for additional data collection. Lastly, educational theory and contexts should be taken into consideration in designing courses and developing the prediction models. Therefore, instructors and researchers, in designing courses, the consideration of educational theories and contexts is the essential process. This dissertation provides insight regarding authentic relations between motivation, metacognition, and academic achievement. Specifically, instructors can understand how multiple types of motivations work together, and the motivational profiles influence metacognitive learning strategies. In courses, by examining motivational profiles, instructors can provide more effective intervention with which students change their resolve their weak learning easier. Practically, by investigating each type of predictor from data resources including demographic, motivation, and behavioral variables, findings from this dissertation can enable researchers to prioritize development of prediction models to identify students who are more likely to experience failure in courses. Additionally, instructors can figure out the importance of interpreting variables through educational theories and in context through the comparison of courses with differing instructional designs. Further, by appraising these results in light of theory, instructors can take action to improve student’s learning outcomes by adjusting the design of their courses

Eye quietness and quiet eye in expert and novice golf performance: an electrooculographic analysis

Quiet eye (QE) is the final ocular fixation on the target of an action (e.g., the ball in golf putting). Camerabased eye-tracking studies have consistently found longer QE durations in experts than novices; however, mechanisms underlying QE are not known. To offer a new perspective we examined the feasibility of measuring the QE using electrooculography (EOG) and developed an index to assess ocular activity across time: eye quietness (EQ). Ten expert and ten novice golfers putted 60 balls to a 2.4 m distant hole. Horizontal EOG (2ms resolution) was recorded from two electrodes placed on the outer sides of the eyes. QE duration was measured using a EOG voltage threshold and comprised the sum of the pre-movement and post-movement initiation components. EQ was computed as the standard deviation of the EOG in 0.5 s bins from –4 to +2 s, relative to backswing initiation: lower values indicate less movement of the eyes, hence greater quietness. Finally, we measured club-ball address and swing durations. T-tests showed that total QE did not differ between groups (p = .31); however, experts had marginally shorter pre-movement QE (p = .08) and longer post-movement QE (p < .001) than novices. A group × time ANOVA revealed that experts had less EQ before backswing initiation and greater EQ after backswing initiation (p = .002). QE durations were inversely correlated with EQ from –1.5 to 1 s (rs = –.48 - –.90, ps = .03 - .001). Experts had longer swing durations than novices (p = .01) and, importantly, swing durations correlated positively with post-movement QE (r = .52, p = .02) and negatively with EQ from 0.5 to 1s (r = –.63, p = .003). This study demonstrates the feasibility of measuring ocular activity using EOG and validates EQ as an index of ocular activity. Its findings challenge the dominant perspective on QE and provide new evidence that expert-novice differences in ocular activity may reflect differences in the kinematics of how experts and novices execute skills

A Prediction-Based Framework to Reduce Procrastination in Adaptive Learning Systems

Procrastination and other types of dilatory behaviour are common in online learning, especially in higher education. While procrastination is associated with worse performance and discomfort, positive forms of delay can be used as a deliberate strategy without any such consequences. Although dilatory behaviour has received attention in research, it has to my knowledge never been included as an integral part of an adaptive learning system. Differentiating between different types of delay within such a system would allow for tailored interventions to be provided in the future without alienating students who use delay as a successful strategy. In this thesis, I present four studies that provide the basis for such an endeavour. I first discuss the results of two studies that focussed on the prediction of the extent of dilatory behaviour in online assignments. The results of both studies revealed an advantage of objective predictors based on log data over subjective variables based on questionnaires. The predictive performance slightly improved when both sets of predictors were combined. In one of these studies, we implemented Bayesian multilevel models while the other aimed at comparing various machine learning algorithms to determine the best candidates for a future inclusion in real-time predictive models. The results reveal that the most suitable algorithm depended on the type of predictor, implying that multiple models should be implemented in the field, rather than selecting just one. I then present a framework for an adaptive learning system based on the other two studies, where I highlight how dilatory behaviour can be incorporated into such a system, in light of the previously discussed results. I conclude this thesis by providing an outlook into the necessary next steps before an adaptive learning system focussing on delay can be established

Strategies for promoting autonomous reading motivation: a multiple case study research in primary education

It is important to reveal strategies which foster students’ reading motivation in order to break through the declining trend in reading motivation throughout children’s educational careers. Consequently, the present study advances an underexposed field in reading motivation research by studying and identifying the strategies of teachers excellent in promoting fifth-grade students’ volitional or autonomous reading motivation through multiple case study analysis. Data on these excellent teachers were gathered from multiple sources (interviews with teachers, SEN coordinators, and school leaders; classroom observations; teacher and student questionnaires) and analysed. The results point to the teaching dimensions of autonomy support, structure, and involvement – as indicated by self-determination theory – as well as to reading aloud as critical strategies to promote students’ autonomous reading motivation in the classroom. A school culture supporting students’ and teachers’ interest in reading is also an essential part of reading promotion. The theoretical and practical significance of the study is discussed