Early-warning prediction of student performance and engagement in open book assessment by reading behavior analysis

Digitized learning materials are a core part of modern education, and analysis of the use can offer insight into the learning behavior of high and low performing students. The topic of predicting student characteristics has gained a lot of attention in recent years, with applications ranging from affect to performance and at-risk student prediction. In this paper, we examine students reading behavior using a digital textbook system while taking an open-book test from the perspective of engagement and performance to identify the strategies that are used. We create models to predict the performance and engagement of learners before the start of the assessment and extract reading behavior characteristics employed before and after the start of the assessment in a higher education setting. It was found that strategies, such as: revising and previewing are indicators of how a learner will perform in an open ebook assessment. Low performing students take advantage of the open ebook policy of the assessment and employ a strategy of searching for information during the assessment. Also compared to performance, the prediction of overall engagement has a higher accuracy, and therefore could be more appropriate for identifying intervention candidates as an early-warning intervention system

Time-Dependent Performance Prediction System for Early Insight in Learning Trends

Performance prediction systems allow knowing the learning status of students during a term and produce estimations on future status, what is invaluable information for teachers. The majority of current systems statically classify students once in time and show results in simple visual modes. This paper presents an innovative system with progressive, time-dependent and probabilistic performance predictions. The system produces by-weekly probabilistic classifications of students in three groups: high, medium or low performance. The system is empirically tested and data is gathered, analysed and presented. Predictions are shown as point graphs over time, along with calculated learning trends. Summary blocks are with latest predictions and trends are also provided for teacher efficiency. Moreover, some methods for selecting best moments for teacher intervention are derived from predictions. Evidence gathered shows potential to give teachers insights on students' learning trends, early diagnose learning status and selecting best moment for intervention

Time-Dependent Performance Prediction System for Early Insight in Learning Trends

Performance prediction systems allow knowing the learning status of students during a term and produce estimations on future status, what is invaluable information for teachers. The majority of current systems statically classify students once in time and show results in simple visual modes. This paper presents an innovative system with progressive, time-dependent and probabilistic performance predictions. The system produces by-weekly probabilistic classifications of students in three groups: high, medium or low performance. The system is empirically tested and data is gathered, analysed and presented. Predictions are shown as point graphs over time, along with calculated learning trends. Summary blocks are with latest predictions and trends are also provided for teacher efficiency. Moreover, some methods for selecting best moments for teacher intervention are derived from predictions. Evidence gathered shows potential to give teachers insights on students' learning trends, early diagnose learning status and selecting best moment for intervention

Predicting Academic Student Performance based on e-Learning Platform Engagement using Learning Management System Data

The identification of at-risk students has become increasingly more significant as these students are in the precarious position of failing their courses. This study aims to achieve the objective of proposing a student performance prediction model to identify the stage of the course where at-risk students (students with the highest potential of failing their courses) can be identified based on student information system and learning management system data. The proposed student performance prediction model leverages machine learning methods to predict at-risk students, combining data from Universiti Putra Malaysia’s (UPM) Student Information System (SIS) and learning management system (PutraBlast). Two experiments were conducted to satisfy the objective. The first experiment uses the full semester data to test multiple machine learning models to identify the best model for this dataset. In the second experiment, the dataset was separated into four course stages with four predictive models trained on each stage. Students. Results show that GB outperforms other classifiers when trained on the full semester data. However, classifier performance decreases when trained on data from earlier stages of the course. Hence, based on these results, the earliest stage to predict at-risk students is identified to be the W1—W12 stage

Effects of Early Warning Emails on Student Performance

We use learning data of an e-assessment platform for an introductory mathematical statistics course to predict the probability of passing the final exam for each student. Subsequently, we send warning emails to students with a low predicted probability to pass the exam. We detect a positive but imprecisely estimated effect of this treatment, suggesting the effectiveness of such interventions only when administered more intensively.Comment: arXiv admin note: text overlap with arXiv:1906.0986

Fine Grain Synthetic Educational Data: Challenges and Limitations of Collaborative Learning Analytics

While data privacy is a key aspect of Learning Analytics, it often creates difficulty when promoting research into underexplored contexts as it limits data sharing. To overcome this problem, the generation of synthetic data has been proposed and discussed within the LA community. However, there has been little work that has explored the use of synthetic data in real-world situations. This research examines the effectiveness of using synthetic data for training academic performance prediction models, and the challenges and limitations of using the proposed data sharing method. To evaluate the effectiveness of the method, we generate synthetic data from a private dataset, and distribute it to the participants of a data challenge to train prediction models. Participants submitted their models as docker containers for evaluation and ranking on holdout synthetic data. A post-hoc analysis was conducted on the top 10 participant’s models by comparing the evaluation of their performance on synthetic and private validation datasets. Several models trained on synthetic data were found to perform significantly poorer when applied to the non-synthetic private dataset. The main contribution of this research is to understand the challenges and limitations of applying predictive models trained on synthetic data in real-world situations. Due to these challenges, the paper recommends model designs that can inform future successful adoption of synthetic data in real-world educational data systems

Educational anomaly analytics : features, methods, and challenges

Anomalies in education affect the personal careers of students and universities' retention rates. Understanding the laws behind educational anomalies promotes the development of individual students and improves the overall quality of education. However, the inaccessibility of educational data hinders the development of the field. Previous research in this field used questionnaires, which are time- and cost-consuming and hardly applicable to large-scale student cohorts. With the popularity of educational management systems and the rise of online education during the prevalence of COVID-19, a large amount of educational data is available online and offline, providing an unprecedented opportunity to explore educational anomalies from a data-driven perspective. As an emerging field, educational anomaly analytics rapidly attracts scholars from a variety of fields, including education, psychology, sociology, and computer science. This paper intends to provide a comprehensive review of data-driven analytics of educational anomalies from a methodological standpoint. We focus on the following five types of research that received the most attention: course failure prediction, dropout prediction, mental health problems detection, prediction of difficulty in graduation, and prediction of difficulty in employment. Then, we discuss the challenges of current related research. This study aims to provide references for educational policymaking while promoting the development of educational anomaly analytics as a growing field. Copyright © 2022 Guo, Bai, Tian, Firmin and Xia

Qualitative Analyse und Modellierung des wissenschaftlichen Arbeitens

Diese Masterarbeitet bietet einen Überblick der bestehenden Literatur zum Stand der Digitalisierung des geisteswissenschaflichen Arbeitens und den Stellenwert des Exzerpierens und Notierens während der Forschung. Die Erkentnisse aus der Literatur werden durch eine Interviewreihe, ausgewertet auf Basis der Grounded Theory, bestätigt. Basierend auf elf Interviews mit Promovierenden und Masterstudierenden wird ein informelles Aktitivätenmodell des (geistes)wissenschafltichen Arbeitens erstellt. Unter Miteinbeziehung des Forschungsstands auf dem Gebiet des Personal Information Management wird anschließend ein Concurrent Task Tree Modell für digitale Assistenz im Rahmen geisteswissenschaftlicher Forschung vorgestellt. Basierend darauf wurde ein Prototyp zur Evaluierung einer stillen Ausführungs- und Übersetzungsassistenz entwickelt, der im Labor getestet wurde. Die Nutzung des Prototypen führte entgegen der Erwartung zu keiner Effizienzsteigerung beim Zusammenfassen einer Textquelle. Gleichzeitig konnet aber bestätigt werden, dass die Nutzung eines Eye-Trackers und einer Webcam die Verortung von Papiernotizen im digitalen Quelltext ermöglicht. Bei die Auswertung der Interviews wurden zudem zwei Typen der Literaturverwaltung beobachtet, die den Stellenwert von Exzerpten unterstreichen und die zukünftige Entwicklung von Literaturverwaltungssoftware für Geisteswissenschaftler beeinflussen sollten

Score Reporting Research and Applications

Score reporting research is no longer limited to the psychometric properties of scores and subscores. Today, it encompasses design and evaluation for particular audiences, appropriate use of assessment outcomes, the utility and cognitive affordances of graphical representations, interactive report systems, and more. By studying how audiences understand the intended messages conveyed by score reports, researchers and industry professionals can develop more effective mechanisms for interpreting and using assessment data. Score Reporting Research and Applications brings together experts who design and evaluate score reports in both K-12 and higher education contexts and who conduct foundational research in related areas. The first section covers foundational validity issues in the use and interpretation of test scores; design principles drawn from related areas including cognitive science, human-computer interaction, and data visualization; and research on presenting specific types of assessment information to various audiences. The second section presents real-world applications of score report design and evaluation and of the presentation of assessment information. Across ten chapters, this volume offers a comprehensive overview of new techniques and possibilities in score reporting

Score Reporting Research and Applications

Score reporting research is no longer limited to the psychometric properties of scores and subscores. Today, it encompasses design and evaluation for particular audiences, appropriate use of assessment outcomes, the utility and cognitive affordances of graphical representations, interactive report systems, and more. By studying how audiences understand the intended messages conveyed by score reports, researchers and industry professionals can develop more effective mechanisms for interpreting and using assessment data. Score Reporting Research and Applications brings together experts who design and evaluate score reports in both K-12 and higher education contexts and who conduct foundational research in related areas. The first section covers foundational validity issues in the use and interpretation of test scores; design principles drawn from related areas including cognitive science, human-computer interaction, and data visualization; and research on presenting specific types of assessment information to various audiences. The second section presents real-world applications of score report design and evaluation and of the presentation of assessment information. Across ten chapters, this volume offers a comprehensive overview of new techniques and possibilities in score reporting