Predicting student performance in interactive online question pools using mouse interaction features

Modeling student learning and further predicting the performance is a well-established task in online learning and is crucial to personalized education by recommending different learning resources to different students based on their needs. Interactive online question pools (e.g., educational game platforms), an important component of online education, have become increasingly popular in recent years. However, most existing work on student performance prediction targets at online learning platforms with a well-structured curriculum, predefined question order and accurate knowledge tags provided by domain experts. It remains unclear how to conduct student performance prediction in interactive online question pools without such well-organized question orders or knowledge tags by experts. In this paper, we propose a novel approach to boost student performance prediction in interactive online question pools by further considering student interaction features and the similarity between questions. Specifically, we introduce new features (e.g., think time, first attempt, and first drag-and-drop) based on student mouse movement trajectories to delineate students' problem-solving details. In addition, heterogeneous information network is applied to integrating students' historical problem-solving information on similar questions, enhancing student performance predictions on a new question. We evaluate the proposed approach on the dataset from a real-world interactive question pool using four typical machine learning models.Comment: 10 pages, 7 figures, conference lak20, has been accepted, proceeding now. link: https://lak20.solaresearch.org/list-of-accepted-paper

Peer-inspired Student Performance Prediction in Interactive Online Question Pools with Graph Neural Network

Student performance prediction is critical to online education. It can benefit many downstream tasks on online learning platforms, such as estimating dropout rates, facilitating strategic intervention, and enabling adaptive online learning. Interactive online question pools provide students with interesting interactive questions to practice their knowledge in online education. However, little research has been done on student performance prediction in interactive online question pools. Existing work on student performance prediction targets at online learning platforms with predefined course curriculum and accurate knowledge labels like MOOC platforms, but they are not able to fully model knowledge evolution of students in interactive online question pools. In this paper, we propose a novel approach using Graph Neural Networks (GNNs) to achieve better student performance prediction in interactive online question pools. Specifically, we model the relationship between students and questions using student interactions to construct the student-interaction-question network and further present a new GNN model, called R^2GCN, which intrinsically works for the heterogeneous networks, to achieve generalizable student performance prediction in interactive online question pools. We evaluate the effectiveness of our approach on a real-world dataset consisting of 104,113 mouse trajectories generated in the problem-solving process of over 4000 students on 1631 questions. The experiment results show that our approach can achieve a much higher accuracy of student performance prediction than both traditional machine learning approaches and GNN models.Comment: 8 pages, 8 figures. Accepted at CIKM 202

Peer-inspired student performance prediction in interactive online question pools with graph neural network

Student performance prediction is critical to online education. It can benefit many downstream tasks on online learning platforms, such as estimating dropout rates, facilitating strategic intervention, and enabling adaptive online learning. Interactive online question pools provide students with interesting interactive questions to practice their knowledge in online education. However, little research has been done on student performance prediction in interactive online question pools. Existing work on student performance prediction targets at online learning platforms with predefined course curriculum and accurate knowledge labels like MOOC platforms, but they are not able to fully model knowledge evolution of students in interactive online question pools. In this paper, we propose a novel approach using Graph Neural Networks (GNNs) to achieve better student performance prediction in interactive online question pools. Specifically, we model the relationship between students and questions using student interactions to construct the student-interaction-question network and further present a new GNN model, called R^2GCN, which intrinsically works for the heterogeneous networks, to achieve generalizable student performance prediction in interactive online question pools. We evaluate the effectiveness of our approach on a real-world dataset consisting of 104,113 mouse trajectories generated in the problem-solving process of over 4000 students on 1631 questions. The experiment results show that our approach can achieve a much higher accuracy of student performance prediction than both traditional machine learning approaches and GNN models.Comment: 8 pages, 8 figures. Accepted at CIKM 202

QLens: Visual analytics of multi-step problem-solving behaviors for improving question design

With the rapid development of online education in recent years, there has been an increasing number of learning platforms that provide students with multi-step questions to cultivate their problem-solving skills. To guarantee the high quality of such learning materials, question designers need to inspect how students' problem-solving processes unfold step by step to infer whether students' problem-solving logic matches their design intent. They also need to compare the behaviors of different groups (e.g., students from different grades) to distribute questions to students with the right level of knowledge. The availability of fine-grained interaction data, such as mouse movement trajectories from the online platforms, provides the opportunity to analyze problem-solving behaviors. However, it is still challenging to interpret, summarize, and compare the high dimensional problem-solving sequence data. In this paper, we present a visual analytics system, QLens, to help question designers inspect detailed problem-solving trajectories, compare different student groups, distill insights for design improvements. In particular, QLens models problem-solving behavior as a hybrid state transition graph and visualizes it through a novel glyph-embedded Sankey diagram, which reflects students' problem-solving logic, engagement, and encountered difficulties. We conduct three case studies and three expert interviews to demonstrate the usefulness of QLens on real-world datasets that consist of thousands of problem-solving traces

An Investigation of the Correlation Between Mental Workload and Web User’s Interaction

Mental Workload, a psychological concept, was identified as being linked with task’s and system’s performance. In the context of Human-Computer Interaction, recent research has identified Mental Workload as an important measure in the designing and evaluation of web interfaces, and as an additional and supplemental insight to typical usability evaluation methods. Simultaneously, web logs containing data related to web users’ interaction (e.g. scrolling; mouse clicks) have been proved useful in evaluating the usability of web sites by analysing the data tracked for hundreds of users. In order to study if the potential of logs of user interaction can be applied in the study of Mental Workload in Web design, an online experiment with 145 participants was performed. Additionally, the experiment, composed of alternative interfaces, sought to assess the role of Mental Workload in the evaluation of interfaces using interactive Infographics, which were identified by literature as bringing new challenges and concerns in the field of Web Design. The online experiment’s results suggested that correlations between mental demands and users’ interaction can only be observed when taking in consideration the web interface used or the profile of the users. Moreover, the used measurement methods for assessing Mental Workload were not capable of predicting task performance, as previous research suggested (in the context of other types of web interfaces)

An investigation of the correlation between Mental Workload and Web User’s Interaction

Mental Workload, a Psychology concept, was identified as being linked with task’s and system’s performance. In the context of Human-Computer Interaction, recent research has identified Mental Workload as an important measure in the designing and evaluation of web interfaces, and as an additional and supplemental insight to typical Usability evaluation methods. Simultaneously, web logs containing data related to web users’ interaction (e.g. scrolling; mouse clicks) have been proved useful in evaluating the Usability of web sites by levering the data tracked for hundreds of users. In order to study if the potential of logs of user interaction can be applied in the study of Mental Workload in Web design, an online experiment with 145 participations was performed. Additionally, the experiment, composed of alternative interfaces, sought to assess the role of Mental Workload in the evaluation of interfaces using interactive Infographics, which were identified by literature as bringing new challenges and concerns in the field of Web Design. The online experiment’s results suggested that correlations between mental demands and users’ interaction can only be observed when taking in consideration the web interface used or the profile of the users. Moreover, the used measurement methods for assessing Mental Workload were not capable of predicting task performance, as previous research suggested (in the context of other types of web interfaces)

Enhancing Student Performance Prediction on Learnersourced Questions with SGNN-LLM Synergy

As an emerging education strategy, learnersourcing offers the potential for personalized learning content creation, but also grapples with the challenge of predicting student performance due to inherent noise in student-generated data. While graph-based methods excel in capturing dense learner-question interactions, they falter in cold start scenarios, characterized by limited interactions, as seen when questions lack substantial learner responses. In response, we introduce an innovative strategy that synergizes the potential of integrating Signed Graph Neural Networks (SGNNs) and Large Language Model (LLM) embeddings. Our methodology employs a signed bipartite graph to comprehensively model student answers, complemented by a contrastive learning framework that enhances noise resilience. Furthermore, LLM's contribution lies in generating foundational question embeddings, proving especially advantageous in addressing cold start scenarios characterized by limited graph data interactions. Validation across five real-world datasets sourced from the PeerWise platform underscores our approach's effectiveness. Our method outperforms baselines, showcasing enhanced predictive accuracy and robustness

PREDIKSI TINGKAT PEMAHAMAN MATERI PESERTA DIDIK DALAM PEMBELAJARAN ONLINE MENGGUNAKAN ALGORITMA SUPPORT VECTOR MACHINE (SVM)

Penelitian ini bertujuan untuk memprediksi tingkat pemahaman materi peserta didik dalam pembelajaran online menggunakan algoritma Support Vector Machine (SVM). Algoritma ini digunakan agar mendapatkan penilaian terhadap pemahaman materi peserta didik yang lebih akurat dan objektif berdasarkan aktivitas peserta didik selama proses pembelajaran online. Penelitian ini menggunakan pendekatan eksperimen untuk mengimplementasikan algoritma Support Vector Machine (SVM), sehingga memiliki kinerja yang optimal dalam melakukan prediksi, menggunakan data dari Kaggle. Tahapan implementasi ini diantaranya data preprocessing yang terdiri dari Encoding Categorical dan normalisasi, kemudian proses pemodelan yaitu proses pelatihan dan pengujian model. Hasil implementasi algoritma Support Vector Machine (SVM) didapatkan akurasi sebesar 88.5% menggunakan parameter C = 24 dan gamma = 0.21, selain itu F1-Score yang didapatkan juga menunjukan hasil yang baik diantaranya kelas Low sebesar 88%, kelas Mid sebesar 87% dan kelas High sebesar 91%. Kemudian, fitur yang memiliki pengaruh sangat besar terhadap hasil prediksi tingkat pemahaman materi pembelajaran dalam pembelajaran online adalah fitur ketidakhadiran. This study aims to predict understanding level of students in online learning using the Support Vector Machine (SVM) algorithm. This algorithm is used in order to get an assessment understanding level of students that is more accurate and objective based on students activities during the online learning process. This study uses an experimental approach to implement the Support Vector Machine (SVM) algorithm, so that it has optimal performance in making predictions, using data from Kaggle. These stages of implementation include preprocessing data which consists of Encoding Categorical and Normalization, then the modeling process is the process of training and testing the model. The results of the implementation of the Support Vector Machine (SVM) algorithm obtained an accuracy of 88.5% using the parameters C = 24 and gamma = 0.21, besides that the F1-Score obtained also showed good results including the Low class 88%, the Mid class 87% and the High class 91%. Then, the feature that has a very big influence on the results of the prediction understanding level of students in online learning is the absence feature