Psychometrics in Practice at RCEC

A broad range of topics is dealt with in this volume: from combining the psychometric generalizability and item response theories to the ideas for an integrated formative use of data-driven decision making, assessment for learning and diagnostic testing. A number of chapters pay attention to computerized (adaptive) and classification testing. Other chapters treat the quality of testing in a general sense, but for topics like maintaining standards or the testing of writing ability, the quality of testing is dealt with more specifically.\ud All authors are connected to RCEC as researchers. They present one of their current research topics and provide some insight into the focus of RCEC. The selection of the topics and the editing intends that the book should be of special interest to educational researchers, psychometricians and practitioners in educational assessment

Implementing Machine Learning Techniques for Predicting Student Performance in an E-Learning Environment

The pandemic of COVID-19 has altered the way people learn. Learning has moved from offline to online throughout this pandemic. Predicting student performance based on relevant data has opened up a new field for educational institutions to improve teaching and learning processes, as well as course curriculum adjustments. Machine learning technology can assist universities in forecasting student performance so that necessary changes in lecture delivery and curriculum can be made. The performance of the pupils was predicted using machine learning techniques in this research. Open University (OU) educational data is examined. Demographic, engagement, and performance metrics are used. The results of the experiment. The k-NN strategy outperformed all other algorithms on the OU dataset in some circumstances, but the ANN approach outperformed them all in others

Student Attrition Prediction Using Machine Learning Techniques

In educational systems, students’ course enrollment is fundamental performance metrics to academic and financial sustainability. In many higher institutions today, students’ attrition rates are caused by a variety of circumstances, including demographic and personal factors such as age, gender, academic background, financial abilities, and academic degree of choice. In this study, machine learning approaches was used to develop prediction models that predicted students’ attrition rate in pursuing computer science degree, as well as students who have a high risk of dropping out before graduation. This can help higher education institutes to develop proper intervention plans to reduce attrition rates and increase the probability of student academic success. Student’s data were collected from the Federal University Lokoja (FUL), Nigeria. The data were preprocessed using existing weka machine learning libraries where the data was converted into attribute related file form (arff) and resampling techniques was used to partition the data into training set and testing set. The correlation-based feature selection was extracted and used to develop the students’ attrition model and to identify the students’ risk of dropping out. Random forest and random tree machine learning algorithms were used to predict students' attrition. The results showed that the random forest had an accuracy of 79.45%, while the random tree's accuracy was 78.09%. This is an improvement over previous results where 66.14% and 57.48% accuracy was recorded for random forest and random tree respectively. This improvement was as a result of the techniques used. It is therefore recommended that applying techniques to the classification model can improve the performance of the model

Design and Implementation of Real-time Student Performance Evaluation and Feedback System

Undergraduate education is challenged by high dropout rates and by delayed student graduation due to dropping courses or having to repeat courses due to low academic performance. In this context, an early prediction of student-performance may help students to understand where they stand amongst their peers and to change the attitude with about the course they are taking. Moreover, it is important to identify students in time who need special attention and providing appropriate interventions, such as mentoring and conducting review sessions. The goal of this thesis is the design and implementation of real-time student-performance evaluation and feedback system (RSPEF) to improve graduation rates. RSPEF is an interactive, web-based system consisting of a Predictive Analysis System (PAS) that uses machine-learning techniques to interpolate past student-performance into future, and the development of an Emergency Warning System (EWS) that identifies poor-performing students in courses. Moreover, a unified representation of student-background and student-performance data is provided in form of a relational database schema that is suitable to be used to assess student’s performance across multiple courses, which is critical for the generalizability of RSPEF system. The system design includes core machine-learning & data-analysis engine, a relational database that is reusable across courses and an interactive web-based interface to continuously collect data and create dashboards for users.Computer Science, Department o

La détection d'anomalies comme outil de renforcement d'analyse des données et de prédiction dans l'éducation

Les établissements d'enseignement cherchent à concevoir des mécanismes efficaces pour améliorer les résultats scolaires, renforcer le processus d'apprentissage et éviter l'abandon scolaire. L'analyse et la prédiction des performances des étudiants au cours de leurs études peuvent mettre en évidence certaines lacunes d'une formation et détecter les étudiants ayant des problèmes d'apprentissage. Il s'agit donc de développer des techniques et des modèles basés sur des données qui visent à améliorer l'enseignement et l'apprentissage. Les modèles classiques ignorent généralement les étudiants présentant des comportements et incohérences inhabituels, bien qu'ils puissent fournir des informations importantes aux experts du domaine et améliorer les modèles de prédiction. Les profils atypiques dans l'éducation sont à peine explorés et leur impact sur les modèles de prédiction n'a pas encore été étudié dans la littérature. Cette thèse vise donc à étudier les valeurs anormales dans les données éducatives et à étendre les connaissances existantes à leur sujet. La thèse présente trois études de cas de détection de données anormales pour différents contextes éducatifs et modes de représentation des données (jeu de données numériques pour une université allemande, jeu de données numériques pour une université russe, jeu de données séquentiel pour les écoles d'infirmières françaises). Pour chaque cas, l'approche de prétraitement des données est proposée en tenant compte des particularités du jeu de données. Les données préparées ont été utilisées pour détecter les valeurs anormales dans des conditions de vérité terrain inconnue. Les caractéristiques des valeurs anormales détectées ont été explorées et analysées, ce qui a permis d'étendre les connaissances sur le comportement des étudiants dans un processus d'apprentissage. L'une des principales tâches dans le domaine de l'éducation est de développer des mécanismes essentiels qui permettront d'améliorer les résultats scolaires et de réduire l'abandon scolaire. Ainsi, il est nécessaire de construire des modèles de prédiction de performance qui sont capables de détecter les étudiants ayant des problèmes d'apprentissage, qui ont besoin d'une aide spéciale. Le deuxième objectif de la thèse est d'étudier l'impact des valeurs anormales sur les modèles de prédiction. Nous avons considéré deux des tâches de prédiction les plus courantes dans le domaine de l'éducation: (i) la prédiction de l'abandon scolaire, (ii) la prédiction du score final. Les modèles de prédiction ont été comparés en fonction de différents algorithmes de prédiction et de la présence de valeurs anormales dans les données d'entraînement. Cette thèse ouvre de nouvelles voies pour étudier les performances des élèves dans les environnements éducatifs. La compréhension des valeurs anormales et des raisons de leur apparition peut aider les experts du domaine à extraire des informations précieuses des données. La détection des valeurs aberrantes pourrait faire partie du pipeline des systèmes d'alerte précoce pour détecter les élèves à haut risque d'abandon. De plus, les tendances comportementales des valeurs aberrantes peuvent servir de base pour fournir des recommandations aux étudiants dans leurs études ou prendre des décisions concernant l'amélioration du processus éducatif.Educational institutions seek to design effective mechanisms that improve academic results, enhance the learning process, and avoid dropout. The performance analysis and performance prediction of students in their studies may show drawbacks in the educational formations and detect students with learning problems. This induces the task of developing techniques and data-based models which aim to enhance teaching and learning. Classical models usually ignore the students-outliers with uncommon and inconsistent characteristics although they may show significant information to domain experts and affect the prediction models. The outliers in education are barely explored and their impact on the prediction models has not been studied yet in the literature. Thus, the thesis aims to investigate the outliers in educational data and extend the existing knowledge about them. The thesis presents three case studies of outlier detection for different educational contexts and ways of data representation (numerical dataset for the German University, numerical dataset for the Russian University, sequential dataset for French nurse schools). For each case, the data preprocessing approach is proposed regarding the dataset peculiarities. The prepared data has been used to detect outliers in conditions of unknown ground truth. The characteristics of detected outliers have been explored and analysed, which allowed extending the comprehension of students' behaviour in a learning process. One of the main tasks in the educational domain is to develop essential tools which will help to improve academic results and reduce attrition. Thus, plenty of studies aim to build models of performance prediction which can detect students with learning problems that need special help. The second goal of the thesis is to study the impact of outliers on prediction models. The two most common prediction tasks in the educational field have been considered: (i) dropout prediction, (ii) the final score prediction. The prediction models have been compared in terms of different prediction algorithms and the presence of outliers in the training data. This thesis opens new avenues to investigate the students' performance in educational environments. The understanding of outliers and the reasons for their appearance can help domain experts to extract valuable information from the data. Outlier detection might be a part of the pipeline in the early warning systems of detecting students with a high risk of dropouts. Furthermore, the behavioral tendencies of outliers can serve as a basis for providing recommendations for students in their studies or making decisions about improving the educational process

Predictive models as early warning systems for student academic performance in introductory programming

Computer programming is fundamental to Computer Science and IT curricula. At the novice level it covers programming concepts that are essential for subsequent advanced programming courses. However, introductory programming courses are among the most challenging courses for novices and high failure and attrition rates continue even as computer science education has seen improvements in pedagogy. Consequently, the quest to identify factors that affect student learning and academic performance in introductory computer programming courses has been a long-standing activity. Specifically, weak novice learners of programming need to be identified and assisted early in the semester in order to alleviate any potential risk of failing or withdrawing from their course. Hence, it is essential to identify at-risk programming students early, in order to plan (early) interventions. The goal of this thesis was to develop a validated, predictive model(s) with suitable predictors of student academic performance in introductory programming courses. The proposed model utilises the Naïve Bayes classification machine learning algorithm to analyse student performance data, based on the principle of parsimony. Furthermore, an additional objective was to propose this validated predictive model as an early warning system (EWS), to predict at-risk students early in the semester and, in turn, to potentially inform instructors (and students) for early interventions. We obtained data from two introductory programming courses in our study to develop and test the predictive models. The models were built with student presage and in progress-data for which instructors may easily collect or access despite the nature of pedagogy of educational settings. In addition, our work analysed the predictability of selected data sources and looked for the combination of predictors, which yields the highest prediction accuracy to predict student academic performance. The prediction accuracies of the models were computed by using confusion matrix data including overall model prediction accuracy, prediction accuracy sensitivity and specificity, balanced accuracy and the area under the ROC curve (AUC) score for generalisation. On average, the models developed with formative assessment tasks, which were partially assisted by the instructor in the classroom, returned higher at-risk prediction accuracies than the models developed with take-home assessment task only as predictors. The unknown data test results of this study showed that it is possible to predict 83% of students that need support as early as Week 3 in a 12-week introductory programming course. The ensemble method-based results suggest that it is possible to improve overall at-risk prediction performance with low false positives and to incorporate this in early warning systems to identify students that need support, in order to provide early intervention before they reach critical stages (at-risk of failing). The proposed model(s) of this study were developed on the basis of the principle of parsimony as well as previous research findings, which accounted for variations in academic settings, such as academic environment, and student demography. The predictive model could potentially provide early warning indicators to facilitate early warning intervention strategies for at-risk students in programming that allow for early interventions. The main contribution of this thesis is a model that may be applied to other programming and non-programming courses, which have both continuous formative and a final exam summative assessment, to predict final student performance early in the semester.Ohjelmointi on informaatioteknologian ja tietojenkäsittelytieteen opinto-ohjelmien olennainen osa. Aloittelijatasolla opetus kattaa jatkokurssien kannalta keskeisiä ohjelmoinnin käsitteitä. Tästä huolimatta ohjelmoinnin peruskurssit ovat eräitä haasteellisimmista kursseista aloittelijoille. Korkea keskeyttämisprosentti ja opiskelijoiden asteittainen pois jättäytyminen ovat vieläkin tunnusomaisia piirteitä näille kursseille, vaikka ohjelmoinnin opetuksen pedagogiikka onkin kehittynyt. Näin ollen vaikuttavia syitä opiskelijoiden heikkoon suoriutumiseen on etsitty jo pitkään. Erityisesti heikot, aloittelevat ohjelmoijat tulisi tunnistaa mahdollisimman pian, jotta heille voitaisiin tarjota tukea ja pienentää opiskelijan riskiä epäonnistua kurssin läpäimisessä ja riskiä jättää kurssi kesken. Heikkojen opiskelijoiden tunnistaminen on tärkeää, jotta voidaan suunnitella aikainen väliintulo. Tämän väitöskirjatyön tarkoituksena oli kehittää todennettu, ennustava malli tai malleja sopivilla ennnustusfunktioilla koskien opiskelijan akateemista suoriutumista ohjelmoinnin peruskursseilla. Kehitetty malli käyttää koneoppivaa naiivia bayesilaista luokittelualgoritmia analysoimaan opiskelijoiden suoriutumisesta kertynyttä aineistoa. Lähestymistapa perustuu yksinkertaisimpien mahdollisten selittävien mallien periaatteeseen. Lisäksi, tavoitteena oli ehdottaa tätä validoitua ennustavaa mallia varhaiseksi varoitusjärjestelmäksi, jolla ennustetaan putoamisvaarassa olevat opiskelijat opintojakson alkuvaiheessa sekä informoidaan ohjaajia (ja opiskelijaa) aikaisen väliintulon tarpeellisuudesta. Keräsimme aineistoa kahdelta ohjelmoinnin peruskurssilta, jonka pohjalta ennustavaa mallia kehitettiin ja testattiin. Mallit on rakennettu opiskelijoiden ennakkotietojen ja kurssin kestäessä kerättyjen suoriutumistietojen perusteella, joita ohjaajat voivat helposti kerätä tai joihin he voivat päästä käsiksi oppilaitoksesta tai muusta ympäristöstä huolimatta. Lisäksi väitöskirjatyö analysoi valittujen datalähteiden ennustettavuutta ja sitä, mitkä mallien muuttujista ja niiden kombinaatioista tuottivat kannaltamme korkeimman ennustetarkkuuden opiskelijoiden akateemisessa suoriutumisessa. Mallien ennustusten tarkkuuksia laskettiin käyttämällä sekaannusmatriisia, josta saadaan laskettua ennusteen tarkkuus, ennusteen spesifisyys, sensitiivisyys, tasapainotettu tarkkuus sekä luokitteluvastekäyriä (receiver operating characteristics (ROC)) ja näiden luokitteluvastepinta-ala (area under curve (AUC)) Mallit, jotka kehitettiin formatiivisilla tehtävillä, ja joissa ohjaaja saattoi osittain auttaa luokkahuonetilanteessa, antoivat keskimäärin tarkemman ennustuksen putoamisvaarassa olevista opiskelijoista kuin mallit, joissa käytettiin kotiin vietäviä tehtäviä ainoina ennusteina. Tuntemattomalla testiaineistolla tehdyt mallinnukset osoittavat, että voimme tunnistaa jo 3. viikon kohdalla 83% niistä opiskelijoista, jotka tarvitsevat lisätukea 12 viikkoa kestävällä ohjelmoinnin kurssilla. Tulosten perusteella vaikuttaisi, että yhdistämällä metodeja voidaan saavuttaa parempi yleinen ennustettavuus putoamisvaarassa olevien opiskelijoiden suhteen pienemmällä määrällä väärin luokiteltuja epätositapauksia. Tulokset viittaavat myös siihen, että on mahdollista sisällyttää yhdistelmämalli varoitusjärjestelmiin, jotta voidaan tunnistaa avuntarpeessa olevia opiskelijoita ja tarjota täten varhaisessa vaiheessa tukea ennen kuin on liian myöhäistä. Tässä tutkimuksessa esitellyt mallit on kehitetty nojautuen yksinkertaisimman selittävän mallin periaatteeseen ja myös aiempiin tutkimustuloksiin, joissa huomioidaan erilaiset akateemiset ympäristöt ja opiskelijoiden tausta. Ennustava malli voi tarjota indikaattoreita, jotka voivat mahdollisesti toimia pohjana väliintulostrategioihin kurssilta putoamisvaarassa olevien opiskelijoiden tukemiseksi. Tämän tutkimuksen keskeisin anti on malli, jolla opiskelijoiden suoriutumista voidaan arvioida muilla ohjelmointia ja muita aihepiirejä käsittelevillä kursseilla, jotka sisältävät sekä jatkuvaa arviointia että loppukokeen. Malli ennustaisi näillä kursseilla lopullisen opiskelijan suoritustason opetusjakson alkuvaiheessa

Case Study: Using Data Mining to Predict Student Performance Based on Demographic Attributes

This study predicts student performance at Universiti Pertahanan Nasional Malaysia (UPNM) based on their socio-demographic profile; it also determines how a prediction algorithm can be used to classify the student data for the most significant demographic attributes. The analytical pattern in academic results per batch has been identified using demographic attributes and the student's grades to improve short-term and long-term learning and teaching plans. Understanding the likely outcome of the education process based on predictions can help UPNM lecturers enhance the achievements of the subsequent batch of students by modifying the factors contributing to the prior success. This study identifies and predicts student performance using data mining and classification techniques such as decision trees, neural networks, and k-nearest neighbors. This frequently adopted method comprises data selection and preparation, cleansing, incorporating previous knowledge datasets, and interpreting precise solutions. This study presents the simplified output from each data mining method to facilitate a better understanding of the result and determine the best data mining method. The results show that the critical attributes influencing student performance are gender, age, and student status. The Neural Networks method has the lowest Root of the Mean of the Square of Errors (RMSE) for accuracy measurement. In contrast, the decision tree method has the highest RMSE, which indicates that the decision tree method has a lower performance accuracy. Moreover, the correlation coefficient for the k-nearest neighbor has been recorded as less than one