16 research outputs found
Predictive models as early warning systems for student academic performance in introductory programming
Computer programming is fundamental to Computer Science and IT curricula. At the novice level it covers programming concepts that are essential for subsequent advanced programming courses. However, introductory programming courses are among the most challenging courses for novices and high failure and attrition rates continue even as computer science education has seen improvements in pedagogy. Consequently, the quest to identify factors that affect student learning and academic performance in introductory computer programming courses has been a long-standing activity. Specifically, weak novice learners of programming need to be identified and assisted early in the semester in order to alleviate any potential risk of failing or withdrawing from their course. Hence, it is essential to identify at-risk programming students early, in order to plan (early) interventions.
The goal of this thesis was to develop a validated, predictive model(s) with suitable predictors of student academic performance in introductory programming courses. The proposed model utilises the NaĂŻve Bayes classification machine learning algorithm to analyse student performance data, based on the principle of parsimony. Furthermore, an additional objective was to propose this validated predictive model as an early warning system (EWS), to predict at-risk students early in the semester and, in turn, to potentially inform instructors (and students) for early interventions.
We obtained data from two introductory programming courses in our study to develop and test the predictive models. The models were built with student presage and in progress-data for which instructors may easily collect or access despite the nature of pedagogy of educational settings. In addition, our work analysed the predictability of selected data sources and looked for the combination of predictors, which yields the highest prediction accuracy to predict student academic performance. The prediction accuracies of the models were computed by using confusion matrix data including overall model prediction accuracy, prediction accuracy sensitivity and specificity, balanced accuracy and the area under the ROC curve (AUC) score for generalisation.
On average, the models developed with formative assessment tasks, which were partially assisted by the instructor in the classroom, returned higher at-risk prediction accuracies than the models developed with take-home assessment task only as predictors. The unknown data test results of this study showed that it is possible to predict 83% of students that need support as early as Week 3 in a 12-week introductory programming course. The ensemble method-based results suggest that it is possible to improve overall at-risk prediction performance with low false positives and to incorporate this in early warning systems to identify students that need support, in order to provide early intervention before they reach critical stages (at-risk of failing).
The proposed model(s) of this study were developed on the basis of the principle of parsimony as well as previous research findings, which accounted for variations in academic settings, such as academic environment, and student demography. The predictive model could potentially provide early warning indicators to facilitate early warning intervention strategies for at-risk students in programming that allow for early interventions. The main contribution of this thesis is a model that may be applied to other programming and non-programming courses, which have both continuous formative and a final exam summative assessment, to predict final student performance early in the semester.Ohjelmointi on informaatioteknologian ja tietojenkÀsittelytieteen opinto-ohjelmien olennainen osa. Aloittelijatasolla opetus kattaa jatkokurssien kannalta keskeisiÀ ohjelmoinnin kÀsitteitÀ. TÀstÀ huolimatta ohjelmoinnin peruskurssit ovat erÀitÀ haasteellisimmista kursseista aloittelijoille. Korkea keskeyttÀmisprosentti ja opiskelijoiden asteittainen pois jÀttÀytyminen ovat vielÀkin tunnusomaisia piirteitÀ nÀille kursseille, vaikka ohjelmoinnin opetuksen pedagogiikka onkin kehittynyt. NÀin ollen vaikuttavia syitÀ opiskelijoiden heikkoon suoriutumiseen on etsitty jo pitkÀÀn. Erityisesti heikot, aloittelevat ohjelmoijat tulisi tunnistaa mahdollisimman pian, jotta heille voitaisiin tarjota tukea ja pienentÀÀ opiskelijan riskiÀ epÀonnistua kurssin lÀpÀimisessÀ ja riskiÀ jÀttÀÀ kurssi kesken. Heikkojen opiskelijoiden tunnistaminen on tÀrkeÀÀ, jotta voidaan suunnitella aikainen vÀliintulo.
TÀmÀn vÀitöskirjatyön tarkoituksena oli kehittÀÀ todennettu, ennustava malli tai malleja sopivilla ennnustusfunktioilla koskien opiskelijan akateemista suoriutumista ohjelmoinnin peruskursseilla. Kehitetty malli kÀyttÀÀ koneoppivaa naiivia bayesilaista luokittelualgoritmia analysoimaan opiskelijoiden suoriutumisesta kertynyttÀ aineistoa. LÀhestymistapa perustuu yksinkertaisimpien mahdollisten selittÀvien mallien periaatteeseen. LisÀksi, tavoitteena oli ehdottaa tÀtÀ validoitua ennustavaa mallia varhaiseksi varoitusjÀrjestelmÀksi, jolla ennustetaan putoamisvaarassa olevat opiskelijat opintojakson alkuvaiheessa sekÀ informoidaan ohjaajia (ja opiskelijaa) aikaisen vÀliintulon tarpeellisuudesta.
KerÀsimme aineistoa kahdelta ohjelmoinnin peruskurssilta, jonka pohjalta ennustavaa mallia kehitettiin ja testattiin. Mallit on rakennettu opiskelijoiden ennakkotietojen ja kurssin kestÀessÀ kerÀttyjen suoriutumistietojen perusteella, joita ohjaajat voivat helposti kerÀtÀ tai joihin he voivat pÀÀstÀ kÀsiksi oppilaitoksesta tai muusta ympÀristöstÀ huolimatta. LisÀksi vÀitöskirjatyö analysoi valittujen datalÀhteiden ennustettavuutta ja sitÀ, mitkÀ mallien muuttujista ja niiden kombinaatioista tuottivat kannaltamme korkeimman ennustetarkkuuden opiskelijoiden akateemisessa suoriutumisessa. Mallien ennustusten tarkkuuksia laskettiin kÀyttÀmÀllÀ sekaannusmatriisia, josta saadaan laskettua ennusteen tarkkuus, ennusteen spesifisyys, sensitiivisyys, tasapainotettu tarkkuus sekÀ luokitteluvastekÀyriÀ (receiver operating characteristics (ROC)) ja nÀiden luokitteluvastepinta-ala (area under curve (AUC))
Mallit, jotka kehitettiin formatiivisilla tehtÀvillÀ, ja joissa ohjaaja saattoi osittain auttaa luokkahuonetilanteessa, antoivat keskimÀÀrin tarkemman ennustuksen putoamisvaarassa olevista opiskelijoista kuin mallit, joissa kÀytettiin kotiin vietÀviÀ tehtÀviÀ ainoina ennusteina. Tuntemattomalla testiaineistolla tehdyt mallinnukset osoittavat, ettÀ voimme tunnistaa jo 3. viikon kohdalla 83% niistÀ opiskelijoista, jotka tarvitsevat lisÀtukea 12 viikkoa kestÀvÀllÀ ohjelmoinnin kurssilla. Tulosten perusteella vaikuttaisi, ettÀ yhdistÀmÀllÀ metodeja voidaan saavuttaa parempi yleinen ennustettavuus putoamisvaarassa olevien opiskelijoiden suhteen pienemmÀllÀ mÀÀrÀllÀ vÀÀrin luokiteltuja epÀtositapauksia. Tulokset viittaavat myös siihen, ettÀ on mahdollista sisÀllyttÀÀ yhdistelmÀmalli varoitusjÀrjestelmiin, jotta voidaan tunnistaa avuntarpeessa olevia opiskelijoita ja tarjota tÀten varhaisessa vaiheessa tukea ennen kuin on liian myöhÀistÀ.
TÀssÀ tutkimuksessa esitellyt mallit on kehitetty nojautuen yksinkertaisimman selittÀvÀn mallin periaatteeseen ja myös aiempiin tutkimustuloksiin, joissa huomioidaan erilaiset akateemiset ympÀristöt ja opiskelijoiden tausta. Ennustava malli voi tarjota indikaattoreita, jotka voivat mahdollisesti toimia pohjana vÀliintulostrategioihin kurssilta putoamisvaarassa olevien opiskelijoiden tukemiseksi. TÀmÀn tutkimuksen keskeisin anti on malli, jolla opiskelijoiden suoriutumista voidaan arvioida muilla ohjelmointia ja muita aihepiirejÀ kÀsittelevillÀ kursseilla, jotka sisÀltÀvÀt sekÀ jatkuvaa arviointia ettÀ loppukokeen. Malli ennustaisi nÀillÀ kursseilla lopullisen opiskelijan suoritustason opetusjakson alkuvaiheessa
Cultural Issues That Affect Computer Programming: A Study of Vietnamese in Higher Education
Every society has its own cultural system which ultimately permeates and influences their educational system. Cultural attitudes impact the way students learn and participate in education. A few social practices or culture values may affect student engagement, learning process and learning experience in computer programming education. The purpose of this research was to investigate the impact of students’ cultural attitudes on learning programming courses in Vietnam. To identify this, a questionnaire was designed and distributed to Vietnamese undergraduate IT students in order to measure the relationship between cultural attitudes and student engagement in learning. This study used Geert Hofstede’s defined cultural dimensions of power distance, and collectivism versus individualism as combined factors that affect student participation and engagement. The survey results were analyzed by the correlation coefficient statistical method to examine the statistical relationships that exist between student’s power dependency culture and participation in learning. The survey results confirmed that many students completed their secondary education in a teacher centered environment and students are hesitant to express their opinions to teachers, and prefer to use existing solutions to complete their work. Moreover, statistical results partially supported the conclusion that, student’s high power distance culture may affect their engagement in learning programming courses though many students are interested in active learning environments.</p
Formative assessment tasks as indicators of student engagement for predicting at-risk students in programming courses
Previous studies have proposed many indicators to assess the effect of student engagement in learning and academic achievement but have not yet been clearly articulated. In addition, while student engagement tracking systems have been designed, they rely on the log data but not on performance data. This paper presents results of a non-machine learning model developed using ongoing formative assessment scores as indicators of student engagement. Visualisation of the classification tree results is employed as student engagement indicators for instructors to observe and intervene with students. The results of this study showed that ongoing assessment is related to student engagement and subsequent final programming exam performance and possible to identify students at-risk of failing the final exam. Finally, our study identified students impacted by the Covid-19 pandemic. These were students who attended the final programming exam in the semester 2019-2020 and who scored well in formative assessments. Based on these results we present a simple student engagement indicator and its potential application as a student progress monitor for early identification of students at risk.</p
Prediction of Student Final Exam Performance in an Introductory Programming Course: Development and Validation of the Use of a Support Vector Machine-Regression Model
This paper presents a Support Vector Machine predictive model to determine if prior programming knowledge and completion of in-class and take home formative assessment tasks might be suitable predictors of examination performance. Student data from the academic years 2012 - 2016 for an introductory programming course was captured via ViLLE e-learning tool for analysis. The results revealed that student prior programming knowledge and assessment scores captured in a predictive model, is a good fit of the data. However, while overall success of the model is significant, predictions on identifying at-risk students is neither high nor low and that persuaded us to include two more research questions. However, our preliminary post analysis on these test results show that on average students who secured less than 70% in formative assessment scores with little or basic prior programming knowledge in programming may fail in the final programming exam and increase the prediction accuracy in identifying at-risk students from 46% to nearly 63%. Hence, these results provide immediate information for programming course instructors and students to enhance teaching and learning process.</p
Relationship between perceived problem-solving skills and academic performance of novice learners in introductory programming courses
Past research has shown that student problemâsolving skills may be used to determine student final exam performance. This study reports on the relationship between student perceived problemâsolving skills and academic performance in introductory programming, in formative and summative programming assessment tasks. We found that the more effective problem solvers achieved better final exam scores. There was no significant difference in formative assessment performances between effective and poor problem solvers. It is also possible to categorize students on the basis of problemâsolving skills, in order to exploit opportunities to improve learning around constructivist learning theory. Finally, our study identified transferability skills and the study may be extended to identify the impact of problem solving transfer skills on student problem solving for programming.</p
The Impact of Lecture Attendance on Exams for Novice Programming Students
In this paper, the correlation between lecture attendance and assessment tasks on final exam performance of introductory programming students has been analyzed to identify if lecture attendance, and completion of in-class and take home formative assessment tasks affects student performance in the final examination. In this study, only lecture attendance, homework exercises and class demonstration scores, and final exam marks have been considered. This study used Spearman's Rank correlation coefficient and multiple regression techniques via SPSS software to analyze the student data of the academic years 2012, 2013 and 2014 of an introductory programming course to test the hypotheses. It is found that, there is a significant correlation between homework exercises and final exam scores. However, formal lecture attendance and final exam performance were negatively correlated. Moreover, multiple regression results of assessment tasks such as homework exercises, class activities and lecture attendance on final exam scores, did not provide any significant value to support the statement "Marks achieved in homework, class demonstrations, and lecture attendance, have a significant positive impact on final examination results".</p
Reducing the environmental impact of surgery on a global scale: systematic review and co-prioritization with healthcare workers in 132 countries
Abstract
Background
Healthcare cannot achieve net-zero carbon without addressing operating theatres. The aim of this study was to prioritize feasible interventions to reduce the environmental impact of operating theatres.
Methods
This study adopted a four-phase Delphi consensus co-prioritization methodology. In phase 1, a systematic review of published interventions and global consultation of perioperative healthcare professionals were used to longlist interventions. In phase 2, iterative thematic analysis consolidated comparable interventions into a shortlist. In phase 3, the shortlist was co-prioritized based on patient and clinician views on acceptability, feasibility, and safety. In phase 4, ranked lists of interventions were presented by their relevance to high-income countries and lowâmiddle-income countries.
Results
In phase 1, 43 interventions were identified, which had low uptake in practice according to 3042 professionals globally. In phase 2, a shortlist of 15 intervention domains was generated. In phase 3, interventions were deemed acceptable for more than 90 per cent of patients except for reducing general anaesthesia (84 per cent) and re-sterilization of âsingle-useâ consumables (86 per cent). In phase 4, the top three shortlisted interventions for high-income countries were: introducing recycling; reducing use of anaesthetic gases; and appropriate clinical waste processing. In phase 4, the top three shortlisted interventions for lowâmiddle-income countries were: introducing reusable surgical devices; reducing use of consumables; and reducing the use of general anaesthesia.
Conclusion
This is a step toward environmentally sustainable operating environments with actionable interventions applicable to both highâ and lowâmiddleâincome countries
Identifying Novice Student Programming Misconceptions and Errors From Summative Assessments
Intelligent Computing: Proceedings of the 2021 Computing Conference, Volume 2
The pursuit of a deeper understanding of factors that influence student performance outcomes has long been of interest to the computing education community. Among these include the development of effective predictive models to predict student academic performance. Predictive models may serve as earlywarning systems to identify students at risk of failing or quitting early. This paper presents a class of machine learning predictive models based on Naive Bayes classification, to predict student performance in introductory programming. The models use formative assessment tasks and self-reported cognitive features such as prior programming knowledge and problem-solving skills. Our analysis revealed that the use of just three variables was a good fit for the models employed. The models that used in-class assessment and cognitive features as predictors returned best at-risk prediction accuracies, compared with models that used take-home assessment and cognitive features as predictors. The prediction accuracy in identifyingat-risk students on unknown data for the coursewas 71% (overall prediction accuracy) in compliance with the area under the curve (ROC) score (0.66). Based on these results we present a generic predictive model and its potential application as an early warning system for early identification of students at risk.</p