Relationship between perceived problem-solving skills and academic performance of novice learners in introductory programming courses

Past research has shown that student problem‐solving skills may be used to determine student final exam performance. This study reports on the relationship between student perceived problem‐solving skills and academic performance in introductory programming, in formative and summative programming assessment tasks. We found that the more effective problem solvers achieved better final exam scores. There was no significant difference in formative assessment performances between effective and poor problem solvers. It is also possible to categorize students on the basis of problem‐solving skills, in order to exploit opportunities to improve learning around constructivist learning theory. Finally, our study identified transferability skills and the study may be extended to identify the impact of problem solving transfer skills on student problem solving for programming.</p

Predictive models as early warning systems for student academic performance in introductory programming

Computer programming is fundamental to Computer Science and IT curricula. At the novice level it covers programming concepts that are essential for subsequent advanced programming courses. However, introductory programming courses are among the most challenging courses for novices and high failure and attrition rates continue even as computer science education has seen improvements in pedagogy. Consequently, the quest to identify factors that affect student learning and academic performance in introductory computer programming courses has been a long-standing activity. Specifically, weak novice learners of programming need to be identified and assisted early in the semester in order to alleviate any potential risk of failing or withdrawing from their course. Hence, it is essential to identify at-risk programming students early, in order to plan (early) interventions. The goal of this thesis was to develop a validated, predictive model(s) with suitable predictors of student academic performance in introductory programming courses. The proposed model utilises the Naïve Bayes classification machine learning algorithm to analyse student performance data, based on the principle of parsimony. Furthermore, an additional objective was to propose this validated predictive model as an early warning system (EWS), to predict at-risk students early in the semester and, in turn, to potentially inform instructors (and students) for early interventions. We obtained data from two introductory programming courses in our study to develop and test the predictive models. The models were built with student presage and in progress-data for which instructors may easily collect or access despite the nature of pedagogy of educational settings. In addition, our work analysed the predictability of selected data sources and looked for the combination of predictors, which yields the highest prediction accuracy to predict student academic performance. The prediction accuracies of the models were computed by using confusion matrix data including overall model prediction accuracy, prediction accuracy sensitivity and specificity, balanced accuracy and the area under the ROC curve (AUC) score for generalisation. On average, the models developed with formative assessment tasks, which were partially assisted by the instructor in the classroom, returned higher at-risk prediction accuracies than the models developed with take-home assessment task only as predictors. The unknown data test results of this study showed that it is possible to predict 83% of students that need support as early as Week 3 in a 12-week introductory programming course. The ensemble method-based results suggest that it is possible to improve overall at-risk prediction performance with low false positives and to incorporate this in early warning systems to identify students that need support, in order to provide early intervention before they reach critical stages (at-risk of failing). The proposed model(s) of this study were developed on the basis of the principle of parsimony as well as previous research findings, which accounted for variations in academic settings, such as academic environment, and student demography. The predictive model could potentially provide early warning indicators to facilitate early warning intervention strategies for at-risk students in programming that allow for early interventions. The main contribution of this thesis is a model that may be applied to other programming and non-programming courses, which have both continuous formative and a final exam summative assessment, to predict final student performance early in the semester.Ohjelmointi on informaatioteknologian ja tietojenkäsittelytieteen opinto-ohjelmien olennainen osa. Aloittelijatasolla opetus kattaa jatkokurssien kannalta keskeisiä ohjelmoinnin käsitteitä. Tästä huolimatta ohjelmoinnin peruskurssit ovat eräitä haasteellisimmista kursseista aloittelijoille. Korkea keskeyttämisprosentti ja opiskelijoiden asteittainen pois jättäytyminen ovat vieläkin tunnusomaisia piirteitä näille kursseille, vaikka ohjelmoinnin opetuksen pedagogiikka onkin kehittynyt. Näin ollen vaikuttavia syitä opiskelijoiden heikkoon suoriutumiseen on etsitty jo pitkään. Erityisesti heikot, aloittelevat ohjelmoijat tulisi tunnistaa mahdollisimman pian, jotta heille voitaisiin tarjota tukea ja pienentää opiskelijan riskiä epäonnistua kurssin läpäimisessä ja riskiä jättää kurssi kesken. Heikkojen opiskelijoiden tunnistaminen on tärkeää, jotta voidaan suunnitella aikainen väliintulo. Tämän väitöskirjatyön tarkoituksena oli kehittää todennettu, ennustava malli tai malleja sopivilla ennnustusfunktioilla koskien opiskelijan akateemista suoriutumista ohjelmoinnin peruskursseilla. Kehitetty malli käyttää koneoppivaa naiivia bayesilaista luokittelualgoritmia analysoimaan opiskelijoiden suoriutumisesta kertynyttä aineistoa. Lähestymistapa perustuu yksinkertaisimpien mahdollisten selittävien mallien periaatteeseen. Lisäksi, tavoitteena oli ehdottaa tätä validoitua ennustavaa mallia varhaiseksi varoitusjärjestelmäksi, jolla ennustetaan putoamisvaarassa olevat opiskelijat opintojakson alkuvaiheessa sekä informoidaan ohjaajia (ja opiskelijaa) aikaisen väliintulon tarpeellisuudesta. Keräsimme aineistoa kahdelta ohjelmoinnin peruskurssilta, jonka pohjalta ennustavaa mallia kehitettiin ja testattiin. Mallit on rakennettu opiskelijoiden ennakkotietojen ja kurssin kestäessä kerättyjen suoriutumistietojen perusteella, joita ohjaajat voivat helposti kerätä tai joihin he voivat päästä käsiksi oppilaitoksesta tai muusta ympäristöstä huolimatta. Lisäksi väitöskirjatyö analysoi valittujen datalähteiden ennustettavuutta ja sitä, mitkä mallien muuttujista ja niiden kombinaatioista tuottivat kannaltamme korkeimman ennustetarkkuuden opiskelijoiden akateemisessa suoriutumisessa. Mallien ennustusten tarkkuuksia laskettiin käyttämällä sekaannusmatriisia, josta saadaan laskettua ennusteen tarkkuus, ennusteen spesifisyys, sensitiivisyys, tasapainotettu tarkkuus sekä luokitteluvastekäyriä (receiver operating characteristics (ROC)) ja näiden luokitteluvastepinta-ala (area under curve (AUC)) Mallit, jotka kehitettiin formatiivisilla tehtävillä, ja joissa ohjaaja saattoi osittain auttaa luokkahuonetilanteessa, antoivat keskimäärin tarkemman ennustuksen putoamisvaarassa olevista opiskelijoista kuin mallit, joissa käytettiin kotiin vietäviä tehtäviä ainoina ennusteina. Tuntemattomalla testiaineistolla tehdyt mallinnukset osoittavat, että voimme tunnistaa jo 3. viikon kohdalla 83% niistä opiskelijoista, jotka tarvitsevat lisätukea 12 viikkoa kestävällä ohjelmoinnin kurssilla. Tulosten perusteella vaikuttaisi, että yhdistämällä metodeja voidaan saavuttaa parempi yleinen ennustettavuus putoamisvaarassa olevien opiskelijoiden suhteen pienemmällä määrällä väärin luokiteltuja epätositapauksia. Tulokset viittaavat myös siihen, että on mahdollista sisällyttää yhdistelmämalli varoitusjärjestelmiin, jotta voidaan tunnistaa avuntarpeessa olevia opiskelijoita ja tarjota täten varhaisessa vaiheessa tukea ennen kuin on liian myöhäistä. Tässä tutkimuksessa esitellyt mallit on kehitetty nojautuen yksinkertaisimman selittävän mallin periaatteeseen ja myös aiempiin tutkimustuloksiin, joissa huomioidaan erilaiset akateemiset ympäristöt ja opiskelijoiden tausta. Ennustava malli voi tarjota indikaattoreita, jotka voivat mahdollisesti toimia pohjana väliintulostrategioihin kurssilta putoamisvaarassa olevien opiskelijoiden tukemiseksi. Tämän tutkimuksen keskeisin anti on malli, jolla opiskelijoiden suoriutumista voidaan arvioida muilla ohjelmointia ja muita aihepiirejä käsittelevillä kursseilla, jotka sisältävät sekä jatkuvaa arviointia että loppukokeen. Malli ennustaisi näillä kursseilla lopullisen opiskelijan suoritustason opetusjakson alkuvaiheessa

Prediction of Student Final Exam Performance in an Introductory Programming Course: Development and Validation of the Use of a Support Vector Machine-Regression Model

This paper presents a Support Vector Machine predictive model to determine if prior programming knowledge and completion of in-class and take home formative assessment tasks might be suitable predictors of examination performance. Student data from the academic years 2012 - 2016 for an introductory programming course was captured via ViLLE e-learning tool for analysis. The results revealed that student prior programming knowledge and assessment scores captured in a predictive model, is a good fit of the data. However, while overall success of the model is significant, predictions on identifying at-risk students is neither high nor low and that persuaded us to include two more research questions. However, our preliminary post analysis on these test results show that on average students who secured less than 70% in formative assessment scores with little or basic prior programming knowledge in programming may fail in the final programming exam and increase the prediction accuracy in identifying at-risk students from 46% to nearly 63%. Hence, these results provide immediate information for programming course instructors and students to enhance teaching and learning process.</p

Relationship between perceived problem-solving skills and academic performance of novice learners in introductory programming courses

Past research has shown that student problem-solving skills may be used to determine student final exam performance. This study reports on the relationship between student perceived problem-solving skills and academic performance in introductory programming, in formative and summative programming assessment tasks. We found that the more effective problem solvers achieved better final exam scores. There was no significant difference in formative assessment performances between effective and poor problem solvers. It is also possible to categorize students on the basis of problem-solving skills, in order to exploit opportunities to improve learning around constructivist learning theory. Finally, our study identified transferability skills and the study may be extended to identify the impact of problem solving transfer skills on student problem solving for programming

Comparing importance of knowledge and professional skill areas for engineering programming utilizing a two group Delphi survey

All engineering careers require some level of programming proficiency. However, beginning programming classes are challenging for many students. Difficulties have been well-documented and contribute to high drop-out rates which prevent students from pursuing engineering. While many approaches have been tried to improve the performance of students and reduce the dropout rate, continued work is needed. This research seeks to re-examine what items are critical for programming education and how those might inform what is taught in introductory programming classes (CS1). Following trends coming from accreditation and academic boards on the importance of professional skills, we desire to rank knowledge and professional skill areas in one list. While programming curricula focus almost exclusively on knowledge areas, integrating critical professional skill areas could provide students with a better high-level understanding of what engineering encompasses. Enhancing the current knowledge centric syllabi with critical professional skills should allow students to have better visibility into what an engineering job might be like at the earliest classes in the engineering degree. To define our list of important professional skills, we use a two-group, three-round Delphi survey to build consensus ranked lists of knowledge and professional skill areas from industry and academic experts. Performing a gap analysis between the expert groups shows that industry experts focus more on professional skills then their academic counterparts. We use this resulting list to recommend ways to further integrate professional skills into engineering programming curriculum

Impact of Scratch on the achievements of first-year computer science students in programming in some Nigerian polytechnics

To support the advancement of modern civilisation, our institutions of higher learning must produce the right pool of professionals, who can develop innovative software. However, the teaching and learning of the first programming language (CS1) remains a great challenge for most educators and novice computer students. Indicators such as failure and attrition rates, and CS1 student engagement, continue to show that conventional pedagogy does not adequately meet the needs of some beginning CS students. For its ease in introducing novices to programming, Scratch—a visual programming environment following the constructionism philosophy of Seymour Papert—is now employed even in some higher education CS1 classes with mixed evidence of its impact. Scratch captures the constructionist agenda by its slogan: “Imagine, Program, Share.” Therefore, this study explored the impart of using a constructionist Scratch programming pedagogy on higher education CS1 students’ achievements. This study also sought to compare the impacts of the two CS1 modes: the conventional class - involving textual programming language, lectures and labs, and the constructionist Scratch inquiry-based programming class. It further aims to discover if gender, academic level, age, prior programming, and visual artistic abilities moderate the effects of programming pedagogy on students’ achievements. To realize the study’s aims, the study employed a quasi-experimental pretest-posttest nonequivalent groups design, involving four intact CS1 classes of polytechnic students (N = 418) in north-central Nigeria. The investigation was conducted in phases: a pilot (n = 236) and main (n=182) studies lasting two academic sessions, with each study comprising one experimental and one control group. In each session, learning in both modes lasted for six weeks. In both studies, purposive sampling was employed to select institutions, and selected institutions were randomly assigned to treatment groups. Instruments employed included CS1 Student Profile Questionnaire (CSPROQ) and Introductory Programming Achievement Test (IPAT). To strengthen the research design, I employed Coarsened Exact Matching (CEM) algorithm—after conducting a priori power analysis—to generate matched random samples of cases from both studies. Thus, research data employed in the analysis include: from the pilot, 41 cases in each treatment group; from the main study, 42 cases in each treatment group. Descriptive and inferential statistics were employed to find answers to research questions and test the research hypothesis. Data from both studies satisfied the requirements for statistical tests employed, i.e., t-test and ANCOVA. The alpha level used in testing hypotheses was p = 0.05. The dependent variable is the IPAT post-test score, while the independent variables are treatment, gender, age, academic achievement level, prior programming, and prior visual art. The covariate was the IPAT pretest score. Statistical analyses were conducted using SPSS version 23. The t-test results from both pilot and main studies indicated that, both programming pedagogies had significant effects on student IPAT scores, although the effect of the constructionist Scratch intervention was higher. Results from the one-way ANCOVA analysis of both pilot and main study data—while controlling for students’ IPAT pretest scores—yielded the same outcome: There was significant main effect of treatment on students’ IPAT posttest scores, although the impact was moderate. Controlling for pre test scores, analysis of the main studies data yielded no significant main effects of: gender, age, academic level, prior programming and prior visual artistic ability. The result from the main study also reveals no interaction effect of treatment, gender, academic level, age, prior programming, and prior artistic ability. While the quality of CS1 students’ performance in each session varies as their IPAT achievements show, yet the results of this research revealed a consistent pattern: Students in the constructionist Scratch class outperformed those in the conventional class, although the impart was moderate. This finding implies college students without prior programming experience can perform better in a class following a constructionist Scratch programming pedagogy. The study recommends the use of Scratch, following a constructionist pedagogy with first-year students in colleges, especially those without prior background in programmingSchool of ComputingPh. D. (Computing Education

The Development of Seychellois Primary School Teachers' Mathematical Knowledge for Teaching and Self-Efficacy Beliefs through Reflective Practice: A Quasi-experimental Study

Primary school pupils in Seychelles are underperforming with low standards of proficiency in mathematics, relative to local and international benchmarks. There are concerns that primary school teachers might not have the appropriate Mathematical Knowledge for Teaching (MKT) nor confidence to teach mathematics. While it is widely accepted that MKT, and self-efficacy beliefs that assist the development of MKT, are important for effective teaching, less is known about how they are enhanced through reflective practice (RP). Enhancing teachers’ MKT and self-efficacy beliefs is important because these have been connected to pupils’ achievement. This mixed-methods research explores to what extent the MKT sub-domains, MKT self-efficacy beliefs, and Mathematics Teaching Self-efficacy (MTSE), of seventeen in-service primary school teachers, in the areas of number concepts, number operations and word problem solving, are enhanced through RP. Using a pragmatic approach, this multiple case study integrates inductive designs and quasi-experimental methodologies to analyse reflective journals, semi-structured interviews, pre-post-tests, and pre-post questionnaire surveys, within an interpretivist paradigm. Independent-samples t-test, Paired-samples t-test, Cluster analysis, and Difference in Differences (DiD) methodology were used to analyse the quantitative data. The qualitative data were analysed thematically. This study employed Desimone’s professional development framework, Mezirow’s transformative learning theory (TLT), and the social-constructivist theory as conceptual and theoretical lenses respectively to frame this research. Analysis of the quantitative data revealed that there were no statistically significant differences between the intervention group and the control group in their MKT sub-domains and MTSE. However, the DiD estimator showed that the intervention group had a systematic increase in both their MKT sub-domains and MTSE levels through RP. On the other hand, there was statistically significant difference in the MKT self-efficacy beliefs between the intervention and control groups. Seven themes wove through the teachers’ ‘stories’, thus supporting positive influence of RP on MKT sub-domains, MKT self-efficacy, and MTSE of the teachers. Merging the qualitative and quantitative evidence indicated that the ten weeks of RP of the teachers started to have positive effects on their MKT and self-efficacy beliefs; not identified in previous studies. The teachers expressed positive change in their MKT and self-efficacy beliefs pertaining to the teaching of word problem solving strategies. Nevertheless, the results also demonstrated that educational change is difficult and requires commitment and time. This research makes two original contributions to knowledge. Firstly, to the MKT literature through a proposed framework for teachers’ MKT growth through RP. Secondly, to Bandura’s sources of efficacy theory through a typology of teachers’ self-efficacy growth through RP. The findings of this study are proposed as frameworks for teachers’ professional growth in the context of Seychelles and also in Small Island Developing States (SIDS)