Programming: Predicting student success early in CS1. A re-validation and replication study

This paper describes a large, multi-institutional revalidation study conducted in the academic year 2015-16. Six hundred and ninetytwo students participated in this study, from 11 institutions (ten institutions in Ireland and one in Denmark). The primary goal was to validate and further develop an existing computational prediction model called Predict Student Success (PreSS). In doing so, this study addressed a call from the 2015 ITiCSE working group (the second Grand Challenge ), to systematically analyse and verify previous studies using data from multiple contexts to tease out tacit factors that contribute to previously observed outcomes . PreSS was developed and validated in a longitudinal study conducted over a three year period (twelve years previous from 2004- 06). PreSS could predict with near 80% accuracy, how a student would likely perform on an introductory programming module. Notably this could be achieved at a very early stage in the module. This paper describes a revalidation of the original PreSS model on a significantly larger multi-institutional data set twelve years after its initial development and looks at recent research on additional factors that may improve the model. The work involved the development of a fully automated end-to-end tool, which can predict student success early in CS1, with an accuracy of 71%. This paper describes, in detail the PreSS model, recent research, pilot studies and the re-validation and replication study of the PreSS model

CS1: how will they do? How can we help? A decade of research and practice

Background and Context: Computer Science attrition rates (in the western world) are very concerning, with a large number of students failing to progress each year. It is well acknowledged that a significant factor of this attrition, is the students’ difficulty to master the introductory programming module, often referred to as CS1. Objective: The objective of this article is to describe the evolution of a prediction model named PreSS (Predict Student Success) over a 13-year period (2005–2018). Method: This article ties together, the PreSS prediction model; pilot studies; a longitudinal, multi-institutional re-validation and replication study; improvements to the model since its inception; and interventions to reduce attrition rates. Findings: The outcome of this body of work is an end-to-end real-time web-based tool (PreSS#), which can predict student success early in an introductory programming module (CS1), with an accuracy of 71%. This tool is enhanced with interventions that were developed in conjunction with PreSS#, which improved student performance in CS1. Implications: This work contributes significantly to the computer science education (CSEd) community and the ITiCSE 2015 working group’s call (in particular the second grand challenge), by re-validating and developing further the original PreSS model, 13 years after it was developed, on a modern, disparate, multi-institutional data set

Online tools to support novice programming: A systematic review

Novice programming is a challenging subject to both the students and the educators. A novice programmer is required to acquire new knowledge to solve a problem and propose a solution systematically. This is followed by constructing the solution in a development environment that they are unfamiliar with. This research looks at the challenges faced by a novice programmer and the online methods that are popular to assist the students. Online block programming is a popular option. One of the software that had been implemented in the various research project is Scratch. From the reviewed research, it shows that the trend is moving towards an intelligent tutoring system, where students can have personalized engagement for their learning experience. This paper presents a systematic review conducted using the keywords ”novice programming”, ”introductory”, ”CS1”, ”difficulties”, ”challenges”, and ”threshold concepts”. From the review conducted, it is observed that most of the work is carried out to ease the implementation of the solution through an integrated development environment, and block programming. On the support for instructors, the discussion on curriculum and challenges in CS1 tops the chart. This is followed by active learning through online tools

Predicting and Improving Performance on Introductory Programming Courses (CS1)

This thesis describes a longitudinal study on factors which predict academic success in introductory programming at undergraduate level, including the development of these factors into a fully automated web based system (which predicts students who are at risk of not succeeding early in the introductory programming module) and interventions to address attrition rates on introductory programming courses (CS1). Numerous studies have developed models for predicting success in CS1, however there is little evidence on their ability to generalise or on their use beyond early investigations. In addition, they are seldom followed up with interventions, after struggling students have been identified. The approach overcomes this by providing a web-based real time system, with a prediction model at its core that has been longitudinally developed and revalidated, with recommendations for interventions which educators could implement to support struggling students that have been identified. This thesis makes five fundamental contributions. The first is a revalidation of a prediction model named PreSS. The second contribution is the development of a web-based, real time implementation of the PreSS model, named PreSS#. The third contribution is a large longitudinal, multi-variate, multi-institutional study identifying predictors of performance and analysing machine learning techniques (including deep learning and convolutional neural networks) to further develop the PreSS model. This resulted in a prediction model with approximately 71% accuracy, and over 80% sensitivity, using data from 11 institutions with a sample size of 692 students. The fourth contribution is a study on insights on gender differences in CS1; identifying psychological, background, and performance differences between male and female students to better inform the prediction model and the interventions. The final, fifth contribution, is the development of two interventions that can be implemented early in CS1, once identified by PreSS# to potentially improve student outcomes. The work described in this thesis builds substantially on earlier work, providing valid and reliable insights on gender differences, potential interventions to improve performance and an unsurpassed, generalizable prediction model, developed into a real time web-based system