Introductory programming: a systematic literature review

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming. This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research

"It's not like Jarvis, but it's pretty close!" -- Examining ChatGPT's Usage among Undergraduate Students in Computer Science

Large language models (LLMs) such as ChatGPT and Google Bard have garnered significant attention in the academic community. Previous research has evaluated these LLMs for various applications such as generating programming exercises and solutions. However, these evaluations have predominantly been conducted by instructors and researchers, not considering the actual usage of LLMs by students. This study adopts a student-first approach to comprehensively understand how undergraduate computer science students utilize ChatGPT, a popular LLM, released by OpenAI. We employ a combination of student surveys and interviews to obtain valuable insights into the benefits, challenges, and suggested improvements related to ChatGPT. Our findings suggest that a majority of students (over 57%) have a convincingly positive outlook towards adopting ChatGPT as an aid in coursework-related tasks. However, our research also highlights various challenges that must be resolved for long-term acceptance of ChatGPT amongst students. The findings from this investigation have broader implications and may be applicable to other LLMs and their role in computing education.Comment: Accepted in ACE 2024: https://aceconference2024.github.io/aceconference2024/ We thank Shamik Sinha, Palak Bhardwaj, Rayyan Hussain, Kashvi Panvanda, Sidhartha Garg, Shagun Yadav, Yash Chillar, and Sourav Goyal for their assistance in the development of questionnaires, as well as their involvement in conducting selected interview

Predicting and Improving Performance on Introductory Programming Courses (CS1)

This thesis describes a longitudinal study on factors which predict academic success in introductory programming at undergraduate level, including the development of these factors into a fully automated web based system (which predicts students who are at risk of not succeeding early in the introductory programming module) and interventions to address attrition rates on introductory programming courses (CS1). Numerous studies have developed models for predicting success in CS1, however there is little evidence on their ability to generalise or on their use beyond early investigations. In addition, they are seldom followed up with interventions, after struggling students have been identified. The approach overcomes this by providing a web-based real time system, with a prediction model at its core that has been longitudinally developed and revalidated, with recommendations for interventions which educators could implement to support struggling students that have been identified. This thesis makes five fundamental contributions. The first is a revalidation of a prediction model named PreSS. The second contribution is the development of a web-based, real time implementation of the PreSS model, named PreSS#. The third contribution is a large longitudinal, multi-variate, multi-institutional study identifying predictors of performance and analysing machine learning techniques (including deep learning and convolutional neural networks) to further develop the PreSS model. This resulted in a prediction model with approximately 71% accuracy, and over 80% sensitivity, using data from 11 institutions with a sample size of 692 students. The fourth contribution is a study on insights on gender differences in CS1; identifying psychological, background, and performance differences between male and female students to better inform the prediction model and the interventions. The final, fifth contribution, is the development of two interventions that can be implemented early in CS1, once identified by PreSS# to potentially improve student outcomes. The work described in this thesis builds substantially on earlier work, providing valid and reliable insights on gender differences, potential interventions to improve performance and an unsurpassed, generalizable prediction model, developed into a real time web-based system

Does Creating Programming Assignments with Tests Lead to Improved Performance in Writing Unit Tests?

We have constructed a tool, CrowdSorcerer, in which students create programming assignments, their model solutions and associated test cases using a simple input-output format. We have used the tool as a part of an introductory programming course with normal course activities such as programming assignments and a final exam. In our work, we focus on whether creating programming assignments and associated tests correlate with students' performance in a testing-related exam question. We study this through an analysis of the quality of student-written tests within the tool, measured using the number of test cases, line coverage and mutation coverage, and students' performance in testing related exam question, measured using exam points. Finally, we study whether previous programming experience correlates with how students act within the tool and within the testing related exam question.Peer reviewe

Crowdsourcing Content Creation for SQL Practice

Crowdsourcing refers to the act of using the crowd to create content or to collect feedback on some particular tasks or ideas. Within computer science education, crowdsourcing has been used -- for example -- to create rehearsal questions and programming assignments. As a part of their computer science education, students often learn relational databases as well as working with the databases using SQL statements. In this article, we describe a system for practicing SQL statements. The system uses teacher-provided topics and assignments, augmented with crowdsourced assignments and reviews. We study how students use the system, what sort of feedback students provide to the teacher-generated and crowdsourced assignments, and how practice affects the feedback. Our results suggest that students rate assignments highly, and there are only minor differences between assignments generated by students and assignments generated by the instructor.Peer reviewe

Toward Using Games to Teach Fundamental Computer Science Concepts

Video and computer games have become an important area of study in the field of education. Games have been designed to teach mathematics, physics, raise social awareness, teach history and geography, and train soldiers in the military. Recent work has created computer games for teaching computer programming and understanding basic algorithms. We present an investigation where computer games are used to teach two fundamental computer science concepts: boolean expressions and recursion. The games are intended to teach the concepts and not how to implement them in a programming language. For this investigation, two computer games were created. One is designed to teach basic boolean expressions and operators and the other to teach fundamental concepts of recursion. We describe the design and implementation of both games. We evaluate the effectiveness of these games using before and after surveys. The surveys were designed to ascertain basic understanding, attitudes and beliefs regarding the concepts. The boolean game was evaluated with local high school students and students in a college level introductory computer science course. The recursion game was evaluated with students in a college level introductory computer science course. We present the analysis of the collected survey information for both games. This analysis shows a significant positive change in student attitude towards recursion and modest gains in student learning outcomes for both topics

On the Quality of Crowdsourced Programming Assignments

Crowdsourcing has been used in computer science education to alleviate the teachers’ workload in creating course content, and as a learning and revision method for students through its use in educational systems. Tools that utilize crowdsourcing can act as a great way for students to further familiarize themselves with the course concepts, all while creating new content for their peers and future course iterations. In this study, student-created programming assignments from the second week of an introductory Java programming course are examined alongside the peer reviews these assignments received. The quality of the assignments and the peer reviews is inspected, for example, through comparing the peer reviews with expert reviews using inter-rater reliability. The purpose of this study is to inspect what kinds of programming assignments novice students create, and whether the same novice students can act as reliable reviewers. While it is not possible to draw definite conclusions from the results of this study due to limitations concerning the usability of the tool, the results seem to indicate that novice students are able to recognise differences in programming assignment quality, especially with sufficient guidance and well thought-out instructions

Task-related models for teaching and assessing iteration learning in high school

A number of studies report about students’ difficulties with basic flow-control constructs, and specifically with iteration. Although such issues are less explored in the context of pre-tertiary education, this seems to be especially the case for high-school programming learning, where the difficulties concern both the “mechanical” features of the notional machine as well as the logical aspects connected with the constructs, ranging from the implications of loop conditions to a more abstract grasp of the underlying algorithms. For these reasons, the aim of this work is to: i) identifying methodological tools to enhance a comprehensive understanding of the iteration constructs, ii) suggest strategies to teach iterations. We interviewed 20 experienced upper secondary teachers of introductory programming in different kinds of schools. The interviews were mainly aimed at ascertaining teachers’ beliefs about major sources of issues for basic programming concepts and their approach to the teaching and learning of iteration constructs. Once teachers’ perception of students’ difficulties have been identified, we have submitted, to a sample of 164 students, a survey which included both questions on their subjective perception of difficulty and simple tasks probing their understanding of iteration. Data collected from teachers and students confirm that iteration is a central programming concept and indicate that the treatment of conditions and nested constructs are major sources of students’ difficulties with iteration. The interviews allowed us to identify a list of problems that are typically presented by teachers to explain the iterations. Hence, a catalogue of significant program examples has been built to support students’ learning, tasks with characteristics different from those typically presented in class. Based on the outcome of previous steps, a survey to collect related information and good practices from a larger sample of teachers has been designed. Data collected have been analysed distinguishing an orientation towards more conceptual objectives, and one towards more practical objectives. Furthermore, regarding evaluation, a orientation focused on process-based assessment and another on product-based assessment. Finally, based on the outcome of previous students’ survey and drawing from the proposed examples catalogue, we have designed and submitted a new students’ survey, composed of a set of small tasks, or tasklets, to investigate in more depth on high-school students’ understanding of iteration in terms of code reading abilities. The chosen tasklets covered the different topics: technical program feature, correlation between tracing effort and abstraction, the role of flow-charts, students’ perception of self-confidence concerning high-level thinking skills.A number of studies report about students’ difficulties with basic flow-control constructs, and specifically with iteration. Although such issues are less explored in the context of pre-tertiary education, this seems to be especially the case for high-school programming learning, where the difficulties concern both the “mechanical” features of the notional machine as well as the logical aspects connected with the constructs, ranging from the implications of loop conditions to a more abstract grasp of the underlying algorithms. For these reasons, the aim of this work is to: i) identifying methodological tools to enhance a comprehensive understanding of the iteration constructs, ii) suggest strategies to teach iterations. We interviewed 20 experienced upper secondary teachers of introductory programming in different kinds of schools. The interviews were mainly aimed at ascertaining teachers’ beliefs about major sources of issues for basic programming concepts and their approach to the teaching and learning of iteration constructs. Once teachers’ perception of students’ difficulties have been identified, we have submitted, to a sample of 164 students, a survey which included both questions on their subjective perception of difficulty and simple tasks probing their understanding of iteration. Data collected from teachers and students confirm that iteration is a central programming concept and indicate that the treatment of conditions and nested constructs are major sources of students’ difficulties with iteration. The interviews allowed us to identify a list of problems that are typically presented by teachers to explain the iterations. Hence, a catalogue of significant program examples has been built to support students’ learning, tasks with characteristics different from those typically presented in class. Based on the outcome of previous steps, a survey to collect related information and good practices from a larger sample of teachers has been designed. Data collected have been analysed distinguishing an orientation towards more conceptual objectives, and one towards more practical objectives. Furthermore, regarding evaluation, a orientation focused on process-based assessment and another on product-based assessment. Finally, based on the outcome of previous students’ survey and drawing from the proposed examples catalogue, we have designed and submitted a new students’ survey, composed of a set of small tasks, or tasklets, to investigate in more depth on high-school students’ understanding of iteration in terms of code reading abilities. The chosen tasklets covered the different topics: technical program feature, correlation between tracing effort and abstraction, the role of flow-charts, students’ perception of self-confidence concerning high-level thinking skills

Enhancing the social issues components in our computing curriculum: Computing for the social good

The acceptance and integration of social issues into computing curricula is still a work in progress twenty years after it was first incorporated into the ACM Computing Curricula. Through an international survey of computing instructors, this paper corroborates prior work showing that most institutions include the societal impact of ICT in their programs. However, topics often concentrate on computer history, codes of ethics and intellectual property, while neglecting broader issues of societal impact. This paper explores how these neglected topics can be better developed through a subtle change of focus to the significant role that ICT plays in addressing the needs of the community. Drawing on the survey and a set of implementation cases, the paper provides guidance by means of examples and resources to empower teaching teams to engage students in the application of ICT to bring about positive social outcomes – computing for the social good