Introductory programming: a systematic literature review

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming. This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research

How Novices Use LLM-Based Code Generators to Solve CS1 Coding Tasks in a Self-Paced Learning Environment

As Large Language Models (LLMs) gain in popularity, it is important to understand how novice programmers use them. We present a thematic analysis of 33 learners, aged 10-17, independently learning Python through 45 code-authoring tasks using Codex, an LLM-based code generator. We explore several questions related to how learners used these code generators and provide an analysis of the properties of the written prompts and the generated code. Specifically, we explore (A) the context in which learners use Codex, (B) what learners are asking from Codex, (C) properties of their prompts in terms of relation to task description, language, and clarity, and prompt crafting patterns, (D) the correctness, complexity, and accuracy of the AI-generated code, and (E) how learners utilize AI-generated code in terms of placement, verification, and manual modifications. Furthermore, our analysis reveals four distinct coding approaches when writing code with an AI code generator: AI Single Prompt, where learners prompted Codex once to generate the entire solution to a task; AI Step-by-Step, where learners divided the problem into parts and used Codex to generate each part; Hybrid, where learners wrote some of the code themselves and used Codex to generate others; and Manual coding, where learners wrote the code themselves. The AI Single Prompt approach resulted in the highest correctness scores on code-authoring tasks, but the lowest correctness scores on subsequent code-modification tasks during training. Our results provide initial insight into how novice learners use AI code generators and the challenges and opportunities associated with integrating them into self-paced learning environments. We conclude with various signs of over-reliance and self-regulation, as well as opportunities for curriculum and tool development.Comment: 12 pages, Peer-Reviewed, Accepted for publication in the proceedings of the 2023 ACM Koli Calling International Conference on Computing Education Researc

SMAS: A solution-based multi-agent system for improving problem solving skills in computer programming

In this research, a solution-based multi-agent system (SMAS) is proposed, which benefits from a novel automatic text-to-flowchart conversion approach in order to improve students' problem solving skills. The aim is to introduce the early stages of learning programming (CS1). By using SMAS, students can focus on solution designing activities in the form of flowchart development more than on language and syntax. Ultimately, an experimental study is devised to assess the success of SMAS as a tool to aid students with problem solving activities and learning computer programming. In total, 30.4% of problems that were left unresolved in previous sessions were solved by students in the control group, whereas 69.7% of previously unresolved problems were solved by students in the experimental group who used SMAS. Therefore, the use of SMAS in practice is supported, as the results indicate considerable gains for the experimental group over the control group

The Role of Artificial Intelligence in Educating Novice Programmers

Programming is an inherently difficult skill to acquire and develop. Those who attempt to learn programming may be easily discouraged. The current landscape for computer science education does not address the needs of every novice programmer. Literature reports a discrepancy between student misconceptions and instructors’ perceptions of those misconceptions. Those who can afford a one-on-one human tutor perform on average two standard deviations better than those who learn via conventional methods, suggesting there is a need for a comparable, cheaper replacement. As a result, a number of intelligent tutoring systems have been developed for the purpose of teaching introductory programming concepts and replicating the benefits of one-on-one human tutoring. In this thesis, we analyze and discuss the literature pertaining to student misconceptions, selecting five fundamental misconception categories for introductory programming to demonstrate the effectiveness of existing intelligent tutoring systems. The features of existing intelligent tutoring systems are discussed and analyzed with respect to their effectiveness in addressing student misconceptions. Finally, we highlight the current gap in research on intelligent tutoring systems, hypothesizing the architecture and features of an ideal intelligent tutoring system for introductory programming.Electrical and Computer Engineerin

Beyond Automated Assessment: Building Metacognitive Awareness in Novice Programmers in CS1

The primary task of learning to program in introductory computer science courses (CS1) cognitively overloads novices and must be better supported. Several recent studies have attempted to address this problem by understanding the role of metacognitive awareness in novices learning programming. These studies have focused on teaching metacognitive awareness to students by helping them understand the six stages of learning so students can know where they are in the problem-solving process, but these approaches are not scalable. One way to address scalability is to implement features in an automated assessment tool (AAT) that build metacognitive awareness in novice programmers. Currently, AATs that provide feedback messages to students can be said to implement the fifth and sixth learning stages integral to metacognitive awareness: implement solution (compilation) and evaluate implemented solution (test cases). The computer science education (CSed) community is actively engaged in research on the efficacy of compile error messages (CEMs) and how best to enhance them to maximize student learning and it is currently heavily disputed whether or not enhanced compile error messages (ECEMs) in AATs actually improve student learning. The discussion on the effectiveness of ECEMs in AATs remains focused on only one learning stage critical to metacognitive awareness in novices: implement solution. This research carries out an ethnomethodologically-informed study of CS1 students via think-aloud studies and interviews in order to propose a framework for designing an AAT that builds metacognitive awareness by supporting novices through all six stages of learning. The results of this study provide two important contributions. The first is the confirmation that ECEMs that are designed from a human-factors approach are more helpful for students than standard compiler error messages. The second important contribution is that the results from the observations and post-assessment interviews revealed the difficulties novice programmers often face to developing metacognitive awareness when using an AAT. Understanding these barriers revealed concrete ways to help novice programmers through all six stages of the problem-solving process. This was presented above as a framework of features, which when implemented properly, provides a scalable way to implicitly produce metacognitive awareness in novice programmers

An Intelligent Debugging Tutor For Novice Computer Science Students

Debugging is a necessary aspect of computer science that can be difficult for novices and experienced programmers alike. This skill is mainly self-taught and is generally gained through trial and error, perhaps with some assistance from a professor or other expert figure. Novices encountering their first software defects may have few avenues open to them depending on the environment in which they are learning to program. The evident problem here is that the potential for a student to become stuck, frustrated, and/or losing confidence in their ability to pursue computer science is great. For a student to be successful when working professionally or progressing through academia they need to be able to function independently; trusting their own knowledge on par or above that of others so that their productivity does not rely on the knowledge of someone else. In order to solve this problem an Intelligent Tutoring System for teaching debugging skills to the novice utilizing Case Based Reasoning, Static Program Slicing, and the student\u27s preferred learning style was proposed. Case acquisition and automatic Exercise Generation were also explored. The system built for this research program was evaluated using novice students at the College and High School levels. Results of this evaluation produced statistically significant results at the p\u3c.05 and p\u3c.01 levels, with generated exercises exhibiting significance at the p\u3c.01 level. These results prove that the methodology chosen is a valid approach for the problem described, that the system does in fact teach students how to debug programs, and that the system is capable of successfully generating exercises on the fly

Tutor-Complete: An Educational Game and Intelligent Tutoring System for Languages and Automata

Educational games and Intelligent Tutoring Systems have been shown to improve student learning outcomes by increasing engagement and providing individualized instruction. However, while introductory programming students frequently benefit from such systems, students in upper-level theoretical courses such as CS 235 (Languages and Automata) have dense textbooks and dry mathematical readings as their primary or only resources. Tutor-Complete aims to fill this gap by presenting two fundamental CS 235 concepts in a game environment. In the first activity, students construct Deterministic Finite-State Automata in order to guide their character across a landscape. In the second activity, students build proofs using the pumping lemma to defeat the “villain” character. Tutor-Complete also fosters a peer-learning environment by encouraging students to explain concepts to each other and providing hints based on past student work. Finally, Tutor-Complete uses Bayesian Knowledge Tracing to model students’ knowledge and tailor the learning experience accordingly

Predicting Academic Performance: A Systematic Literature Review

The ability to predict student performance in a course or program creates opportunities to improve educational outcomes. With effective performance prediction approaches, instructors can allocate resources and instruction more accurately. Research in this area seeks to identify features that can be used to make predictions, to identify algorithms that can improve predictions, and to quantify aspects of student performance. Moreover, research in predicting student performance seeks to determine interrelated features and to identify the underlying reasons why certain features work better than others. This working group report presents a systematic literature review of work in the area of predicting student performance. Our analysis shows a clearly increasing amount of research in this area, as well as an increasing variety of techniques used. At the same time, the review uncovered a number of issues with research quality that drives a need for the community to provide more detailed reporting of methods and results and to increase efforts to validate and replicate work.Peer reviewe