Dcc --help: Generating Context-Aware Compiler Error Explanations with Large Language Models

In the challenging field of introductory programming, high enrollments and failure rates drive us to explore tools and systems to enhance student outcomes, especially automated tools that scale to large cohorts. This paper presents and evaluates the dcc --help tool, an integration of a Large Language Model (LLM) into the Debugging C Compiler (DCC) to generate unique, novice-focused explanations tailored to each error. dcc --help prompts an LLM with contextual information of compile- and run-time error occurrences, including the source code, error location and standard compiler error message. The LLM is instructed to generate novice-focused, actionable error explanations and guidance, designed to help students understand and resolve problems without providing solutions. dcc --help was deployed to our CS1 and CS2 courses, with 2,565 students using the tool over 64,000 times in ten weeks. We analysed a subset of these error/explanation pairs to evaluate their properties, including conceptual correctness, relevancy, and overall quality. We found that the LLM-generated explanations were conceptually accurate in 90% of compile-time and 75% of run-time cases, but often disregarded the instruction not to provide solutions in code. Our findings, observations and reflections following deployment indicate that dcc-help provides novel opportunities for scaffolding students' introduction to programming.Comment: 7 pages, 2 figures. Accepted in SIGCSE'2

A review and assessment of novice learning tools for problem solving and program development

There is a great demand for the development of novice learning tools to supplement classroom instruction in the areas of problem solving and program development. Research in the area of pedagogy, the psychology of programming, human-computer interaction, and cognition have provided valuable input to the development of new methodologies, paradigms, programming languages, and novice learning tools to answer this demand. Based on the cognitive needs of novices, it is possible to postulate a set of characteristics that should comprise the components an effective novice-learning tool. This thesis will discover these characteristics and provide recommendations for the development of new learning tools. This will be accomplished with a review of the challenges that novices face, an in-depth discussion on modem learning tools and the challenges that they address, and the identification and discussion of the vital characteristics that constitute an effective learning tool based on these tools and personal ideas

Evaluation of Python Error Message Interpretation: Study on Students with Different Levels of Programming Experience

Error messages are a helpful tool for everyone interested in learning a programming language. Even after learning the language, novice and experienced programmers alike, have to interact with an error message one way or another. However, there is no single programming language that is usedacross all platforms and systems, so the programmers have to write programs in many different languages. If the programming languages have similar structures, working with a new language is relatively easy. We have asked the question, whether the same effect could be observed in case of error messages. We designed an online survey which was conducted internationally to measure whether the general experience of programmers from any programming language influences the programmer’s ability to correct python error message. The survey was aimed at students who have some experience with programming. We find in comparison with novices, on average the experiencedprogrammers find and fix an error in the code given the sameerror message with its corresponding code snippet. Additionally,we see correlations between the average of correctly fixed errorsand number of programming languages that the participants hadexperience with, as well as their chosen major and their age

Teaching Introductory Programming from A to Z: Twenty-Six Tips from the Trenches

A solid foundation in computer programming is critical for students to succeed in advanced computing courses, but teaching such an introductory course is challenging. Therefore, it is important to develop better approaches in order to improve teaching effectiveness and enhance student learning. In this paper, we present 26 tips for teaching introductory programming drawn from the experiences of four well-qualified college professors. It is our hope that our peers can pick up some tips from this paper, apply them in their own classroom, improve their teaching effectiveness, and ultimately enhance student learning

On Designing Programming Error Messages for Novices: Readability and its Constituent Factors

The 2021 ACM CHI Virtual Conference on Human Factors in Computing Systems (CHI'21), Virtual Conference, 8-13 May 2021Programming error messages play an important role in learning to program. The cycle of program input and error message response completes a loop between the programmer and the compiler/interpreter and is a fundamental interaction between human and computer. However, error messages are notoriously problematic, especially for novices. Despite numerous guidelines citing the importance of message readability, there is little empirical research dedicated to understanding and assessing it. We report three related experiments investigating factors that influence programming error message readability. In the first two experiments we identify possible factors, and in the third we ask novice programmers to rate messages using scales derived from these factors. We find evidence that several key factors significantly affect message readability: message length, jargon use, sentence structure, and vocabulary. This provides novel empirical support for previously untested long-standing guidelines on message design, and informs future efforts to create readability metrics for programming error messages

Automatically Fixing Syntax Errors Using the Levenshtein Distance

Abstract:To ensure high quality software, much emphasis is laid on software testing. While a number of techniques and tools already exist to identify and locate syntax errors, it is still the duty of programmers to manually fix each of these uncovered syntax errors. In this paper we propose an approach to automate the task of fixing syntax errors by using existing compilers and the levenshtein distance between the identified bug and the possible fixes. The levenshtein distance is a measure of the similarity between two strings. A prototype, called ASBF, has also been built and a number of tests carried out which show that the technique works well in most cases. ASBF is able to automatically fix syntax errors in any erroneous source file and can also process several erroneous files in a source folder. The tests carried out also show that the technique can also be applied to multiple programming languages. Currently ASBF can automatically fix software bugs in the Java and the Python programming languages. The tool also has auto-learning capabilities where it can automatically learn from corrections made manually by a user. It can thereafter couple this learning process with the levenshtein distance to improve its software bugcorrection capabilities.Keywords: Automatically fixing syntax errors, bug fixing, auto-learn, levenshtein distance, Java, Python(Article history: Received 16 September 2016 and accepted 9 December 2016

Would you fix this code for me? Effects of repair source and commenting on trust in code repair

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. Automation and autonomous systems are quickly becoming a more engrained aspect of modern society. The need for effective, secure computer code in a timely manner has led to the creation of automated code repair techniques to resolve issues quickly. However, the research to date has largely ignored the human factors aspects of automated code repair. The current study explored trust perceptions, reuse intentions, and trust intentions in code repair with human generated patches versus automated code repair patches. In addition, comments in the headers were manipulated to determine the effect of the presence or absence of comments in the header of the code. Participants were 51 programmers with at least 3 years’ experience and knowledge of the C programming language. Results indicated only repair source (human vs. automated code repair) had a significant influence on trust perceptions and trust intentions. Specifically, participants consistently reported higher levels of perceived trustworthiness, intentions to reuse, and trust intentions for human referents compared to automated code repair. No significant effects were found for comments in the headers