Developing Beginner-Friendly Programming Error Messages

The motivation for our work is to introduce a recently developed programming language, Clojure, in a beginner computer science (CSci) class at the University of Minnesota, Morris. Clojure is an industryaccepted programming language that provides significant benefits for beginner programmers, such as focus on a functional approach to programming which, in UMM experience, provides a good foundation for subsequent CSci curriculum. Learning Clojure in an introductory class opens opportunities for students to collaborate on numerous worldwide projects, as well as take advantage of improvements in modern computing hardware. However, Clojure is challenging to use because of its complicated handling of programmers’ mistakes. Mistakes in computer programming are a natural part of developing software. When a mistake happens, there is a system to notify the programmer of an error. The specific information that the programmer receives, known as an error message, may or may not be helpful in identifying the issue. Clojure error messages are notorious for being confusing to beginners. We are developing a system that intercepts the existing Clojure error messages and automatically rephrases them for beginner programmers. We will conduct usability tests by observing the interactions between beginner programmers and our system, and the feedback we receive will be used to further improve our project. We present our new error message handling and discuss testing our system with new programmers.https://digitalcommons.morris.umn.edu/urs_2015/1005/thumbnail.jp

Functional Baby Talk: Analysis of Code Fragments from Novice Haskell Programmers

What kinds of mistakes are made by novice Haskell developers, as they learn about functional programming? Is it possible to analyze these errors in order to improve the pedagogy of Haskell? In 2016, we delivered a massive open online course which featured an interactive code evaluation environment. We captured and analyzed 161K interactions from learners. We report typical novice developer behavior; for instance, the mean time spent on an interactive tutorial is around eight minutes. Although our environment was restricted, we gain some understanding of Haskell novice errors. Parenthesis mismatches, lexical scoping errors and do block misunderstandings are common. Finally, we make recommendations about how such beginner code evaluation environments might be enhanced

Report on a User Test and Extension of a Type Debugger for Novice Programmers

A type debugger interactively detects the expressions that cause type errors. It asks users whether they intend the types of identifiers to be those that the compiler inferred. However, it seems that novice programmers often get in trouble when they think about how to fix type errors by reading the messages given by the type debugger. In this paper, we analyze the user tests of a type debugger and report problems of the current type debugger. We then extend the type debugger to address these problems. Specifically, we introduce expression-specific error messages and language levels. Finally, we show type errors that we think are difficult to explain to novice programmers. The subjects of the user tests were 40 novice students belonging to the department of information science at Ochanomizu University.Comment: In Proceedings TFPIE 2014, arXiv:1412.473

Calico: a multi-programming-language, multi-context framework designed for computer science education

The Calico project is a multi-language, multi-context programming framework and learning environment for computing education. This environment is designed to support several interoperable programming languages (including Python, Scheme, and a visual programming language), a variety of pedagogical contexts (including scientific visualization, robotics, and art), and an assortment of physical devices (including different educational robotics platforms and a variety of physical sensors). In addition, the environment is designed to support collaboration and modern, interactive learning. In this paper we describe the Calico project, its design and goals, our prototype system, and its current use

Do Enhanced Compiler Error Messages Help Students? Results Inconclusive.

One common frustration students face when first learning to program in a compiled language is the difficulty in interpreting the compiler error messages they receive. Attempts to improve error messages have produced differing results. Two recently published papers showed conflicting results, with one showing measurable change in student behavior, and the other showing no measurable change. We conducted an experiment comparable to these two over the course of several semesters in a CS1 course. This paper presents our results in the context of previous work in this area. We improved the clarity of the compiler error messages the students receive, so that they may more readily understand their mistakes and be able to make effective corrections. Our goal was to help students better understand their syntax mistakes and, as a reasonable measure of our success, we expected to document a decrease in the number of times students made consecutive submissions with the same compilation error. By doing this, we could demonstrate that this enhancement is effective. After collecting and thoroughly analyzing our own experimental data, we found that—despite anecdotal stories, student survey responses, and instructor opinions testifying to the tool’s helpfulness— enhancing compiler error messages shows no measurable benefit to students. Our results validate one of the existing studies and contradict another. We discuss some of the reasons for these results and conclude with projections for future research

Usability of Error Messages for Introductory Students

Error messages are an important tool programmers use to help find and fix mistakes or issues in their code. When an error message is unhelpful, it can be difficult to find the issue and may impose additional challenges in learning the language and concepts. Error messages are especially critical for introductory programmers in understanding problems with their code. Unfortunately, not all error messages in programming are beneficial for novice programmers. This paper discusses the general usability of error messages for introductory programmers, analyses of error messages in compilers and DrRacket, and two methodologies intended to improve error handling

Effective Compiler Error Message Enhancement for Novice Programming Students

Programming is an essential skill that all computing students must master. However programming can be difficult to learn. Compiler error messages are crucial for correcting errors, but are often difficult to understand and pose a barrier to progress for many novices. High frequencies of errors, particularly repeated errors, have been shown to be indicators of students who are struggling with learning to program. This study involves a custom IDE that enhances Java compiler error messages, intended to be more useful to novices than those supplied by the compiler. The effectiveness of this approach was tested in an empirical control/intervention study of approximately 200 students generating almost 50,000 errors. The design allows for direct comparisons between enhanced and non-enhanced error messages. Results show that the intervention group experienced reductions in the number of overall errors, errors per student, and several repeated error metrics. This work is important for two reasons. First, the effects of error message enhancement have been recently debated in the literature. This study provides substantial evidence that it can be effective. Second, these results should be generalizable at least in part, to other programming languages, students and institutions, as we show that the control group of this study is comparable to several others using Java and other languages

Python Error Messages in Basic Level Programming Studies

Lühikokkuvõte:Magistritöö raames viidi läbi uuring programmeerimiskeele Python veateadetest programmeerimise algõppes ja koostati õppematerjal vastavalt uuringus saadud tulemustele. Tartu Ülikooli vaba juurdepääsuga programmeerimisteemalistel e-kursustel osaleb tuhandeid inimesi, kes lahendavad e-kursuste raames kümneid ülesandeid. Hetkel on program-meerimiskeele Python veateated ingliskeelsed, lühidalt kirjeldatud ja sisaldavad palju tehnilisi detaile, mistõttu on need algajatele programmeerijatele keeruline mõista. Käesolev töö keskendubki veateadete uurimisele ja lahti seletamisele, et algõppe programmeerimise e-kursuslased neid paremini mõistaksid ja seega ka enda programmis tekkinud vigu parandada oskaksid. Läbiviidava uuringu raames koguti tagasisidet e-kursuse Programmeerimisest maalähedaselt osalejatelt programmeerimiskeele Python sisseehitatud kompilaatori veateadete kohta küsitluse ja töövahendina kasutusel oleva programmeerimiskeskkonna Thonny logifailide näol. Kogutud andmete põhjal viidi läbi analüüs ja valiti välja sagedamini esinenud veateated, mille kohta koostati eestikeelne õppematerjal. Materjal koosneb 7 levinumast vea-tüübist, kus omakorda 29 levinumat veateadet on seletatud lahti eesti keeles ja neile on lisatud 35 programminäidet koos võimalike parandusviisidega. Veateadete õppematerjali saab kasutada programmeerimisteemaliste e-kursuste juures abimaterjalina erinevate ülesannete lahendamisel.Abstract:This thesis describes the study of programming language Python error messages in basic level programming studies, which results were the source to an educational material. Thousands of people are taking part in online courses provided by the University of Tartu, where tens of assignments needs to be completed. By default the Python error messages are in English, they are described shortly and consist of many technical details, which makes them difficult to understand for basic level programmers. This master’s thesis focuses on studying Python error messages to explain and simplify them to novice programming learners to develop better understanding of error messages. In order to achieve that a valuable feedback was gathered from Programmeerimisest maalähedaselt online course by conducting a survey and collecting integrated development environment Thonny log files. The most frequently occurred error messages from those log files were extracted and combined with survey results they were a source to an educational material in Estonian. This material includes 7 most common error types and 29 error messages illustrated by 35 programming examples so it can be used in programming online courses as an auxiliary educational material to help novice programmers to complete various assignments

Effects of Error Messages on a Student’s Ability to Understand and Fix Programming Errors

abstract: Assemblers and compilers provide feedback to a programmer in the form of error messages. These error messages become input to the debugging model of the programmer. For the programmer to fix an error, they should first locate the error in the program, understand what is causing that error, and finally resolve that error. Error messages play an important role in all three stages of fixing of errors. This thesis studies the effects of error messages in the context of teaching programming. Given an error message, this work investigates how it effects student’s way of 1) understanding the error, and 2) fixing the error. As part of the study, three error message types were developed – Default, Link and Example, to better understand the effects of error messages. The Default type provides an assembler-centric single line error message, the Link type provides a program-centric detailed error description with a hyperlink for more information, and the Example type provides a program centric detailed error description with a relevant example. All these error message types were developed for assembly language programming. A think aloud programming exercise was conducted as part of the study to capture the student programmer’s knowledge model. Different codes were developed to analyze the data collected as part of think aloud exercise. After transcribing, coding, and analyzing the data, it was found that the Link type of error message helped to fix the error in less time and with fewer steps. Among the three types, the Link type of error message also resulted in a significantly higher ratio of correct to incorrect steps taken by the programmer to fix the error.Dissertation/ThesisMasters Thesis Software Engineering 201