An Exploration Of The Effects Of Enhanced Compiler Error Messages For Computer Programming Novices

Computer programming is an essential skill that all computing students must master and is increasingly important in many diverse disciplines. It is also difficult to learn. One of the many challenges novice programmers face from the start are notoriously cryptic compiler error messages. These report details on errors made by students and are essential as the primary source of information used to rectify those errors. However these difficult to understand messages are often a barrier to progress and a source of discouragement. A high number of student errors, and in particular a high frequency of repeated errors – when a student makes the same error consecutively – have been shown to be indicators of students who are struggling with learning to program. This instrumental case study research investigates the student experience with, and the effects of, software that has been specifically written to help students overcome their challenges with compiler error messages. This software provides help by enhancing error messages, presenting them in a straightforward, informative manner. Two cohorts of first year computing students at an Irish higher education institution participated over two academic years; a control group in 2014-15 that did not experience enhanced error messages, and an intervention group in 2013-14 that did. This thesis lays out a comprehensive view of the student experience starting with a quantitative analysis of the student errors themselves. It then views the students as groups, revealing interesting differences in error profiles. Following this, some individual student profiles and behaviours are investigated. Finally, the student experience is discovered through their own words and opinions by means of a survey that incorporated closed and open-ended questions. In addition to reductions in errors overall, errors per student, and the key metric of repeated error frequency, the intervention group is shown to behave more cohesively with fewer indications of struggling students. A positive learning experience using the software is reported by the students and the lecturer. These results are of interest to educators who have witnessed students struggle with learning to program, and who are looking to help remove the barrier presented by compiler error messages. This work is important for two reasons. First, the effects of error message enhancement have been debated in the literature – this work provides evidence that there can be positive effects. Second, these results should be generalisable at least in part, to other languages, students and institutions

Effective Compiler Error Message Enhancement for Novice Programming Students

Programming is an essential skill that all computing students must master. However programming can be difficult to learn. Compiler error messages are crucial for correcting errors, but are often difficult to understand and pose a barrier to progress for many novices. High frequencies of errors, particularly repeated errors, have been shown to be indicators of students who are struggling with learning to program. This study involves a custom IDE that enhances Java compiler error messages, intended to be more useful to novices than those supplied by the compiler. The effectiveness of this approach was tested in an empirical control/intervention study of approximately 200 students generating almost 50,000 errors. The design allows for direct comparisons between enhanced and non-enhanced error messages. Results show that the intervention group experienced reductions in the number of overall errors, errors per student, and several repeated error metrics. This work is important for two reasons. First, the effects of error message enhancement have been recently debated in the literature. This study provides substantial evidence that it can be effective. Second, these results should be generalizable at least in part, to other programming languages, students and institutions, as we show that the control group of this study is comparable to several others using Java and other languages

Introductory programming: a systematic literature review

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming. This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research

The evaluation of a pedagogical-program development environment for Novice programmers : a comparative study

It is an acknowledged fact that many novice programmers experience difficulty in the process of learning to program. One of the contributing factors to this difficulty is the Program Development Environment (PDE). Professional-PDEs are those developed specifically for professional programmers, but are often used by educational institutions in the instruction of programming. It has long been accepted that such environments are inappropriate in the instruction of programming due to unnecessary complexity and lack of support for novice programmers in the learning process. Numerous pedagogical-PDEs supporting the mechanics of programming have been developed in response to this. A review of literature, however, indicates that very limited empirical studies comparing pedagogical-PDEs and professional-PDEs have been conducted. The current study investigates whether there are measurable benefits to using a pedagogical-PDE supporting the mechanics of programming in the instruction of programming instead of a professional-PDE. A comparative study of this nature requires a representative pedagogical-PDE and representative professional-PDE be compared with one another. The first part of the current study determines a set of requirements that a pedagogical- PDE should adhere to based on literature. A set of representative features for a pedagogical-PDE is derived by examining the features of existing PDEs in conjunction with the set of requirements. Based on these features, a pedagogical-PDE, known as SimplifIDE, is developed that implements the representative set of features and that meets are the requirements for a pedagogical-PDE. The second part of the current study is the specification and administration of an empirical experiment in which SimplifIDE and Borland© DelphiTM are compared with one another. A holistic approach in determining the differences between the PDEs is taken and three main areas are examined, namely academic performance, perceptions and programming behavior

Toward Productivity Improvements in Programming Languages Through Behavioral Analytics

Computer science knowledge and skills have become foundational for success in virtually every professional field. As such, productivity in programming and computer science education is of paramount economic and strategic importance for innovation, employment and economic growth. Much of the research around productivity and computer science education has centered around improving notoriously difficult compiler error messages, with a noted surge in new studies in the last decade. In developing an original research plan for this area, this dissertation begins with an examination of the Case for New Instrumentation, draw- ing inspiration from automated data mining innovations and corporate marketing techniques in behavioral analytics as a model for understanding and prediction of human behavior. This paper then develops and explores techniques for automated measurement of programmer behavior based on token level lexical analysis of computer code. The techniques are applied in two empirical studies on parallel programming tasks with 88 and 91 student participants from the University of Nevada, Las Vegas as well as 108,110 programs from a database code repository. In the first study, through a re-analysis of previously captured data, the token accuracy mapping technique provided direct insight into the root cause for observed performance differences comparing thread-based vs. process-oriented parallel programming paradigms. In the second study com- paring two approaches to GPU programming at different levels of abstraction, we found that students who completed programming tasks in the CUDA paradigm (considered a lower level abstraction) performed at least equal to or better than students using the Thrust library (a higher level of abstraction) across four different abstraction tests. The code repository of programs with compiler errors was gathered from an online programming interface on curriculum pages available in the Quorum language (quorumlanguage.com) for Code.org’s Hour of Code, Quorum’s Common Core-mapped curriculum, activities from Girls Who Code and curriculum for Skynet Junior Scholars for a National Science Foundation funded grant entitled Inno- vators Developing Accessible Tools for Astronomy (IDATA). A key contribution of this research project is the development of a novel approach to compiler error categorization and hint generation based on token patterns called the Token Signature Technique. Token Signature analysis occurs as a post-processing step after a compilation pass with an ANTLR LL* parser triggers and categorizes an error. In this project, we use this technique to i.) further categorize and measure the root causes of the most common compiler errors in the Quorum database and then ii.) serve as an analysis tool for the development of a rules engine for enhancing compiler errors and providing live hint suggestions to programmers. The observed error patterns both in the overall error code categories in the Quorum database and in the specific token signatures within each error code category show error concentration patterns similar to other compiler error studies of the Java and Python programming languages, suggesting a potentially high impact of automated error messages and hints based on this technique. The automated nature of token signature analysis also lends itself to future development with sophisticated data mining technologies in the areas of machine learning, search, artificial intelligence, databases and statistics

How to provide automated feedback helping students with negative semantic transfer when learning a second programming language

Earlier studies have shown that when students see matching syntax across programming languages, they believe the semantics will match. Typically this is true, but occasionally the syntax between two languages is similar while the semantics are different. Given that the syntax in Java is correct, the code will compile with no warnings, and the cause of the error can take a longer time to find and be harder to correct. This thesis collects six semantic errors in Java with no preexisting error message that might be problematic for a student when transferring from Python to Java. We aim to find out if the errors are a problem for the students and uncover that current environments lack feedback we believe is beneficial for novice Java students. We develop a tool, Uncoil, to detect the errors and provide an error message to fill this gap. Seven novice students in Java with previous Python knowledge tried to solve the errors and evaluated Uncoil in a mixed method study. Our results indicate that novice Java students need help with some of the errors earlier in the semester but do not transfer the semantics from Python to Java later in the semester. At the time of the study, few students needed Uncoil to solve the errors, but especially the weaker students found it helpful.Masteroppgave i Programutvikling samarbeid med HVLPROG399MAMN-PRO

On Designing Programming Error Messages for Novices: Readability and its Constituent Factors

The 2021 ACM CHI Virtual Conference on Human Factors in Computing Systems (CHI'21), Virtual Conference, 8-13 May 2021Programming error messages play an important role in learning to program. The cycle of program input and error message response completes a loop between the programmer and the compiler/interpreter and is a fundamental interaction between human and computer. However, error messages are notoriously problematic, especially for novices. Despite numerous guidelines citing the importance of message readability, there is little empirical research dedicated to understanding and assessing it. We report three related experiments investigating factors that influence programming error message readability. In the first two experiments we identify possible factors, and in the third we ask novice programmers to rate messages using scales derived from these factors. We find evidence that several key factors significantly affect message readability: message length, jargon use, sentence structure, and vocabulary. This provides novel empirical support for previously untested long-standing guidelines on message design, and informs future efforts to create readability metrics for programming error messages

Exploring student perceptions about the use of visual programming environments, their relation to student learning styles and their impact on student motivation in undergraduate introductory programming modules

My research aims to explore how students perceive the usability and enjoyment of visual/block-based programming environments (VPEs), to what extent their learning styles relate to these perceptions and finally to what extent these tools facilitate student understanding of basic programming constructs and impact their motivation to learn programming

Programação em duplas: estado da arte

Resumo: Programação em Duplas (Pair Programming – PP) é uma prática colaborativa de desenvolvimento de software em que dois programadores trabalham ao mesmo tempo em um único computador e na mesma tarefa de programação. Foi relatado na literatura que o conhecimento sobre PP encontra-se disperso e desorganizado. Com o intuito de colocar um pouco de ordem a esse caos, o presente estudo realizou uma busca exaustiva de trabalhos sobre PP em algumas das bibliotecas digitais mais importantes do mundo em Ciência da Computação (Sociedade Brasileira de Computação, ACM, IEEE Explore, Springer, CiteSeer e ScienceDirect, entre outras) e no Google/Scholar. A partir da completa leitura dos trabalhos encontrados, procurou-se definir temas chave dentro da área descrevendo todos os estudos que se relacionam com cada tema. Os achados são interessantes e extensos – eles podem ser encontrados durante toda a leitura do presente artigo