Meaningful Categorisation of Novice Programmer Errors

The frequency of different kinds of error made by students learning to write computer programs has long been of interest to researchers and educators. In the past, various studies investigated this topic, usually by recording and analysing compiler error messages, and producing tables of relative frequencies of specific errors diagnostics produced by the compiler. In this paper, we improve on such prior studies by investigating actual logical errors in student code, as opposed to diagnostic messages produced by the compiler. The actual errors reported here are more precise, more detailed and more accurate than the diagnostic produced automatically

Source code readability improvement using heuristic-based dynamic error reporting during editing

University of Technology, Sydney. Faculty of Engineering.This research considers whether dynamically reporting poor identifier-naming practices at the time when the source code is written can improve readability and hence maintainability. Poor identifier-naming practices have little effect on the production phase of the software lifecycle. However, poor identifier-naming practices can have a substantial impact during the maintenance phase of the software life cycle, particularly for the maintenance of large (i.e., 1M SLOC) computer programs. Of the nineteen identifier-naming style guidelines employed to support the research and used to identity poor identifier-naming practices, thirteen were found to be useful in improving source code readability. A questionnaire was employed to ascertain whether expert programmers accepted these guidelines; a textbook survey was used to identify the potential to transmit poor identifier-naming practices; a survey of contemporary source code was used to ascertain current identifier-naming practices; and a survey of dated source code was used to ascertain how identifier-naming practices have changed over an extended period of time. In addition, a controlled experiment was used to evaluate the effects of poor identifier-naming during a maintenance exercise and to evaluate the generation of poor identifier-naming during a production activity. A novice programmer case study and a programming team case study were executed to identify the longer term effects of dynamically reporting poor identifier-naming practices. The benefit of dynamically reporting poor identifier-naming practices was most pronounced for novice programmers with the percentage of meaningful identifier names increasing from 12% to 28%. The results for expert programmers were less pronounced with the percentage of meaningful identifier names correspondingly increasing from 53% to 60%. The identifier-naming style guidelines that proved to be the most useful to programmers required that identifier names should be composed of from two to four Natural language words or project accepted acronyms; should not be composed only of abstract words; should not contain plural words; and should conform to the project naming conventions

An Exploration Of The Effects Of Enhanced Compiler Error Messages For Computer Programming Novices

Computer programming is an essential skill that all computing students must master and is increasingly important in many diverse disciplines. It is also difficult to learn. One of the many challenges novice programmers face from the start are notoriously cryptic compiler error messages. These report details on errors made by students and are essential as the primary source of information used to rectify those errors. However these difficult to understand messages are often a barrier to progress and a source of discouragement. A high number of student errors, and in particular a high frequency of repeated errors – when a student makes the same error consecutively – have been shown to be indicators of students who are struggling with learning to program. This instrumental case study research investigates the student experience with, and the effects of, software that has been specifically written to help students overcome their challenges with compiler error messages. This software provides help by enhancing error messages, presenting them in a straightforward, informative manner. Two cohorts of first year computing students at an Irish higher education institution participated over two academic years; a control group in 2014-15 that did not experience enhanced error messages, and an intervention group in 2013-14 that did. This thesis lays out a comprehensive view of the student experience starting with a quantitative analysis of the student errors themselves. It then views the students as groups, revealing interesting differences in error profiles. Following this, some individual student profiles and behaviours are investigated. Finally, the student experience is discovered through their own words and opinions by means of a survey that incorporated closed and open-ended questions. In addition to reductions in errors overall, errors per student, and the key metric of repeated error frequency, the intervention group is shown to behave more cohesively with fewer indications of struggling students. A positive learning experience using the software is reported by the students and the lecturer. These results are of interest to educators who have witnessed students struggle with learning to program, and who are looking to help remove the barrier presented by compiler error messages. This work is important for two reasons. First, the effects of error message enhancement have been debated in the literature – this work provides evidence that there can be positive effects. Second, these results should be generalisable at least in part, to other languages, students and institutions

Heuristic Evaluation for Novice Programming Systems

The past few years has seen a proliferation of novice programming tools. The availability of a large number of systems has made it difficult for many users to choose among them. Even for education researchers, comparing the relative quality of these tools, or judging their respective suitability for a given context, is hard in many instances. For designers of such systems, assessing the respective quality of competing design decisions can be equally difficult. Heuristic evaluation provides a practical method of assessing the quality of alternatives in these situations and of identifying potential problems with existing systems for a given target group or context. Existing sets of heuristics, however, are not specific to the domain of novice programming and thus do not evaluate all aspects of interest to us in this specialised application domain. In this article, we propose a set of heuristics to be used in heuristic evaluations of novice programming systems. These heuristics have the potential to allow a useful assessment of the quality of a given system with lower cost than full formal user studies and greater precision than the use of existing sets of heuristics. The heuristics are described and discussed in detail. We present an evaluation of the effectiveness of the heuristics that suggests that the new set of heuristics provides additional useful information to designers not obtained with existing heuristics sets

The Cost of Syntax and How To Avoid It: Text versus Frame-Based Editing

Plain text has always been the predominant medium for writing and editing programs for expert users. Text is powerful and flexible, but requires more careful manipulation than structural editors, such as those found in block-based environments. In addition, in textual editors programmers are responsible for managing detailed orthography and layout – when beginners work with text, significant time is spent managing syntax problems, indentation and spacing. Frame-based editing is a new editing paradigm that combines the structural editing of block-based systems with the flexibility and keyboard-focus of text editing. In this paper, we empirically examine how much time and effort is spent by beginners on managing syntax errors and indentation, which can be automatically saved by switching to frame-based editing. The data is obtained using the Blackbox dataset; the results predict a clear advantage of frame-based editing over traditional text editors

Novice Programmer Errors - Analysis and Diagnostics

All programmers make errors when writing program code, and for novices the difficulty of repairing errors can be frustrating and demoralising. It is widely recognised that compiler error diagnostics can be inaccurate, imprecise, or otherwise difficult for novices to comprehend, and many approaches to mitigating the difficulty of dealing with errors are centered around the production of diagnostic messages with improved accuracy and precision, and revised wording considered more suitable for novices. These efforts have shown limited success, partially due to uncertainty surrounding the types of error that students actually have the most difficulty with - which has most commonly been assessed by categorising them according to the diagnostic message already produced - and a traditional approach to the error diagnosis process which has known limitations. In this thesis we detail a systematic and thorough approach both to analysing which errors that are most problematic for students, and to automated diagnosis of errors. We detail a methodology for developing a category schema for errors and for classifying individual errors in student programs according to such a schema. We show that this classification results in a different picture of the distribution of error types when compared to a classification according to diagnostic messages. We formally define the severity of an error type as a product of its frequency and difficulty, and by using repair time as an indicator of difficulty we show that error types rank differently via severity than they do by frequency alone. Having developed a ranking of errors according to severity, we then investigate the contextual information within source code that experienced programmers can use to more accurately and precisely classify errors than compiler tools typically do. We show that, for a number of more severe errors, these techniques can be applied in an automated tool to provide better diagnostics than are provided by traditional compilers

Learning Dimensions: Lessons from Field Studies

In this paper, we describe work to investigate the creation of engaging programming learning experiences. Background research informed the design of four fieldwork studies involving a range of age groups to explore how programming tasks could best be framed to motivate learners. Our empirical findings from these four studies, described here, contributed to the design of a set of programming "Learning Dimensions" (LDs). The LDs provide educators with insights to support key design decisions for the creation of engaging programming learning experiences. This paper describes the background to the identification of these LDs and how they could address the design and delivery of highly engaging programming learning tasks. A web application has been authored to support educators in the application of the LDs to their lesson design

Knowledge restructing and the development of expertise in computer programming

This thesis reports a number of empirical studies exploring the development of expertise in computer programming. Experiments 1 and 2 are concerned with the way in which the possession of design experience can influence the perception and use of cues to various program structures. Experiment 3 examines how violations to standard conventions for constructing programs can affect the comprehension of expert, intermediate and novice subjects. Experiment 4 looks at the differences in strategy that are exhibited by subjects of varying skill level when constructing programs in different languages. Experiment 5 takes these ideas further to examine the temporal distribution of different forms of strategy during a program generation task. Experiment 6 provides evidence for salient cognitive structures derived from reaction time and error data in the context of a recognition task. Experiments 7 and 8 are concerned with the role of working memory in program generation and suggest that one aspect of expertise in the programming domain involves the acquisition of strategies for utilising display-based information. The final chapter attempts to bring these experimental findings together in terms of a model of knowledge organisation that stresses the importance of knowledge restructuring processes in the development of expertise. This is contrasted with existing models which have tended to place emphasis upon schemata acquisition and generalisation as the fundamental modes of learning associated with skill development. The work reported here suggests that a fine-grained restructuring of individual schemata takes places during the later stages of skill development. It is argued that those mechanisms currently thought to be associated with the development of expertise may not fully account for the strategic changes and the types of error typically found in the transition between novice, intermediate and expert problem solvers. This work has a number of implications for existing theories of skill acquisition. In particular, it questions the ability of such theories to account for subtle changes in the various manifestations of skilled performance that are associated with increasing expertise. Secondly, the work reported in this thesis attempts to show how specific forms of training might give rise to the knowledge restructuring process that is proposed. Finally, the thesis stresses the important role of display-based problem solving in complex tasks such as programming and highlights the role of programming language notation as a mediating factor in the development and acquisition of problem solving strategies

Increasing the Effectiveness of Error Messages in a Computer Programming and Simulation Tool

abstract: Each programming language has a compiler associated with it which helps to identify logical or syntactical errors in the program. These compiler error messages play important part in the form of formative feedback for the programmer. Thus, the error messages should be constructed carefully, considering the affective and cognitive needs of programmers. This is especially true for systems that are used in educational settings, as the messages are typically seen by students who are novice programmers. If the error messages are hard to understand then they might discourage students from understanding or learning the programming language. The primary goal of this research is to identify methods to make the error messages more effective so that students can understand them better and simultaneously learn from their mistakes. This study is focused on understanding how the error message affects the understanding of the error and the approach students take to solve the error. In this study, three types of error messages were provided to the students. The first type is Default type error message which is an assembler centric error message. The second type is Link type error message which is a descriptive error message along with a link to the appropriate section of the PLP manual. The third type is Example type error message which is again a descriptive error message with an example of the similar type of error along with correction step. All these error types were developed for the PLP assembly language. A think-aloud experiment was designed and conducted on the students. The experiment was later transcribed and coded to understand different approach students take to solve different type of error message. After analyzing the result of the think-aloud experiment it was found that student read the Link type error message completely and they understood and learned from the error message to solve the error. The results also indicated that Link type was more helpful compare to other types of error message. The Link type made error solving process more effective compared to other error types.Dissertation/ThesisMasters Thesis Software Engineering 201

Threshold concepts and teaching programming

This thesis argues that the urge to build and the adoption of a technocratic disposition have influenced and affected the pursuit and development of a deeper understanding of the discipline of computing and its pedagogy. It proposes the introduction to the discipline of the threshold concept construct to improve both the understanding and the pedagogy. The research examines the threshold concept construct using the theory of concepts. The examination establishes the conceptual coherence of the features attributed to threshold concepts and formalises the basis for threshold concept scholarship. It also provides a refutation for critiques of threshold concepts. The examination reveals the inextricable links between threshold concepts and pedagogic content knowledge. Both rely on the expertise of reflective pedagogues and are situated at the site of student learning difficulties and their encounters with troublesome knowledge. Both have deep understanding of discipline content knowledge at their centre. The two ideas are mutually supportive. A framework for identifying threshold concepts has been developed. The framework uses an elicitation instrument grounded in pedagogic content knowledge and an autoethnographic approach. The framework is used to identify state as a threshold concept in computing. The significant results of the research are two-fold. First, the identification of state as a threshold concept provides an insight into the disparate difficulties that have been persistently reported in the computer science education literature as stumbling blocks for novice programmers and enhances and develops the move towards discipline understanding and teaching for understanding. Second, the embryonic research area of threshold concept scholarship has been provided with a theoretical framework that can act as an organising principle to explicate existing research and provide a coherent focus for further research