Increasing the Effectiveness of Error Messages in a Computer Programming and Simulation Tool

abstract: Each programming language has a compiler associated with it which helps to identify logical or syntactical errors in the program. These compiler error messages play important part in the form of formative feedback for the programmer. Thus, the error messages should be constructed carefully, considering the affective and cognitive needs of programmers. This is especially true for systems that are used in educational settings, as the messages are typically seen by students who are novice programmers. If the error messages are hard to understand then they might discourage students from understanding or learning the programming language. The primary goal of this research is to identify methods to make the error messages more effective so that students can understand them better and simultaneously learn from their mistakes. This study is focused on understanding how the error message affects the understanding of the error and the approach students take to solve the error. In this study, three types of error messages were provided to the students. The first type is Default type error message which is an assembler centric error message. The second type is Link type error message which is a descriptive error message along with a link to the appropriate section of the PLP manual. The third type is Example type error message which is again a descriptive error message with an example of the similar type of error along with correction step. All these error types were developed for the PLP assembly language. A think-aloud experiment was designed and conducted on the students. The experiment was later transcribed and coded to understand different approach students take to solve different type of error message. After analyzing the result of the think-aloud experiment it was found that student read the Link type error message completely and they understood and learned from the error message to solve the error. The results also indicated that Link type was more helpful compare to other types of error message. The Link type made error solving process more effective compared to other error types.Dissertation/ThesisMasters Thesis Software Engineering 201

On the Frequency of Words Used in Answers to Explain in Plain English Questions by Novice Programmers

© 2019 Association for Computing Machinery. Most previous research studies using Explain in Plain English questions have focussed on categorising the answers of novice programmers according to the SOLO taxonomy, and/or the relationship between explaining code and writing code. In this paper, we study the words used in the explanations of novice programmers. Our data is from twelve Explain in plain English questions presented to over three hundred students in an exam at the end of the students' first semester of programming. For each question, we compare the frequency of certain words used in correct answers, between students who scored a perfect twelve on all the Explain in plain English questions and students with lower scores. We report a number of statistically significant differences in word frequency between the students who answered all questions correctly and students who did not. The students who answered all twelve questions correctly tended to be more precise, more comprehensive, and more likely to choose words not explicitly in the code, but instead words that are an abstraction beyond the code

Understanding notional machines through traditional teaching with conceptual contraposition and program memory tracing

A correct understanding about how computers run code is mandatory in order to effectively learn to program. Lectures have historically been used in programming courses to teach how computers execute code, and students are assessed through traditional evaluation methods, such as exams. Constructivism learning theory objects to students’ passiveness during lessons, and traditional quantitative methods for evaluating a complex cognitive process such as understanding. Constructivism proposes complimentary techniques, such as conceptual contraposition and colloquies. We enriched lectures of a “Programming II” (CS2) course combining conceptual contraposition with program memory tracing, then we evaluated students’ understanding of programming concepts through colloquies. Results revealed that these techniques applied to the lecture are insufficient to help students develop satisfactory mental models of the C++ notional machine, and colloquies behaved as the most comprehensive traditional evaluations conducted in the course.Universidad de Costa Rica/[]/UCR/Costa RicaMinisterio de Ciencia Tecnología y Telecomunicaciones de Costa Rica/[]/MICITT/Costa RicaUCR::Vicerrectoría de Investigación::Unidades de Investigación::Ingeniería::Centro de Investigaciones en Tecnologías de Información y Comunicación (CITIC)UCR::Vicerrectoría de Docencia::Ingeniería::Facultad de Ingeniería::Escuela de Ciencias de la Computación e Informátic

Transformative and troublesome? Students’ and professional programmers’ perspectives on difficult concepts in programming

Programming skills are an increasingly desirable asset for higher education students across disciplines; however, learning to program continues to be difficult to master for many students. The heterogeneity of student cohorts in most computer science courses makes it challenging to isolate the concepts within programming that are particularly hard for all students to understand (i.e., 'threshold concepts'). As a result, there is a lack of consensus over the threshold concepts that are relevant to those learning programming. This paper draws on a qualitative study using focus groups with undergraduates and professional software developers to identify potential threshold concepts in programming. Data from the focus groups were thematically coded and analyzed using a theoretical framework based on established criteria for threshold concepts. In particular, we focused on concepts that were both troublesome and transformative and included other characteristics such as participants' partiality of understanding. Six potential threshold concepts were identified in the majority of the focus groups, including abstract classes, data structures and designing objects. Further analysis of the data identified additional concepts that may hinder rather than help the learning of these threshold concepts, which we have called 'accidental complexities'

Introductory programming: a systematic literature review

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming. This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research

Multi-level code comprehension model for large scale software, A

1996 Fall.Includes bibliographical references (pages 142-147).For the past 20 years researchers have studied how programmers understand code they did not write. Most of this research has concentrated on small-scale code understanding. We consider it necessary to design studies that observe programmers working on large-scale code in production environments. We describe the design and implementation of such a study which included 11 maintenance engineers working on various maintenance tasks. The objective is to build a theory based on observations of programmers working on real tasks. Results show that programmers understand code at different levels of abstraction. Expertise in the application domain, amount of prior experience with the code, and task can determine the types of actions taken during maintenance, the level of abstraction at which the programmer works, and the information needed to complete a maintenance task. A better grasp of how programmers understand large scale code and what is most efficient and effective can lead to better tools, better maintenance guidelines, and documentation

USO DEL MÉTODO "THINK ALOUD" EN LA INVESTIGACIÓN CUALITATIVA

ResumenEl método "Think-Aloud" ha sido utilizado para entender los procesos cognitivos que no pueden ser observados de manera directa en varias áreas de la investigación cualitativa. Existen varias propuestas de la comunidad de investigación para su uso en las áreas de pedagogía, comunicación del habla, e interacción humano-computadora. En este trabajo se presenta una breve introducción al marco teórico del método así como el estado del arte de su uso en la educación y como técnica de inspección de usabilidad, ofreciendo una semblanza de cómo puede ser integrado en estudios de investigación cualitativa en un futuro.Palabra(s) Clave: Lectura de comprensión, Proceso cognitivo, Think aloud, Usabilidad. USE OF THE "THINK ALOUD" METHOD IN QUALITATIVE RESEARCHAbstractThe Think-Aloud method has been used to understand cognitive processes that cannot be directly observed, in areas where qualitative research is conducted. There are various proposals of the research community to use this method in areas such as pedagogy, psychology, speech communications, and human-computer interaction. In this work we present a brief introduction to the theory behind the method, the state of the art in the areas of education and usability, providing an insight of how this method can be used in the future in qualitative studies.Keywords: Cognitive process, Reading comprehension, Think aloud, Usability

Empirical Studies of Novices Learning Programming

This thesis is concerned with the problems that novices have in learning to program: in particular it is concerned with the difficulties experienced by novices learning at a distance, using instructional materials which have been designed especially for novices. One of the major problems for novices is how to link the new information which they encounter with their existing knowledge. Du Boulay, O'Shea and Monk (1981) suggest helping novices to bridge the gap between their existing knowledge and new information by teaching via a conceptual model, which serves to explain the new information in familiar terms.In this thesis the difficulties which novices have when learning to program with the help of a conceptual model were investigated. The curricula and conceptual models of four different programming languages are examined, all of which were designed to teach novices. Du Boulay, O'Shea and Monk (1981) have suggested criteria for analysing conceptual models. It is argued that these criteria, however, do not address the presentation of the conceptual model, and so are insufficient to evaluate them. An additional form of analysis was proposed and used, in addition to the criteria offered by Du Boulay et al. This is a way of describing the conceptual model which distinguishes three views of the conceptual model: state, procedure and function, and which highlights the different aspects which are important for the novice learner by identifying the different kinds of knowledge which are necessary to understand the conceptual model. This analysis of the conceptual models showed that the environments are not as exemplary as the du Boulay et al's criteria suggest, and indicated that three of the environments, SOLO, PT501 and DESMOND, lack a functional representation, and that the fourth, Open Logo, has other different problems.An empirical study was carried out to study the transfer effects of learning two of the languages, a high level and a low level language, sequentially. There was no evidence for such transfer effects. The difficulties novices have in learning the four different languages were also investigated. These studies show that even though the novices were studying environments designed for novices learning at a distance, they did not develop good levels of competence, and the problems they had fall into two main categories: programming and pedagogical.Although the different languages had different aims and curricula, novices had some problems which were common to all or most of the languages. These included understanding flow of control, developing and using programming plans, developing accurate mental models, and in the high level languages, understanding recursion. It is argued that some of these problems are related to the conceptual models. In particular, the difficulties novices had in developing and using plan knowledge, which is one of their main problems, can be explained by the lack of an appropriate functional description in the languages.The subjects' pedagogical problems arose from the relationship between the style and structure of the curriculum, its content, and the subjects themselves. In all the four texts the teaching material is very carefully structured and it is suggested that this may encourage the learner to adopt an over-dependent attitude towards the text, and in some cases, to work at an inappropriate syntactic level.The relationship between the distance learning situation and the novice programmer is discussed, and recommendations are made for improving the curricula used for teaching novices programming

Usability Testing of an Electronic Patient-Reported Outcome System for Survivors of Critical Illness

BACKGROUND: Web-based electronic patient-reported outcomes (ePRO) measures are increasingly used to facilitate patient-centered health assessments. However, it is unknown if ePRO completion is feasible for recently ill intensive care unit (ICU) survivors and their families. OBJECTIVE: To develop and evaluate the usability of a novel ePRO system (ePRO to Support People and Enhance Recovery [ePROSPER]) among ICU survivors and their families within an ongoing clinical trial. METHODS: Paper-based PROs were iteratively adapted to electronic forms (ePROs). Then, the usability of ePROSPER was assessed among 60 patients, their family members, and PRO and programming experts via questionnaires (eg, Systems Usability Scale), "think aloud" open-ended feedback, task completion times, and error rates. RESULTS: Input from patients and their families was used to incorporate user-experience modifications into ePROSPER. This feedback also led to inclusion of automated reminders for questionnaire completion and real-time alerts for staff triggered by high symptom levels. Median usability scores increased over testing cycles from 40 to 73 to 95, nearing the maximum score and showing excellent usability. All users completed ePROSPER within 20 minutes; 87% preferred it to a written version. ePROSPER was then implemented in a clinical trial without data errors. CONCLUSIONS: Automated ePRO systems can be successfully integrated in a post-ICU clinical trial setting. The value of integrating such systems in direct clinical care should be assessed in future studies