On the Frequency of Words Used in Answers to Explain in Plain English Questions by Novice Programmers

© 2019 Association for Computing Machinery. Most previous research studies using Explain in Plain English questions have focussed on categorising the answers of novice programmers according to the SOLO taxonomy, and/or the relationship between explaining code and writing code. In this paper, we study the words used in the explanations of novice programmers. Our data is from twelve Explain in plain English questions presented to over three hundred students in an exam at the end of the students' first semester of programming. For each question, we compare the frequency of certain words used in correct answers, between students who scored a perfect twelve on all the Explain in plain English questions and students with lower scores. We report a number of statistically significant differences in word frequency between the students who answered all questions correctly and students who did not. The students who answered all twelve questions correctly tended to be more precise, more comprehensive, and more likely to choose words not explicitly in the code, but instead words that are an abstraction beyond the code

Multilingual investigation of theory-based intervention for program comprehension

This thesis is the continuation of an experiment called “Eye-movement Modeling Examples in Source Code Comprehension: A Classroom Study”. This first experiment studies how effective is showing novice programmers how experts read code with a video with the expert’s gaze guided by a verbal explanation. Therefore, this thesis studies, using a similar experiment, whether only verbal explanation and visual stimuli without the expert’s gaze could be also helpful for the programming novices.Grado en Ingeniería Informática de Servicios y Aplicacione

Learning to communicate computationally with Flip: a bi-modal programming language for game creation

Teaching basic computational concepts and skills to school children is currently a curricular focus in many countries. Running parallel to this trend are advances in programming environments and teaching methods which aim to make computer science more accessible, and more motivating. In this paper, we describe the design and evaluation of Flip, a programming language that aims to help 11–15 year olds develop computational skills through creating their own 3D role-playing games. Flip has two main components: 1) a visual language (based on an interlocking blocks design common to many current visual languages), and 2) a dynamically updating natural language version of the script under creation. This programming-language/natural-language pairing is a unique feature of Flip, designed to allow learners to draw upon their familiarity with natural language to “decode the code”. Flip aims to support young people in developing an understanding of computational concepts as well as the skills to use and communicate these concepts effectively. This paper investigates the extent to which Flip can be used by young people to create working scripts, and examines improvements in their expression of computational rules and concepts after using the tool. We provide an overview of the design and implementation of Flip before describing an evaluation study carried out with 12–13 year olds in a naturalistic setting. Over the course of 8 weeks, the majority of students were able to use Flip to write small programs to bring about interactive behaviours in the games they created. Furthermore, there was a significant improvement in their computational communication after using Flip (as measured by a pre/post-test). An additional finding was that girls wrote more, and more complex, scripts than did boys, and there was a trend for girls to show greater learning gains relative to the boys

What does this Python code do?: An exploratory analysis of novice students’ code explanations

Motivation. Code reading skills are important for comprehension. Explain-in-plain-English tasks (EiPE) are one type of reading exercises that show promising results on the ability of such exercises to differentiate between particular levels of code comprehension. Code reading/explaining skills also correlate with code writing skills. Objective. This paper aims to provide insight in what novice students express in their explanations after reading a piece of code, and what these insights can tell us about how the students comprehend code. Method. We performed an exploratory analysis on four reading assignments extracted from a university-level beginners course in Python programming. We paid specific attention to 1) the core focus of student answers, 2) elements of the code that are often included or omitted, and 3) errors and misconceptions students may present. Results. We found that students prioritize the output that is generated by print-statements in a program. This is indication that these statements may have the ability to aid students make sense of code. Furthermore, students appear to be selective about which elements they find important in their explanation. Assigning variables and asking input was less often included, whereas control-flow elements, print statements and function definitions were more often included. Finally, students were easily confused or distracted by lines of code that seemed to interfere with the newly learned programming constructs. Also domain knowledge (outside of programming) both positively and negatively interfered with reading and interpreting the code. Discussion. Our results pave the way towards a better understanding of how students understand code by reading and of how an exercise containing self-explanations after reading, as a teaching instrument, may be useful to both teachers and students in programming education.Computer Systems, Imagery and Medi

Comparing Small Programs for Equivalence: A Code Comprehension Task for Novice Programmers

Novice programmers should develop program comprehension skills as they learn to code so that they are able both to read and reason about code created by others, and to reflect on their code when writing, debugging or extending it. This work takes a little-explored perspective on the comprehension of small programs by asking students to decide if two code segments are equivalent or not in terms of carrying out the same computation. A variation of Euclid's algorithm, that extends the greatest common divisor calculation to more than two numbers, was chosen for this work, as it has an adequate level of complexity and its semantics are not obvious. Four program transformations of the original code were developed: two transformations were equivalent and two were not. 73.5% of students were able to identify correctly the four options and 75.5% provided good insights on the equivalent program flow to justify their choices. The overall task has a SOLO mean of 3.19, which indicates code equivalence is a suitable and approachable task to analyse program execution at novice level. In addition, the data analysis suggests that students' code-reading abilities beyond basic tracing may be generally underestimated and we should investigate how to bridge the potential gap between reasoning about program execution and extracting its purpose

Fostering Program Comprehension in Novice Programmers - Learning Activities and Learning Trajectories

This working group asserts that Program Comprehension (ProgComp) plays a critical part in the process of writing programs. For example, this paper is written from a basic draft that was edited and revised until it clearly presented our idea. Similarly, a program is written incrementally, with each step tested, debugged and extended until the program achieves its goal. Novice programmers should develop program comprehension skills as they learn to code so that they are able both to read and reason about code created by others, and to reflect on their code when writing, debugging or extending it. To foster such competencies our group identified two main goals: (g1) to collect and define learning activities that explicitly address key components of program comprehension and (g2) to define tentative theoretical learning trajectories that will guide teachers as they select and sequence those learning activities in their CS0/CS1/CS2 or K-12 courses. The WG has completed the first goal and laid down a strong foundation towards the second goal as presented in this report. After a thorough literature review, a detailed description of the Block Model is provided, as this model has been used with a dual purpose, to classify and present an extensive list of ProgComp tasks, and to describe a possible learning trajectory for a complex task, covering different cells of the Block Model matrix. The latter is intended to help instructors to decompose complex tasks and identify which aspects of ProgComp are being fostered

Task-related models for teaching and assessing iteration learning in high school

A number of studies report about students’ difficulties with basic flow-control constructs, and specifically with iteration. Although such issues are less explored in the context of pre-tertiary education, this seems to be especially the case for high-school programming learning, where the difficulties concern both the “mechanical” features of the notional machine as well as the logical aspects connected with the constructs, ranging from the implications of loop conditions to a more abstract grasp of the underlying algorithms. For these reasons, the aim of this work is to: i) identifying methodological tools to enhance a comprehensive understanding of the iteration constructs, ii) suggest strategies to teach iterations. We interviewed 20 experienced upper secondary teachers of introductory programming in different kinds of schools. The interviews were mainly aimed at ascertaining teachers’ beliefs about major sources of issues for basic programming concepts and their approach to the teaching and learning of iteration constructs. Once teachers’ perception of students’ difficulties have been identified, we have submitted, to a sample of 164 students, a survey which included both questions on their subjective perception of difficulty and simple tasks probing their understanding of iteration. Data collected from teachers and students confirm that iteration is a central programming concept and indicate that the treatment of conditions and nested constructs are major sources of students’ difficulties with iteration. The interviews allowed us to identify a list of problems that are typically presented by teachers to explain the iterations. Hence, a catalogue of significant program examples has been built to support students’ learning, tasks with characteristics different from those typically presented in class. Based on the outcome of previous steps, a survey to collect related information and good practices from a larger sample of teachers has been designed. Data collected have been analysed distinguishing an orientation towards more conceptual objectives, and one towards more practical objectives. Furthermore, regarding evaluation, a orientation focused on process-based assessment and another on product-based assessment. Finally, based on the outcome of previous students’ survey and drawing from the proposed examples catalogue, we have designed and submitted a new students’ survey, composed of a set of small tasks, or tasklets, to investigate in more depth on high-school students’ understanding of iteration in terms of code reading abilities. The chosen tasklets covered the different topics: technical program feature, correlation between tracing effort and abstraction, the role of flow-charts, students’ perception of self-confidence concerning high-level thinking skills.A number of studies report about students’ difficulties with basic flow-control constructs, and specifically with iteration. Although such issues are less explored in the context of pre-tertiary education, this seems to be especially the case for high-school programming learning, where the difficulties concern both the “mechanical” features of the notional machine as well as the logical aspects connected with the constructs, ranging from the implications of loop conditions to a more abstract grasp of the underlying algorithms. For these reasons, the aim of this work is to: i) identifying methodological tools to enhance a comprehensive understanding of the iteration constructs, ii) suggest strategies to teach iterations. We interviewed 20 experienced upper secondary teachers of introductory programming in different kinds of schools. The interviews were mainly aimed at ascertaining teachers’ beliefs about major sources of issues for basic programming concepts and their approach to the teaching and learning of iteration constructs. Once teachers’ perception of students’ difficulties have been identified, we have submitted, to a sample of 164 students, a survey which included both questions on their subjective perception of difficulty and simple tasks probing their understanding of iteration. Data collected from teachers and students confirm that iteration is a central programming concept and indicate that the treatment of conditions and nested constructs are major sources of students’ difficulties with iteration. The interviews allowed us to identify a list of problems that are typically presented by teachers to explain the iterations. Hence, a catalogue of significant program examples has been built to support students’ learning, tasks with characteristics different from those typically presented in class. Based on the outcome of previous steps, a survey to collect related information and good practices from a larger sample of teachers has been designed. Data collected have been analysed distinguishing an orientation towards more conceptual objectives, and one towards more practical objectives. Furthermore, regarding evaluation, a orientation focused on process-based assessment and another on product-based assessment. Finally, based on the outcome of previous students’ survey and drawing from the proposed examples catalogue, we have designed and submitted a new students’ survey, composed of a set of small tasks, or tasklets, to investigate in more depth on high-school students’ understanding of iteration in terms of code reading abilities. The chosen tasklets covered the different topics: technical program feature, correlation between tracing effort and abstraction, the role of flow-charts, students’ perception of self-confidence concerning high-level thinking skills