Beacons and Novice Programming Comprehension

Computer Science courses at tertiary level have one of the highest drop-out rates internationally. One of the main issues for this high attrition rate is often seen as CS1, the first Computer Science module usually encountered by students, which has a strong emphasis on computer programming. In order to aid students in the steep learning curve associated with programming, many different techniques have been utilised, to a varied degree of success. This paper aims to discover if particular lines of programming code exist that can help readers easily identify its functionality - referred to as a “beacon”. In a program containing a sort function, for example, advanced programmers might observe the swap code inside a loop and comprehend that it is a sorting algorithm, and therefore a beacon, without much further examination. This paper details the first phase of a study examining the presence of beacons in CS1 standard Java code using eye-tracking technology. In particular this paper will focus on the collection of data from non-novice programmers to determine whether or not beacons can be detected. Participants in this study were presented with basic Java programs and were asked to determine, from a list of possible options, what output was correct. Data was collected using an eye-tracking devices during a phase of experimentation and this data was subsequently analysed. From the analysis we were able to detect some beacons did exist in the code. In the future, some method of displaying these beacons could potentially be implemented as a form of intervention to aid students within the initial stages of learning a programming language

Beacons and Novice Programming Comprehension

Computer Science courses at tertiary level have one of the highest drop-out rates internationally. One of the main issues for this high attrition rate is often seen as CS1, the first Computer Science module usually encountered by students, which has a strong emphasis on computer programming. In order to aid students in the steep learning curve associated with programming, many different techniques have been utilised, to a varied degree of success. This paper aims to discover if particular lines of programming code exist that can help readers easily identify its functionality - referred to as a “beacon”. In a program containing a sort function, for example, advanced programmers might observe the swap code inside a loop and comprehend that it is a sorting algorithm, and therefore a beacon, without much further examination. This paper details the first phase of a study examining the presence of beacons in CS1 standard Java code using eye-tracking technology. In particular this paper will focus on the collection of data from non-novice programmers to determine whether or not beacons can be detected. Participants in this study were presented with basic Java programs and were asked to determine, from a list of possible options, what output was correct. Data was collected using an eye-tracking devices during a phase of experimentation and this data was subsequently analysed. From the analysis we were able to detect some beacons did exist in the code. In the future, some method of displaying these beacons could potentially be implemented as a form of intervention to aid students within the initial stages of learning a programming language

Early childhood preservice teachers' debugging block-based programs: An eye tracking study

Learning computational skills such as programming and debugging is very important for K-12 students given the increasing need of workforce proficient in computing technologies. Programming is an intricate cognitive task that entails iteratively creating and revising programs to create an artifact. Central to programming is debugging, which consists of systematically identifying and fixing program errors. Given its central role, debugging should be explicitly taught to early childhood preservice teachers so they can support their future students’ learning to program and debug errors. In this study, we propose using eye-tracking data and cued retrospective reporting to assess preservice teachers’ cognitive strategies while debugging. Several eye-tracking studies have investigated learners’ debugging strategies though the literature lacks studies (a) conducted with early childhood preservice teachers and (b) that focus on block-based programming languages, such as Scratch. The present study addresses this gap in the literature. This study used mixed methods to triangulate quantitative findings from eye movement analysis and qualitative findings about employed debugging strategies into the creation of descriptive themes. Results showed that participants developed strategies such as simultaneous review of output and code, use of beacons to narrow down the area to be debugged, and eye fixation on output to form hypotheses. But most often, debugging was not informed by a hypothesis, which led to trial and error. Study limitations and directions for future research are discussed.&nbsp

Multi-level code comprehension model for large scale software, A

1996 Fall.Includes bibliographical references (pages 142-147).For the past 20 years researchers have studied how programmers understand code they did not write. Most of this research has concentrated on small-scale code understanding. We consider it necessary to design studies that observe programmers working on large-scale code in production environments. We describe the design and implementation of such a study which included 11 maintenance engineers working on various maintenance tasks. The objective is to build a theory based on observations of programmers working on real tasks. Results show that programmers understand code at different levels of abstraction. Expertise in the application domain, amount of prior experience with the code, and task can determine the types of actions taken during maintenance, the level of abstraction at which the programmer works, and the information needed to complete a maintenance task. A better grasp of how programmers understand large scale code and what is most efficient and effective can lead to better tools, better maintenance guidelines, and documentation

Representational Learning Approach for Predicting Developer Expertise Using Eye Movements

The thesis analyzes an existing eye-tracking dataset collected while software developers were solving bug fixing tasks in an open-source system. The analysis is performed using a representational learning approach namely, Multi-layer Perceptron (MLP). The novel aspect of the analysis is the introduction of a new feature engineering method based on the eye-tracking data. This is then used to predict developer expertise on the data. The dataset used in this thesis is inherently more complex because it is collected in a very dynamic environment i.e., the Eclipse IDE using an eye-tracking plugin, iTrace. Previous work in this area only worked on short code snippets that do not represent how developers usually program in a realistic setting. A comparative analysis between representational learning and non-representational learning (Support Vector Machine, Naive Bayes, Decision Tree, and Random Forest) is also presented. The results are obtained from an extensive set of experiments (with an 80/20 training and testing split) which show that representational learning (MLP) works well on our dataset reporting an average higher accuracy of 30% more for all tasks. Furthermore, a state-of-the-art method for feature engineering is proposed to extract features from the eye-tracking data. The average accuracy on all the tasks is 93.4% with a recall of 78.8% and an F1 score of 81.6%. We discuss the implications of these results on the future of automated prediction of developer expertise. Adviser: Bonita Shari