Syntax error based quantification of the learning progress of the novice programmer

© 2018 Association for Computing Machinery. Recent data-driven research has produced metrics for quantifying a novice programmer’s error profile, such as Jadud’s error quotient. However, these metrics tend to be context dependent and contain free parameters. This paper reviews the caveats of such metrics and proposes a more general approach to developing a metric. The online implementation of the proposed metric is publicly available at http://online-analysis-demo.herokuapp.com/

An Exploration Of The Effects Of Enhanced Compiler Error Messages For Computer Programming Novices

Computer programming is an essential skill that all computing students must master and is increasingly important in many diverse disciplines. It is also difficult to learn. One of the many challenges novice programmers face from the start are notoriously cryptic compiler error messages. These report details on errors made by students and are essential as the primary source of information used to rectify those errors. However these difficult to understand messages are often a barrier to progress and a source of discouragement. A high number of student errors, and in particular a high frequency of repeated errors – when a student makes the same error consecutively – have been shown to be indicators of students who are struggling with learning to program. This instrumental case study research investigates the student experience with, and the effects of, software that has been specifically written to help students overcome their challenges with compiler error messages. This software provides help by enhancing error messages, presenting them in a straightforward, informative manner. Two cohorts of first year computing students at an Irish higher education institution participated over two academic years; a control group in 2014-15 that did not experience enhanced error messages, and an intervention group in 2013-14 that did. This thesis lays out a comprehensive view of the student experience starting with a quantitative analysis of the student errors themselves. It then views the students as groups, revealing interesting differences in error profiles. Following this, some individual student profiles and behaviours are investigated. Finally, the student experience is discovered through their own words and opinions by means of a survey that incorporated closed and open-ended questions. In addition to reductions in errors overall, errors per student, and the key metric of repeated error frequency, the intervention group is shown to behave more cohesively with fewer indications of struggling students. A positive learning experience using the software is reported by the students and the lecturer. These results are of interest to educators who have witnessed students struggle with learning to program, and who are looking to help remove the barrier presented by compiler error messages. This work is important for two reasons. First, the effects of error message enhancement have been debated in the literature – this work provides evidence that there can be positive effects. Second, these results should be generalisable at least in part, to other languages, students and institutions

Vary: An IDE for Designing Algorithms and Measuring Quality

Pseudocode is one of the recommended methods for teaching students to design algorithms. Having a tool that performs the automatic translation of an algorithm into pseudocode to a programming language would allow the student to understand the complete process of program development. In addition, the introduction of quality measurement of algorithms designed from the first steps of learning programming would enable the student to understand the importance of code quality for maintenance of software processes. This work describes Vary, an integrated development environment based on Eclipse for writing and running pseudocode algorithms. The environment automatically transforms abstract pseudocode into runnable C/C++ source code that can be later executed. Computer programming learners and even computational scientists can use Vary to write and run algorithms, while taking advantage of modern development environment features. Vary is provided with an additional extension to automatically carry out algorithm analysis with SonarQube

Psychophysiological measurements in programming task:guidelines for conducting EMG research

Abstract. Programming languages have been studied and developed throughout history of programming. There are lots of different programming languages that are being used in software development, but only core languages are taught in Universities. Programming languages usually have their own syntax, which may differ greatly from each other. Using different programming languages for same task may provoke different emotions in programmers, depending their knowledge on the language. Research on programming and programming languages have generally focused on technical and exterior aspects. More recently, there has been some research on the programmers and their emotions during the programming tasks. This master’s thesis focuses on latter and aims to provide new information of programmers experienced emotions during the programming tasks by using EMG-recordings. This master thesis’ main study focus is in psychophysiology, which combines psychology to physiological research, by finding correlation between physiological activity and emotional phenomenon. This study assessed university students experienced emotions when conducting programming tasks with C and Python programming languages. EMG measurement device was used on the test participants to record signal data from facial based muscles for smiling and frowning activity, which are linked to positive and negative emotions. This study’s results showed small differences with emotional experiences during the programming tasks, but the overall results were not statistically significant. Therefore, more research on this topic is needed for more consistent results. Additionally, this research has provided guidelines on how EMG studies are conducted on laboratory setting and suggestions for future studies

Software Verification and Graph Similarity for Automated Evaluation of Students' Assignments

In this paper we promote introducing software verification and control flow graph similarity measurement in automated evaluation of students' programs. We present a new grading framework that merges results obtained by combination of these two approaches with results obtained by automated testing, leading to improved quality and precision of automated grading. These two approaches are also useful in providing a comprehensible feedback that can help students to improve the quality of their programs We also present our corresponding tools that are publicly available and open source. The tools are based on LLVM low-level intermediate code representation, so they could be applied to a number of programming languages. Experimental evaluation of the proposed grading framework is performed on a corpus of university students' programs written in programming language C. Results of the experiments show that automatically generated grades are highly correlated with manually determined grades suggesting that the presented tools can find real-world applications in studying and grading

A model for systematically investigating relationships between variables that affect the performance of novice programmers

This research was motivated by an interest in novices learning to program and a desire to understand the factors that affect their learning. The traditional approach to performing such an investigation has been to select factors which may be important and then perform statistical tests on a few potential relationships. A new research model is proposed and tested to ensure that a thorough and systematic investigation of the data is performed. This thesis describes the data, defines the model and explains the application and validation of the model. The research process is managed by a control algorithm that is the heart of the model. This algorithm is seeded by a hypothesis that connects two variables of interest and dictates the testing of a series of hypotheses; as it does this, it also delves deeper into the data to identify additional relationships. In this research the model was applied to investigate the relationships between: learning style and achievement; programming behaviour and achievement; and learning style and programming behaviour. Learning style was assessed using Kolb’s Learning Style Inventory, achievement was based on exam score and programming behaviour was extracted from a log of student activities using a programming tool. The largest number of significant relationships was found between aspects of behaviour and achievement. The model was validated by classifying the significant hypotheses based on the research model’s tree structure, the section of the programming tool in use and the literature. These three classification schemes provided a structure to explore their similarities and differences. The model was thus demonstrated to be robust and repeatable by comparing the results with those from both using a programming tool, and expert opinion. This research has revealed several attributes of the learning behaviour that affected the students’ results within this group, including aspects of timeliness and overall volume of activity. These are suitable targets for future investigations. The research model could be applied to other data sets where an in-depth investigation into pairwise data is required.

Exploring student perceptions about the use of visual programming environments, their relation to student learning styles and their impact on student motivation in undergraduate introductory programming modules

My research aims to explore how students perceive the usability and enjoyment of visual/block-based programming environments (VPEs), to what extent their learning styles relate to these perceptions and finally to what extent these tools facilitate student understanding of basic programming constructs and impact their motivation to learn programming

Self-efficacy and engagement as predictors of student programming performance: An international perspective

High attrition and failure rates are a common phenomenon in introductory programming courses and are a major concern since course instructors are not able to successfully teach novice programmers the fundamental concepts of computer programming and equip them with skills to code solutions to programming problems. Existing solutions that attempt to minimise the high failure and attrition rates have had little impact on improving the performance of the novice programmers. However, the behaviour of the novice programmer has received little attention from introductory programming course instructors although the literature on learning theory suggests that self-efficacy and engagement are two behavioural factors that affect a student’s performance. This study fills the gap in existing research by examining the effect of programming self-efficacy on the engagement of novice programmers, and the effect of their engagement on their programming performance. A research model that proposes a link between programming self-efficacy and the indicators of engagement that are specific to the context of introductory programming courses, and a link between the indicators of engagement to the programming performance of the novice programmer was developed. A three-phased mixed methods approach which consists of two survey questionnaires and focus groups was used to validate the research model. Data was collected in New Zealand and in Malaysia with 433 novice programmers participating in the survey questionnaires while 4 focus groups were held to refine and validate the indicators of engagement in introductory programming courses. The findings of the focus groups confirmed that participation, help-seeking, persistence, effort, deep learning, surface learning, trial and error, interest, and enjoyment were indicators of engagement while gratification emerged as a new indicator of engagement in introductory programming courses. The data from the survey questionnaires were analysed using Partial Least Squares Structural Equation Modeling (PLS-SEM). This study found that the programming self-efficacy beliefs of novice programmers had a strong influence on their engagement behaviour with the exception of help-seeking, while effort, enjoyment, deep learning, and surface learning were predictors of programming performance. These findings have implications for introductory programming course instructors and the recommendations emerging from this study include making clear behavioural expectations, designing courses which stimulate and support effective behaviour, and making novice programmers aware of the engagement behaviour that does not lead to better programming performance. This study contributes to the theory of teaching computer programming, and to the practice of designing and delivering introductory programming courses

The evaluation of a pedagogical-program development environment for Novice programmers : a comparative study

It is an acknowledged fact that many novice programmers experience difficulty in the process of learning to program. One of the contributing factors to this difficulty is the Program Development Environment (PDE). Professional-PDEs are those developed specifically for professional programmers, but are often used by educational institutions in the instruction of programming. It has long been accepted that such environments are inappropriate in the instruction of programming due to unnecessary complexity and lack of support for novice programmers in the learning process. Numerous pedagogical-PDEs supporting the mechanics of programming have been developed in response to this. A review of literature, however, indicates that very limited empirical studies comparing pedagogical-PDEs and professional-PDEs have been conducted. The current study investigates whether there are measurable benefits to using a pedagogical-PDE supporting the mechanics of programming in the instruction of programming instead of a professional-PDE. A comparative study of this nature requires a representative pedagogical-PDE and representative professional-PDE be compared with one another. The first part of the current study determines a set of requirements that a pedagogical- PDE should adhere to based on literature. A set of representative features for a pedagogical-PDE is derived by examining the features of existing PDEs in conjunction with the set of requirements. Based on these features, a pedagogical-PDE, known as SimplifIDE, is developed that implements the representative set of features and that meets are the requirements for a pedagogical-PDE. The second part of the current study is the specification and administration of an empirical experiment in which SimplifIDE and Borland© DelphiTM are compared with one another. A holistic approach in determining the differences between the PDEs is taken and three main areas are examined, namely academic performance, perceptions and programming behavior