Automatic Inference of Programming Performance and Experience from Typing Patterns

Studies on retention and success in introductory programming course have suggested that previous programming experience contributes to students' course outcomes. If such background information could be automatically distilled from students' working process, additional guidance and support mechanisms could be provided even to those, who do not wish to disclose such information. In this study, we explore methods for automatically distinguishing novice programmers from more experienced programmers using fine-grained source code snapshot data. We approach the issue by partially replicating a previous study that used students' keystroke latencies as a proxy to introductory programming course outcomes, and follow this by an exploration of machine learning methods to separate those students with little to no previous programming experience from those with more experience. Our results confirm that students' keystroke latencies can be used as a metric for measuring course outcomes. At the same time, our results show that students programming experience can be identified to some extent from keystroke latency data, which means that such data has potential as a source of information for customizing the students' learning experience.Peer reviewe

An Investigation into the Imposed Cognitive Load of Static & Dynamic Type Systems on Programmers

Static and dynamic type systems have long been a point of contention in the programming language wars. Yet, for many years, arguments on either side were drawn from personal experience and not empirical evidence. A challenge for researchers is that the usability of language constructs is difficult to quantify, especially since usability can be interpreted in many ways. By one definition, language usability can be measured in terms of the level of cognitive load imposed on a developer. This can be done through questionnaires, but ultimately user responses are subject to bias. In recent years, eye-tracking has been shown to be an effective means of measuring cognitive load via direct physiological measures. Towards the goal of measuring type system usability, we present a user study in which participants completed programming tasks in Java and Groovy. This thesis explored the use of the Index of Cognitive Activity (ICA) as a cognitive load measurement tool and considered novices and experts separately in the analysis. We found ICA to be an ineffective means of measuring type system usability and we cannot say conclusively whether it can be generally applied to programming tasks. Despite this, our results contradict previous studies as we found type system did not affect success rate, task completion time, or perceived task difficulty

Dynamic clamp with StdpC software

Dynamic clamp is a powerful method that allows the introduction of artificial electrical components into target cells to simulate ionic conductances and synaptic inputs. This method is based on a fast cycle of measuring the membrane potential of a cell, calculating the current of a desired simulated component using an appropriate model and injecting this current into the cell. Here we present a dynamic clamp protocol using free, fully integrated, open-source software (StdpC, for spike timing-dependent plasticity clamp). Use of this protocol does not require specialist hardware, costly commercial software, experience in real-time operating systems or a strong programming background. The software enables the configuration and operation of a wide range of complex and fully automated dynamic clamp experiments through an intuitive and powerful interface with a minimal initial lead time of a few hours. After initial configuration, experimental results can be generated within minutes of establishing cell recording

The effect of teaching method or student characteristics on student achievement or attitude in a BASIC computer programming undergraduate course in agricultural mechanization at Iowa State University

During two successive semesters, 103 college students in a BASIC programming course were enrolled in classes which were randomly assigned as experimental or control groups. The control group received instruction by the traditional lecture method, while the experimental group received instruction by computer-assisted instructional materials. All students were given a pretest and posttest measuring knowledge of BASIC computer programming and computer use, a pretest and posttest measuring attitude toward computers and computer use, and a survey to obtain demographic data on the students. From an analysis of covariance, the following conclusions were drawn: (1) There was no significant difference in attitude or knowledge scores when the students were grouped by teaching method; (2) There was a significant difference in posttest knowledge scores when the students were grouped by pretest attitude score, semesters of vocational agriculture, subject in which the students made their highest and lowest grade in high school, average secondary and post-secondary math grade, student classification, student major, video game experience, occupational plans, or the person that most influenced the student to take the course; (3) There was a significant difference in posttest attitude scores when the students were grouped by typing ability, or computer experience

Seeking Flow from Fine-Grained Log Data

Flow is the experience of deep absorption in a demanding, intrinsically-motivating task conducted with skill. We consider how to measure behavioural correlates of flow from fine-grained process data extracted from programming environments. Specifically, we propose measuring affective factors related to flow non-intrusively based on log data. Presently, such affective factors are typically measured intrusively (by self-report), which naturally will break the flow. We evaluate our approach in a pilot study, where we use log data and survey data collected from an introductory programming course. The log data is fine-grained, containing timestamped actions at the keystroke level from the process of solving programming assignments, while the survey data has been collected at the end of every completed assignment. The survey data in the pilot study comprises of Likert-like items measuring perceived educational value, perceived difficulty, and students' self-reported focus when solving the assignments. We study raw and derived log data metrics, by looking for relationships between the metrics and the survey data. We discuss the results of the pilot study and provide suggestions for future work related to non-intrusive measures of programmer affect.Peer reviewe

Opportunities Unshackled: The Effect of Educational & Vocational Programming on Prison Life

This study examines the impact of educational and work opportunities on incarcerated individuals’ prison experience. Using data from the John Howard Association’s (JHA) ‘Measuring the Quality of Prison Life Survey’ (MQPL) distributed to all Illinois Department of Corrections (IDOC) facilities, we examine prisoner perceptions of well-being, security, professionalism, and harmony to understand the quality of life in prison better. When comparing groups, we found a statistically, significant positive relationship between participation in educational and vocational programming and higher reported scores for all domains. Throughout this experience, we deepened our understanding of Illinois prison culture and informed JHA\u27s prison advocacy work

Assessment of work performed by paired students: a pedagogical experience

“Teamwork” is one of the abilities most valued by employers. In [16] we describe the process of adapting to the ECTS methodologies (for ongoing assessment), a course in computer programming for students in a technical degree (Marine Engineering, UPM) not specifically dedicated to computing. As a further step in this process we have emphasized cooperative learning. For this, the students were paired and the work of each pair was evaluated via surprise tests taken and graded jointly, and constituting a substantial part of the final grade. Here we document this experience, discussing methodological aspects, describing indicators for measuring the impact of these methodologies on the educational experience, and reporting on the students’ opinion of it

LabVIEW application with embedded Lua scripting for a laser based measuring machine

Published ArticleThis paper presents the work on the development of software for an industrial laser based measuring machine. The goal being not only for a working application, but also to optimise the development process and ease future maintenance of the software. LabVIEW with its graphical method of programming allows engineers to easily create large software applications to control industrial processes and machines. This software if not properly designed can lead to stability and maintenance problems. The experience gained from developing, maintaining and improving a LabVIEW application for a laser measuring machine, results in the integration of the Lua scripting language into LabVIEW. It is shown how the embedded Lua allows the LabVIEW software application for the machine to be structured for simpler development and maintenance

Towards a Theory of Software Development Expertise

Software development includes diverse tasks such as implementing new features, analyzing requirements, and fixing bugs. Being an expert in those tasks requires a certain set of skills, knowledge, and experience. Several studies investigated individual aspects of software development expertise, but what is missing is a comprehensive theory. We present a first conceptual theory of software development expertise that is grounded in data from a mixed-methods survey with 335 software developers and in literature on expertise and expert performance. Our theory currently focuses on programming, but already provides valuable insights for researchers, developers, and employers. The theory describes important properties of software development expertise and which factors foster or hinder its formation, including how developers' performance may decline over time. Moreover, our quantitative results show that developers' expertise self-assessments are context-dependent and that experience is not necessarily related to expertise.Comment: 14 pages, 5 figures, 26th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2018), ACM, 201

Dynamical transitions in the evolution of learning algorithms by selection

We study the evolution of artificial learning systems by means of selection. Genetic programming is used to generate a sequence of populations of algorithms which can be used by neural networks for supervised learning of a rule that generates examples. In opposition to concentrating on final results, which would be the natural aim while designing good learning algorithms, we study the evolution process and pay particular attention to the temporal order of appearance of functional structures responsible for the improvements in the learning process, as measured by the generalization capabilities of the resulting algorithms. The effect of such appearances can be described as dynamical phase transitions. The concepts of phenotypic and genotypic entropies, which serve to describe the distribution of fitness in the population and the distribution of symbols respectively, are used to monitor the dynamics. In different runs the phase transitions might be present or not, with the system finding out good solutions, or staying in poor regions of algorithm space. Whenever phase transitions occur, the sequence of appearances are the same. We identify combinations of variables and operators which are useful in measuring experience or performance in rule extraction and can thus implement useful annealing of the learning schedule.Comment: 11 pages, 11 figures, 2 table