Early childhood preservice teachers' debugging block-based programs: An eye tracking study

Learning computational skills such as programming and debugging is very important for K-12 students given the increasing need of workforce proficient in computing technologies. Programming is an intricate cognitive task that entails iteratively creating and revising programs to create an artifact. Central to programming is debugging, which consists of systematically identifying and fixing program errors. Given its central role, debugging should be explicitly taught to early childhood preservice teachers so they can support their future students’ learning to program and debug errors. In this study, we propose using eye-tracking data and cued retrospective reporting to assess preservice teachers’ cognitive strategies while debugging. Several eye-tracking studies have investigated learners’ debugging strategies though the literature lacks studies (a) conducted with early childhood preservice teachers and (b) that focus on block-based programming languages, such as Scratch. The present study addresses this gap in the literature. This study used mixed methods to triangulate quantitative findings from eye movement analysis and qualitative findings about employed debugging strategies into the creation of descriptive themes. Results showed that participants developed strategies such as simultaneous review of output and code, use of beacons to narrow down the area to be debugged, and eye fixation on output to form hypotheses. But most often, debugging was not informed by a hypothesis, which led to trial and error. Study limitations and directions for future research are discussed.&nbsp

Fifty years of the Psychology of Programming

This paper reflects on the evolution (past, present and future) of the ‘psychology of programming' over the 50 year period of this anniversary issue. The International Journal of Human-Computer Studies (IJHCS) has been a key venue for much seminal work in this field, including its first foundations, and we review the changing research concerns seen in publications over these five decades. We relate this thematic evolution to research taking place over the same period within more specialist communities, especially the Psychology of Programming Interest Group (PPIG), the Empirical Studies of Programming series (ESP), and the ongoing community in Visual Languages and Human-Centric Computing (VL/HCC). Many other communities have interacted with psychology of programming, both influenced by research published within the specialist groups, and in turn influencing research priorities. We end with an overview of the core theories that have been developed over this period, as an introductory resource for new researchers, and also with the authors’ own analysis of key priorities for future research

On the Influence of Representation Type and Gender on Recognition Tasks of Program Comprehension

RÉSUMÉ L’objectif de la maintenance logicielle est d’améliorer les logiciels existants en préservant leur intégrité. La maintenance peut représenter jusqu’à 60% du budget d’un logiciel. Ainsi, améliorer la maintenabilité des logiciels est bénéfique aussi bien pour les fournisseurs que les utilisateurs de logiciels. Les développeurs de logiciels consacrent un effort considérable à la compréhension des programmes, qui est une étape primordiale à la maintenance logicielle. Nous faisons l’hypothèque que le genre des développeurs de logiciels et le type de représentation peut affecter leur effort et leur efficacité. Ces facteurs doivent être considérés et minutieusement analysés dans la mesure où ils peuvent cacher certains effets significatifs pouvant être identifiés en analysant le processus de compréhension. Dans cette thèse, nous nous inspirons de l’utilisation de l’occulomètre pour l’étude du processus cognitif lors de la résolution des problèmes. Nous avons effectué une étude fonctionnelle pour évaluer tous les travaux de recherche faisant usage de l’occulomètre en génie logiciel. Les résultats obtenus nous ont motivé à utiliser l’occulomètre pour effectuer un ensemble d’études afin analyser l’effet de deux facteurs importants sur la compréhension des programmes : le type de représentation (textuelle ou graphique) et le genre du développeur. Afin de comprendre comment les différents types de représentations et le genre influencent les stratégies de visualisation, nous avons étudié la différence de stratégie entre développeurs. Les résultats obtenus montrent que, comparé à une représentation graphique, la représentation sous forme de texte structuré aide mieux le développeur dans son processus cognitif lors de la compréhension des programmes de petite taille. Ainsi, la représentation textuelle requiert moins de temps et d’effort aux participants. Par contre, la représentation graphique est celle préférée par les développeurs. Nos résultats montrent que la structure topologique de la représentation graphique aide les développeurs à mémoriser l’emplacement des éléments et à retrouver plus rapidement les éléments pertinents comparé à la représentation textuelle. En plus, la structure hiérarchique de la représentation graphique guide les développeurs à suivre une stratégie de visualisation spécifique. Nous avons observé que les femmes et les hommes ont des stratégies de visualisation différentes lors de la lecture du code ou de la mémorisation des noms des identificateurs. Les femmes ont tendance à inspecter minutieusement toutes les options afin de procéder à l’élimination de la mauvaise réponse. Au contraire, les hommes ont tendance à inspecter brièvement certaines réponses. Pendant que les femmes consacrent plus de temps à analyser chaque type d’entité l’un après l’autre, les hommes alternent leur attention entre différents type d’entité.----------ABSTRACT The purpose of software maintenance is to correct and enhance an existing software system while preserving its integrity. Software maintenance can cost more than 60% of the budget of a software system, thus improving the maintainability of software is important for both the software industry and its customers. Program comprehension is the initial step of software maintenance that requires the major amount of maintenance’s time and effort. We conjuncture that developers’ gender and the type of representations that developers utilize to perform program comprehension impact their efficiency and effectiveness. These factors must be considered and carefully studied, because they may hide some significant effects to be found by analyzing the comprehension process. In this dissertation, inspired by the literature on the usefulness of eye-trackers to study the cognitive process involved in problem solving activities, we perform a mapping study and evaluate all research relevant to the use of eye-tracking technique in software engineering. The results motivate us to perform a set of eye-tracking studies to analyze the impact of two important factors on program comprehension: representation type (textual vs. graphical) and developers’ gender. Moreover, we investigate and compare viewing strategies variability amongst developers to understand how the representation type and gender differences influence viewing strategies. Overall, our results indicate that structured text provides more cognitive support for developers while performing program comprehension with small systems compared to a graphical representation. Developers spend less time and effort working with textual representations. However, developers mostly preferred to use graphical representations and our results confirm that the topological structure of graphical representations helps developers to memorize the location of the elements and to find the relevant ones faster in comparison with textual representation. Moreover, the hierarchical structure of the representation guides developers to follow specific viewing strategies while working with representations. Regarding the impact of gender, our results emphasize that male and female developers exploit different viewing strategies while reading source code or recalling the names of identifiers. Female developers seem to carefully weigh all options and rule out wrong answers, while male developers seem to quickly set their minds on some answers and move forward. Moreover, female developers spend more time on each source code entity and analyze it before going to the next one. In contrast, male developers utilize several attention switching strategies between different source code entities

Eye movements in code reading:relaxing the linear order

Abstract—Code reading is an important skill in programming. Inspired by the linearity that people exhibit while natural lan-guage text reading, we designed local and global gaze-based mea-sures to characterize linearity (left-to-right and top-to-bottom) in reading source code. Unlike natural language text, source code is executable and requires a specific reading approach. To validate these measures, we compared the eye movements of novice and expert programmers who were asked to read and comprehend short snippets of natural language text and Java programs. Our results show that novices read source code less linearly than natural language text. Moreover, experts read code less linearly than novices. These findings indicate that there are specific differences between reading natural language and source code, and suggest that non-linear reading skills increase with expertise. We discuss the implications for practitioners and educators. I

Overcoming Language Dichotomies: Toward Effective Program Comprehension for Mobile App Development

Mobile devices and platforms have become an established target for modern software developers due to performant hardware and a large and growing user base numbering in the billions. Despite their popularity, the software development process for mobile apps comes with a set of unique, domain-specific challenges rooted in program comprehension. Many of these challenges stem from developer difficulties in reasoning about different representations of a program, a phenomenon we define as a "language dichotomy". In this paper, we reflect upon the various language dichotomies that contribute to open problems in program comprehension and development for mobile apps. Furthermore, to help guide the research community towards effective solutions for these problems, we provide a roadmap of directions for future work.Comment: Invited Keynote Paper for the 26th IEEE/ACM International Conference on Program Comprehension (ICPC'18

Knowledge restructing and the development of expertise in computer programming

This thesis reports a number of empirical studies exploring the development of expertise in computer programming. Experiments 1 and 2 are concerned with the way in which the possession of design experience can influence the perception and use of cues to various program structures. Experiment 3 examines how violations to standard conventions for constructing programs can affect the comprehension of expert, intermediate and novice subjects. Experiment 4 looks at the differences in strategy that are exhibited by subjects of varying skill level when constructing programs in different languages. Experiment 5 takes these ideas further to examine the temporal distribution of different forms of strategy during a program generation task. Experiment 6 provides evidence for salient cognitive structures derived from reaction time and error data in the context of a recognition task. Experiments 7 and 8 are concerned with the role of working memory in program generation and suggest that one aspect of expertise in the programming domain involves the acquisition of strategies for utilising display-based information. The final chapter attempts to bring these experimental findings together in terms of a model of knowledge organisation that stresses the importance of knowledge restructuring processes in the development of expertise. This is contrasted with existing models which have tended to place emphasis upon schemata acquisition and generalisation as the fundamental modes of learning associated with skill development. The work reported here suggests that a fine-grained restructuring of individual schemata takes places during the later stages of skill development. It is argued that those mechanisms currently thought to be associated with the development of expertise may not fully account for the strategic changes and the types of error typically found in the transition between novice, intermediate and expert problem solvers. This work has a number of implications for existing theories of skill acquisition. In particular, it questions the ability of such theories to account for subtle changes in the various manifestations of skilled performance that are associated with increasing expertise. Secondly, the work reported in this thesis attempts to show how specific forms of training might give rise to the knowledge restructuring process that is proposed. Finally, the thesis stresses the important role of display-based problem solving in complex tasks such as programming and highlights the role of programming language notation as a mediating factor in the development and acquisition of problem solving strategies

Expectation incongruence in music and code reading: The eye-tracking approach

Humans create and use different kinds of languages in order to store, view, and convey various types of information. Natural languages, such as English, allow people to communicate with each other in everyday and professional contexts. In contrast, symbolic languages, such as Western music notation or programming languages, enable people to make use of technical devices like musical instruments or computers. Research on the eye movements of expert musicians and programmers has revealed certain similarities in how these symbolic languages are read: unlike text reading, experts read music and code with more regressive eye movements. The current dissertation is the first project that explores music and code reading together. It focuses on one of the aspects of music and code reading that is equally important for both symbolic languages—the skill of working with unexpected information from a notation. In music and programming, this skill is especially required in tasks such as handling surprising melodic patterns and debugging, respectively. This dissertation had three main aims: (1) theoretical exploration of similarities and differences in creating expectations that help with pattern recognition in music and programming; (2) development (in music reading) and creation (in code reading) of research methodologies that can be applied in research on incongruent patterns; and (3) exploration of the cognitive processing of incongruent notation in experienced music readers and one experienced code reader. Surprising elements in familiar patterns hamper their recognition. Article I presents a theoretical exploration of the similarities and differences in building expectations that allow pattern recognition in music and programming. The proposed prediction model, which serves as the solution to Aim 1, includes three components that are common for both music and programming: (1) knowledge of language systems, (2) knowledge of meaning, and (3) knowledge of context. In addition, it also contains two components that differ for music and programming: (4) translation of information and (5) temporal and motor requirements. Experiments presented in this dissertation can be considered to be the first steps toward looking at certain components of the proposed prediction model in detail when prediction works normally (congruent notation) and when prediction is violated (incongruent notation). In order to study the reading of surprising incongruent patterns in music and code, special experimental settings, which provide the solution to Aim 2, were developed for music reading and created for code reading. Hence, the selected setup for the music reading study was based on a prior study (Penttinen et al., 2015), where incongruences were introduced in the “Mary Had a Little Lamb” melody and all music performances were temporarily controlled using a metronome. Experiment 1 developed this set-up by inserting incongruent notes into two different tonalities and by asking participants to play on a piano or sing from the notation. Thus, the music reading experiment focused on the first, second, fourth, and fifth components of the proposed model in music reading. It explored how the meaning of congruent and incongruent musical symbols is processed by experienced music readers. In addition, it also explored the translation of music information into two different performance ways (singing and playing piano) that have different motor requirements. Combining three different eye movement parameters allowed the researcher to describe different aspects of the cognitive processing of incongruent music reading—the temporal aspect, with the help of the eye-time span parameter (ETS), the cognitive effort aspect with the help of the mean pupil size parameter measured only in first-pass fixations, and the attention aspect with the help of the first-pass fixation duration. Experiment 2 on code reading was carefully designed on the basis of the music reading study. Consequently, incongruences were introduced in different parts of the familiar notation. The Bubble sort algorithm—a well-known sorting algorithm—was chosen as an analogue of the “Mary Had a Little Lamb” melody in programming. As in the music reading study, all code reading performances were temporarily controlled. The case code study provided some insights into the first and second components of the proposed model in programming by investigating how an experienced programmer reads sorting algorithms with and without surprising patterns. It particularly focuses on the phenomenon of an experienced reader overlooking the surprising pattern, which is considered to be the original one instead so-called proof-readers’ error. In addition, this study explored the issue of the unit of code reading analysis by comparing two different options: lines and elements. The study introduced saccade velocity as a parameter of cognitive effort for the incongruent code reading analysis. Research findings from these experimental studies provided the solution to Aim 3 and revealed that—in both music and code reading—incongruent patterns in the notation led to changes in fixation and cognitive effort parameters (pupil size and saccadic velocity). In contrast to code reading, strict temporal requirements for the processing of incongruence exist in music reading. The application of the eye-time span (ETS) parameter that describes the distance between the performer’s gaze and musical time, allowed the researcher to investigate the temporal aspect of incongruence processing in the music reading experiment. Hence, experienced readers had longer ETS when they approached the incongruent part of the notation and shorter ETS when they were in the process of struggling with the incongruent part. In addition to incongruent reading, the difference in the performance mode of the same music task associated with the translation of information and motor requirements was studied in the music reading experiment by comparing singing and playing from music scores. Despite the fact that the participants played incongruent melodies better than they sang them, the analysis of eye movement parameters allowed the researcher to discover that singing might be less cognitively demanding than playing. These findings are discussed within the proposed theoretical model of prediction and associated expertise theories.Odotusten vastaisten symbolien lukeminen: katseenseurantatutkimus nuotin- ja koodinluvusta Ihmiset luovat ja käyttävät erilaisia kieliä tallentaakseen, tarkastellakseen ja välittääkseen informaatiota. Luonnolliset kielet, kuten englanti, mahdollistavat ihmisten välisen kommunikaation arkisissa ja ammatillisissa tilanteissa. Sen sijaan symboliset kielet, kuten länsimainen nuottikirjoitus tai ohjelmointikielet, mahdollistavat erilaisten laitteiden, kuten soittimien tai tietokoneiden, operoinnin. Taitavien muusikkojen ja koodinlukijoiden silmänliikkeiden tutkimus on paljastanut joitakin samankaltaisuuksia siitä, miten näitä kahta symbolikieltä luetaan: toisin kuin tekstin lukemisessa, taitavat nuotin- ja koodinlukijat tekevät enemmän regressiivisiä, eli taaksepäin suuntautuvia silmänliikkeitä. Tämä väitöstutkimus on ensimmäinen tutkimushanke, jossa tarkastellaan nuotin- ja koodinlukua rinnakkain. Tutkimus keskittyy tiettyyn, molemmissa symbolikielissä tärkeään piirteeseen, eli taitoon selvitä lukemisen aikana notaatiossa havaittuun yllättävään informaatioon. Sekä musiikin että ohjelmoinnin aloilla tätä taitoa tarvitaan silloin, kun lukijan täytyy käsitellä yllättäviä melodisia kuvioita musiikkikappaletta lukiessaan tai etsiä virheitä koodista. Tällä väitöstutkimuksella oli kolme päätavoitetta: (1) teoreettinen pohdinta nuotin- ja koodinluvun yhtäläisyyksistä ja eroavaisuuksista ja erityisesti siitä, miten taitavat lukijat muodostavat ennakko-oletuksia lukemastaan symbolien tunnistamisen helpottamiseksi; (2) menetelmien kehittäminen (nuotinluvussa) ja luominen (koodinluvussa) inkongruenttien, eli epäyhdenmukaisten, kuvioiden lukemisen tutkimukseen; ja (3) taitavien nuotinlukijoiden ja yhden taitavan koodinlukijan kognitiivisen prosessoinnin tutkiminen silloin, kun lukijat käsittelevät inkongruenttia informaatiota. Yllättävät elementit tutussa visuaalisessa materiaalissa vaikeuttavat kyseessä olevan materiaalin prosessointia. Artikkelissa I pohditaan nuotin- ja koodinluvun teoreettisia eroja ja eroavaisuuksista ja sitä, miten taitavat lukijat muodostavat ennakko-oletuksia lukiessaan. Ehdotettu ennustusmalli, jonka avulla vastataan päätavoitteeseen 1, sisältää kolme molemmille symbolikielille yhteistä komponenttia: (1) tiedon kielijärjestelmästä, (2) tiedon merkityksestä ja (3) tiedon kontekstista. Tämän lisäksi malli sisältää kaksi komponenttia, joissa nuotin- ja koodinluku eroavat toisistaan: (4) informaation kääntäminen laitteelle ja (5) temporaaliset ja motoriset vaatimukset. Tässä väitöstutkimuksessa esiteltävät empiiriset osahankkeet olivat ensiaskelia ennustusmallin komponenttien tutkimuksessa. Kahdessa osahankkeessa tarkasteltiin yksityiskohtaisesti tilanteita, joissa ennakko-oletuksia voi hyödyntää tavalliseen tapaan (kongruentti notaatio) ja kun odotukset eivät toteudu (inkongruentti notaatio). Tässä väitöstutkimuksessa kehitettiin koeasetelmia nuotin- ja koodinluvun aikaisten, yllättävien ja inkongruenttien kuvioiden lukemisen tutkimusta varten (päätavoite 2). Nuotinlukuaiheinen koeasetelma pohjautui aikaisempaan tutkimukseen (Penttinen et al., 2015), jossa epäyhdenmukaisuuksia sijoitettiin tuttuun ”Maijall’ oli karitsa” –melodiaan ja soittosuoritusten ajoitusta kontrolloitiin metronomin avulla. Osatutkimus 1 kehitti tätä asetelmaa edelleen esittämällä tutun kappaleen osallistujille kahdessa eri sävellajissa ja pyytämällä osallistujia toteuttamaan melodia kahdella eri tavalla, soittaen ja laulaen. Osatutkimus 1 keskittyi siis ennustusmallin ensimmäiseen, toiseen, neljänteen ja viidenteen komponenttiin nuotinlukemisen näkökulmasta. Osatutkimuksessa 1 tutkittiin kuinka taitavat nuotinlukijat prosessoivat kongruenttien ja inkongruenttien nuottisymbolien merkityksiä. Tämän lisäksi osahanke selvitti sitä, miten nuottiinformaatio käännettiin kahdelle motorisilta vaatimuksiltaan erilaiselle ”soittimelle” (pianonsoitto ja laulaminen). Kognitiivisia prosesseja inkongruentin materiaalin lukemisen aikana kuvailtiin kolmen eri silmänliikemuuttujan turvin: lukuprosessin ajallista etenemistä selvitettiin eye-time span –mittarin (ETS) avulla, kognitiivista työmäärää mittaamalla pupillin koon vaihtelua, ja fiksaatioiden kestot kertoivat huomion kohdistumisesta notaation eri osiin ensilukemisen aikana. Koodinlukuun keskittyvä osatutkimus 2 suunniteltiin osatutkimuksen 1 koeasetelman pohjalta ja epäyhdenmukaisuudet sijoitettiin jälleen tuttuun notaatioon. Koodinlukukokeessa nuotinlukukokeen ”Maijall’ oli karitsa” –melodian tilalle valittiin hyvin tunnettu kuplalajittelualgoritmi, ja myös koodinlukutehtävässä kontrolloitiin ajankäyttöä. Tapaustutkimuksessa selvitettiin, miten kokenut ohjelmoija luki lajittelualgoritmia silloin kun siinä joko oli tai ei ollut inkongruentteja kohtia. Näin voitiin tarkastella ennustusmallin ensimmäistä ja toista komponenttia koodinlukemisen näkökulmasta. Tapaustutkimus keskittyi erityisesti tilanteeseen, jossa kokenut lukija ohitti yllättävät elementit notaatiossa ja tulkitsi inkongruentin algoritmin oikeaksi ja alkuperäiseksi (ns. proof-readers’ error). Tämän lisäksi osatutkimuksessa 2 testattiin koodinlukututkimuksiin sopivia analyysiyksiköitä vertaamalla kahta vaihtoehtoa, rivejä ja elementtejä, ja esiteltiin sakkadien nopeus kognitiivisen työmäärän tarkasteluun sopivana, koodinlukututkimuksille uutena mittarina. Osatutkimusten 1 ja 2 perusteella vastattiin päätavoitteeseen 3. Osatutkimuksissa selvisi, että sekä nuotin- että koodinlukutilanteissa inkongruentit kohdat notaatiossa johtivat muutoksiin fiksaatio- ja kognitiivisen työmäärän mittareissa (pupillin koko ja sakkadin nopeus). Toisin kuin koodinluvussa, tiukat temporaaliset rajoitteet säätelevät inkongruenssin prosessointia nuotinluvun aikana. Tästä syystä nuotinlukukokeessa hyödynnettiin ETS-mittaria, joka kertoo katseen kohdan ja musiikillisen ajan välisestä etäisyydestä. Taitavien nuotinlukijoiden ETS oli pidempi, kun he lukiessaan lähestyivät inkongruenttia kohtaa, ja lyhyempi, kun he soittivat tätä samaista kohtaa. Inkongruentin kohdan lukemisen lisäksi tutkittiin kahta erilaista esitystapaa (laulaminen ja soittaminen), sillä esitystapa liittyy nuotti-informaation kääntämiseen oikeiksi motoriksiksi liikkeiksi. Vaikka osallistujat soittivat inkongruentit melodiat paremmin kuin he lauloivat ne, silmänliikkeiden tarkastelu osoitti että laulaminen saattoi silti olla osallistujille kognitiivisesti vähemmän vaativaa kuin soittaminen. Näitä havaintoja pohditaan väitöskirjatutkimuksessa esitetyn ennustusmallin sekä asiantuntijuusteorioiden valossa