Towards Diagram Understanding: A Pilot Study Measuring Cognitive Workload Through Eye-Tracking

We investigate model understanding, in particular , how the quality of the UML diagram layout impacts cognitive load. We hypothesize that this w ill have a significant impact on the structure and effectiveness of engineers’ communication. In previous work, we have studied task performance measurements and subjective assessments; here, we also investigate behavioral indicators such as fixation and pupillary dilation. We use such indicators to explore diagram understanding- and reading strategies and how such strategies are impacted, e.g. by diagram type and expertise level. In the pilot eye-tracking experiment run so far, we have only examined a small number of participants (n=4), so our results are preliminary in nature and do not afford far reaching conclusions. They do, however, corroborate findings from earlier experiments, for example, showing that layout quality indeed matters and improves understanding. Our results also give rise to a number of new hypotheses about diagram understanding strategies that we are investigating in an ongoing data acquisition campaign

A Decade and More of UML: An Overview of UML Semantic and Structural Issues and UML Field Use

More than 10 years ago in 1997, three modeling advocates brought together their own distinct techniques to forge UML (Unified Modeling Language), and the world of modeling was forever changed (Booch, Rumbaugh, & Jacobson, 1999, 2005). The Object Management Group (OMG) immediately adopted the new language as the standard for their newly expanded object-oriented (OO) modeling scope (OMG, 2008), and the stage seemed set for a modeling explosion with UML leading the way into a brave new world of more accurate and better performing systems

Különböző kódhiba-keresési megközelítések hatékonyságának vizsgálata szemkövető rendszerrel

A tanulmány a szemmozgáskövető rendszerek alkalmazhatóságát vizsgálja egy programozási feladat tekintetében, amely során egy helytelenül működő algoritmus hibáinak feltárása és kijavítása alatt megfigyelésre, rögzítésre, valamint kiértékelésre kerültek a szemmozgás paraméterek. A vizsgálat során néhány, a tesztalanyokra jellemző paraméter alkalmazásával, két csoport került kialakításra, ahol az első csoport hibakeresés során inkább az apróbb módosításokat és a gyakoribb fordítások és futtatások technikáját alkalmazta, amíg a második csoport tagjai nagyobb hangsúlyt fektettek az értelmezésre. A kiértékelés során a két csoport szemmozgás követésére jellemző paraméterek, valamint ezen csoportok hatékonysági mutatói kerültek elemzésre statisztikai próbák alkalmazásával. Az eredmények azt mutatták, hogy a kevesebb kapkodás, az alaposabb megfontoltság és figyelem esetében kevesebb információmennyiség feldolgozása is elegendő a hatékonyabb hibakereséshez. | The study examines the suitability of eye movement tracking systems for a programming task, during which the parameters of eye movement were monitored, recorded and evaluated in order to discover and correct the errors of a malfunctioning algorithm. During the investigation, based on some parameters typical of the test subjects, two groups were formed. During the debugging of the first group, smaller modifications and the technique of more frequent translations and runs were used, while the members of the second group placed more emphasis on interpretation. During the evaluation, the eye movement tracking parameters of the two groups, as well as the efficiency ndicators of these groups, were analyzed using statistical tests. The results show that processing less amount of information is sufficient for fault-finding in the case of less haste, more thorough consideration and attention

Early childhood preservice teachers' debugging block-based programs: An eye tracking study

Learning computational skills such as programming and debugging is very important for K-12 students given the increasing need of workforce proficient in computing technologies. Programming is an intricate cognitive task that entails iteratively creating and revising programs to create an artifact. Central to programming is debugging, which consists of systematically identifying and fixing program errors. Given its central role, debugging should be explicitly taught to early childhood preservice teachers so they can support their future students’ learning to program and debug errors. In this study, we propose using eye-tracking data and cued retrospective reporting to assess preservice teachers’ cognitive strategies while debugging. Several eye-tracking studies have investigated learners’ debugging strategies though the literature lacks studies (a) conducted with early childhood preservice teachers and (b) that focus on block-based programming languages, such as Scratch. The present study addresses this gap in the literature. This study used mixed methods to triangulate quantitative findings from eye movement analysis and qualitative findings about employed debugging strategies into the creation of descriptive themes. Results showed that participants developed strategies such as simultaneous review of output and code, use of beacons to narrow down the area to be debugged, and eye fixation on output to form hypotheses. But most often, debugging was not informed by a hypothesis, which led to trial and error. Study limitations and directions for future research are discussed.&nbsp

Near real-time comprehension classification with artificial neural networks: decoding e-Learner non-verbal behaviour

Comprehension is an important cognitive state for learning. Human tutors recognise comprehension and non-comprehension states by interpreting learner non-verbal behaviour (NVB). Experienced tutors adapt pedagogy, materials and instruction to provide additional learning scaffold in the context of perceived learner comprehension. Near real-time assessment for e-learner comprehension of on-screen information could provide a powerful tool for both adaptation within intelligent e-learning platforms and appraisal of tutorial content for learning analytics. However, literature suggests that no existing method for automatic classification of learner comprehension by analysis of NVB can provide a practical solution in an e-learning, on-screen, context. This paper presents design, development and evaluation of COMPASS, a novel near real-time comprehension classification system for use in detecting learner comprehension of on-screen information during e-learning activities. COMPASS uses a novel descriptive analysis of learner behaviour, image processing techniques and artificial neural networks to model and classify authentic comprehension indicative non-verbal behaviour. This paper presents a study in which 44 undergraduate students answered on-screen multiple choice questions relating to computer programming. Using a front-facing USB web camera the behaviour of the learner is recorded during reading and appraisal of on-screen information. The resultant dataset of non-verbal behaviour and question-answer scores has been used to train artificial neural network (ANN) to classify comprehension and non-comprehension states in near real-time. The trained comprehension classifier achieved normalised classification accuracy of 75.8%

Eye gaze and interaction contexts for change tasks – Observations and potential

The more we know about software developers’ detailed navigation behavior for change tasks, the better we are able to provide effective tool support. Currently, most empirical studies on developers performing change tasks are, however, limited to very small code snippets or limited by the granularity and detail of the data collected on developer’s navigation behavior. In our research, we extend this work by combining user interaction monitoring to gather interaction context – the code elements a developer selects and edits – with eye-tracking to gather more detailed and fine-granular gaze context-code elements a developer looked at. In a study with 12 professional and 10 student developers we gathered interaction and gaze contexts from participants working on three change tasks of an open source system. Based on an analysis of the data we found, amongst other results, that gaze context captures different aspects than interaction context and that developers only read small portions of code elements. We further explore the potential of the more detailed and fine-granular data by examining the use of the captured change task context to predict perceived task difficulty and to provide better and more fine-grained navigation recommendations. We discuss our findings and their implications for better tool support

Analyse, à l'aide d'oculomètres, de techniques de visualisation UML de patrons de conception pour la compréhension de programmes

Mémoire numérisé par la Division de la gestion de documents et des archives de l'Université de Montréal