Designing Gaze-Aware Attention Feedback for Learning in Mixed Reality

Mixed Reality (MR) has demonstrated its potential in the application field of education. In particular, in contrast to traditional learning, students using MR get the possibility of learning and exploring the content in a self-directed way. Meanwhile, research in learning technology has revealed the significance of supporting learning activities with feedback. Since such feedback is often missing in MR-based learning environments, we propose a solution of using eye-tracking in MR to provide gaze-aware attention feedback to students and evaluate it with potential users in a preliminary user study

Syntax highlighting as an influencing factor when reading and comprehending source code

Syntax highlighting or syntax colouring, plays a vital role in programming development environments by colour-coding various code elements differently. The supposition is that this syntax highlighting assists programmers when reading and analysing code. However, academic text books are largely only available in black-and-white which could influence the comprehension of novice and beginner programmers. This study investigated whether student programmers experience more difficulty in reading and comprehending source code when it is presented without syntax highlighting. Number of fixations, fixation durations and regressions were all higher for black-and-white code than for colour code but not significantly so. Subjectively students indicated that the colour code snippets were easier to read and more aesthetically pleasing. Based on the analysis it could be concluded that students do not experience significantly more difficulty when reading code in black-and-white as printed in text books

Using Eye-Tracking Data to Compare Differences in Code Comprehension and Code Perceptions between Expert and Novice Programmers

Previous research has examined how eye-tracking metrics can serve as a proxy for directly measuring the amount of cognitive effort and processing required for comprehending computer code. We conducted a pilot study comprising expert (n = 10) and novice (n = 10) computer programmers to examine group differences in code comprehension abilities and perceptions. Programmers were asked to read two pieces of computer code, rate the code on various attributes, and then describe what the code does. Results indicate that experts and novices significantly differ in terms of their fixation counts made during the task, such that experts had more fixations than novices. This was counter to our hypothesis that experts would have fewer fixations than novices. We found no evidence that experts and novices differed in their average fixation durations, trustworthiness and performance perceptions, or willingness to reuse the code

An Empirical Validation of Cognitive Complexity as a Measure of Source Code Understandability

Background: Developers spend a lot of their time on understanding source code. Static code analysis tools can draw attention to code that is difficult for developers to understand. However, most of the findings are based on non-validated metrics, which can lead to confusion and code, that is hard to understand, not being identified. Aims: In this work, we validate a metric called Cognitive Complexity which was explicitly designed to measure code understandability and which is already widely used due to its integration in well-known static code analysis tools. Method: We conducted a systematic literature search to obtain data sets from studies which measured code understandability. This way we obtained about 24,000 understandability evaluations of 427 code snippets. We calculated the correlations of these measurements with the corresponding metric values and statistically summarized the correlation coefficients through a meta-analysis. Results: Cognitive Complexity positively correlates with comprehension time and subjective ratings of understandability. The metric showed mixed results for the correlation with the correctness of comprehension tasks and with physiological measures. Conclusions: It is the first validated and solely code-based metric which is able to reflect at least some aspects of code understandability. Moreover, due to its methodology, this work shows that code understanding is currently measured in many different ways, which we also do not know how they are related. This makes it difficult to compare the results of individual studies as well as to develop a metric that measures code understanding in all its facets.Comment: 12 pages. To be published at ESEM '20: ACM / IEEE International Symposium on Empirical Software Engineering and Measuremen

On the Relationship between Code Verifiability and Understandability

Proponents of software verification have argued that simpler code is easier to verify: that is, that verification tools issue fewer false positives and require less human intervention when analyzing simpler code. We empirically validate this assumption by comparing the number of warnings produced by four state-of-the-art verification tools on 211 snippets of Java code with 20 metrics of code comprehensibility from human subjects in six prior studies. Our experiments, based on a statistical (meta-)analysis, show that, in aggregate, there is a small correlation (r = 0.23) between understandability and verifiability. The results support the claim that easy-to-verify code is often easier to understand than code that requires more effort to verify. Our work has implications for the users and designers of verification tools and for future attempts to automatically measure code comprehensibility: verification tools may have ancillary benefits to understandability, and measuring understandability may require reasoning about semantic, not just syntactic, code properties.Comment: to appear at Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE'23

Understanding Eye Gaze Patterns in Code Comprehension

Program comprehension is a sub-field of software engineering that seeks to understand how developers understand programs. Comprehension acts as a starting point for many software engineering tasks such as bug fixing, refactoring, and feature creation. The dissertation presents a series of empirical studies to understand how developers comprehend software in realistic settings. The unique aspect of this work is the use of eye tracking equipment to gather fine-grained detailed information of what developers look at in software artifacts while they perform realistic tasks in an environment familiar to them, namely a context including both the Integrated Development Environment (Eclipse or Visual Studio) and a web browser (Google Chrome). The iTrace eye tracking infrastructure is used for certain eye tracking studies on large code files as it is able to handle page scrolling and context switching. The first study is a classroom-based study on how students actively trained in the classroom understand grouped units of C++ code. Results indicate students made many transitions between lines that were closer together, and were attracted the most to if statements and to a lesser extent assignment code. The second study seeks to understand how developers use Stack Overflow page elements to build summaries of open source project code. Results indicate participants focused more heavily on question and answer text, and the embedded code, more than they did the title, question tags, or votes. The third study presents a larger code summarization study using different information contexts: Stack Overflow, bug repositories and source code. Results show participants tended to visit up to two codebase files in either the combined or isolated codebase session, but visit more bug report pages, and spend longer time on new Stack Overflow pages they visited, when given either these two treatments in isolation. In the combined session, time spent on the one or two codebase files they viewed dominated the session time. Information learned from tracking developers\u27 gaze in these studies can form foundations for developer behavior models, which we hope can later inform recommendations for actions one might take to achieve workflow goals in these settings. Advisor: Bonita Shari

Introducing Constraints into Web Layouts: Evaluating the Intuitiveness of Current Approaches for Designers

When it comes to web applications and their dynamic content, one seemingly common trouble area is that of layouts. Frequently, web designers resort to frameworks or JavaScript-based solutions to achieve various layouts where the capabilities of Cascading Style Sheets (CSS) fall short. Although the World Wide Web Consortium (W3C) is attempting to address the demand for more robust and concise layout solutions to handle dynamic content with the recent and upcoming specifications, a generic approach to creating layouts using constraint syntax has been proposed and implementations have been created. Yet, the introduction of constraint syntax would change the CSS paradigm in a fundamental way, demanding further analysis to determine the viability of its inclusion in core web standards. This thesis focuses on one particular aspect of the introduction of constraint syntax: how intuitive constraint syntax will be for designers. To this end, an experiment is performed involving participants thinking aloud while reading code snippets. Also, cursor movements are recorded as a proxy for eye movement over the code snippets. The results indicate that, upon first-impression, constraint syntax within CSS is not intuitive for designers