Using a fine-grained comparative evaluation technique to understand and design software visualization tools

Software Visualization can be defined as the use of graphical and textual formalisms to describe the execution of computer programs. A large amount of Software Visualization technology has been developed to support computer science education, using a range of interface techniques. Far less effort has been devoted to evaluating the technology. As a result, it is unclear how effective Software Visualization tools are, either for students or professional programmers. Even more worrying, it is doubtful whether lessons are being learnt in successive designs of Software Visualization tools, or whether the application of new technologies (e.g. 3D animation and the internet) has become the primary goal, rather than the true goal of making computer programs easier to understand. To counter this problem the study reported here used protocol analysis to develop a fine-grained account of user behaviour, identifying (i) information access from the display, (ii) the use of comprehension strategies, and (iii) misunderstandings of the visualization and execution. The results were able to motivate future deigns which in turn could be compared and improved. The approach is compared to other evaluation techniques which aim to inform design. Finally, the generalizability of the approach is considered

Evaluation of the usability of constraint diagrams as a visual modelling language: theoretical and empirical investigations

This research evaluates the constraint diagrams (CD) notation, which is a formal representation for program specification that has some promise to be used by people who are not expert in software design. Multiple methods were adopted in order to provide triangulated evidence of the potential benefits of constraint diagrams compared with other notational systems. Three main approaches were adopted for this research. The first approach was a semantic and task analysis of the CD notation. This was conducted by the application of the Cognitive Dimensions framework, which was used to examine the relative strengths and weaknesses of constraint diagrams and conventional notations in terms of the perceptive facilitation or impediments of these different representations. From this systematic analysis, we found that CD cognitively reduced the cost of exploratory design, modification, incrementation, searching, and transcription activities with regard to the cognitive dimensions: consistency, visibility, abstraction, closeness of mapping, secondary notation, premature commitment, role-expressiveness, progressive evaluation, diffuseness, provisionality, hidden dependency, viscosity, hard mental operations, and error-proneness. The second approach was an empirical evaluation of the comprehension of CD compared to natural language (NL) with computer science students. This experiment took the form of a web-based competition in which 33 participants were given instructions and training on either CD or the equivalent NL specification expressions, and then after each example, they responded to three multiple-choice questions requiring the interpretation of expressions in their particular notation. Although the CD group spent more time on the training and had less confidence, they obtained comparable interpretation scores to the NL group and took less time to answer the questions, although they had no prior experience of CD notation. The third approach was an experiment on the construction of CD. 20 participants were given instructions and training on either CD or the equivalent NL specification expressions, and then after each example, they responded to three questions requiring the construction of expressions in their particular notation. We built an editor to allow the construction of the two notations, which automatically logged their interactions. In general, for constructing program specification, the CD group had more accurate answers, they had spent less time in training, and their returns to the training examples were fewer than those of the NL group. Overall it was found that CD is understandable, usable, intuitive, and expressive with unambiguous semantic notation

Cognitive Dimensions of PrologSpace

We evaluate a new visual programming system PrologSpace. The system addresses classic issues of visual interfaces: layout, shape, colour, and iconography; and those of particular concern to visual programming: multiple views (visual abstractions), synchronised views, integration of visual and textual dimensions, visual debugging, cognitive dimensions, and the problems of scale. We describe the system and examine its cognitive dimensions. We conclude that PrologSpace has a set of cognitive dimensions that seems to enhance Prolog programming and that the system provides potential for teaching Prolog. Keywords Visual programming, cognitive dimensions, Prolog. 1 Introduction PrologSpace is a visual programming system built on top of VisualProlog (a version of Prolog that provides support for X windows, widget creation and management, 3D graphics and animation, and audio [5]). PrologSpace provides all the power of VisualProlog to the application developer. We have chosen to evaluate Prolo..

Synchronised Multiple Views of PrologSpace

We introduce a new visual programming system PrologSpace. The system addresses classic issues of visual interfaces: layout, shape, colour, and iconography; and those of particular concern to visual programming: multiple views (visual abstractions), synchronised views, integration of visual and textual dimensions, visual debugging, cognitive dimensions, and the problems of scale. We briefly describe the system and how it provides synchronised multiple views. KEYWORDS Visual programming, multiple views, Prolog. INTRODUCTION PrologSpace is an extension of a Prolog language implementation ([Hut90]) and a visual environment for application programmers. We have added to Prolog's standard system commands to provide: ffl support for X windows, ffl widget creation and management ([Fou91]), ffl 3-D graphics and animation ([Gra94]), ffl audio. These extra language features are made available to the programmer in the conventional textual program development environment of Prolog (which usua..